C

AI SDET (Quality & AI-First Mindset)

Colombia
Full Remote
This Job is No Longer Active This position is no longer accepting applications
AI Summary

Pioneer and scale AI-driven testing practices to ensure reliable, safe, and high-performing AI capabilities. Contribute to reducing deployment risks, minimizing hallucinations and drift, and ensuring ethical AI. Drive faster releases and improve product trust, compliance, and innovation.

Key Highlights
Evolve a modern, AI-first quality strategy for a fast-scaling SaaS architecture
Design deterministic and statistical testing approaches for non-deterministic LLM-based and agentic systems
Implement red-teaming, bias/fairness checks, and compliance mechanisms
Key Responsibilities
Evolve a modern, AI-first quality strategy for our fast-scaling SaaS architecture, including foundational infrastructure and emerging agentic/intelligent systems.
Integrate AI enhancements into CI/CD pipelines (e.g., predictive flakiness detection, automated test generation, self-healing scripts) to improve isolation, data setup, and execution reliability using existing/suggesting tools.
Establish scalable testing practices that support hyper-growth and petabyte-scale AI data pipelines.
Design deterministic and statistical testing approaches for non-deterministic LLM-based and agentic systems, addressing hallucinations, prompt injection, bias, drift, and safety risks.
Build automated evaluation pipelines and harnesses for correctness, faithfulness, retrieval quality, generation accuracy, tool-calling, planning sequences, and multi-agent flows.
Execute/Develop test frameworks for the full AI lifecycle: prompts, datasets, embeddings, model versions, RAG pipelines (end-to-end validation), and guardrails.
Implement red-teaming, bias/fairness checks, and compliance mechanisms; leverage in trend frameworks for metrics and observability.
Integrate AI-specific quality signals into CI/CD for automated gating and continuous monitoring.
Partner closely with product, data science, AI engineering, and dev teams to test AI features, conduct multi-agent simulations, and ensure high-quality roadmap delivery.
Facilitate knowledge sharing and upskilling on AI testing best practices across the Quality Function.
Drive core metrics (DORA, test coverage/effectiveness) plus AI-specific indicators (e.g., hallucination rate, context precision, drift detection).
Build real-time dashboards and support A/B testing of models with post-deployment monitoring.
Champion a quality-first, ethical AI mindset organization-wide.
Mentor SDET’s, lead workshops on AI risks/validation, and influence design/deploy/incident processes.
As a foundational hire, define roadmaps and best practices for sustainable AI quality assurance.

Interested in remote work opportunities in QA & Testing? Discover QA & Testing Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.

Technical Skills Required
Automated testing infrastructure CI/CD (Jenkins/GitHub Actions) Test pyramid strategies (unit → E2E) Full-stack testing experience (frontend/backend/API) JS/TS Python Java AI evaluation frameworks (e.g., Ragas, DeepEval, LangChain/LangSmith/LangFuse) K6 JMeter Blazemeter NewRelic Statistical testing methods
Benefits & Perks
Contrato a termino Indefinido with all the legal benefits
Prepaid Medicine
Life insurance and funeral assistance
Internet allowance
Home office stipend
Competitive compensation — above the market average
100% remote work environment and an excellent work-life balance
Opportunity to work for a growing global SaaS leader company
A culture that promotes independence, innovation, trust, and accountability
Budget for training, we want you to grow
5 Personal Time Off days per year
Sick Leave Top up to total 100% of salary paid by the employer from Day 3 to 90
Recognition Award, additional paid time off in recognition of the corresponding year of service
Upgrade vacation starting at 5 years of service

Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.

Nice to Have
Knowledge of performance, Stress and Load testing tools like K6, JMeter, Blazemeter will be nice to have.
Knowledge of observability (NewRelic), statistical testing methods, red-teaming, and ethical AI practices.

Job Description

We are at the forefront of AI adoption in our cloud-native SaaS platform, building intelligent, agentic features that transform how users interact with our product. As an AI SDET, you'll pioneer and scale AI-driven testing practices from the ground up—fast-tracking reliable, safe, and high-performing AI capabilities across the organization. You will contribute in areas to reduce deployment risks, minimize hallucinations and drift, ensure ethical AI, and drive faster releases (targeting 20-40% velocity gains through automated validations). This is a high-impact, foundational role in Platform Engineering's Quality function, where your work will directly influence product trust, compliance, and innovation for our end users. 

Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Job Overview
Posted Feb 08, 2026
Employment Full-time
Experience Mid-Senior level
Location Colombia
Category Testing
Keep exploring

Similar Jobs

Explore other opportunities that match your interests

Remote Work Guides

Practical guides for finding and thriving in remote roles.

Browse all articles

Subscribe to our newsletter

New visa sponsorship, relocation and remote tech jobs, delivered to your inbox.