D

Principal AI Evaluation Engineer

Discovered MENA • Emea
Relocation
Apply Now

Design and implement AI evaluation frameworks for a large-scale AI platform in Abu Dhabi, defining standards for LLM, RAG, and agentic system reliability. Own evaluation architecture, golden datasets, regression detection, and CI/CD integration to ensure production-ready AI systems. Senior hands-on role shaping evaluation discipline across engineering teams.

Key Highlights
Define AI evaluation strategy and reference architecture for a next-gen AI platform in Abu Dhabi
Build shared evaluation frameworks, golden datasets, and automated grading systems for LLM, RAG, and agentic systems
Hands-on senior role with direct influence on production AI system approval and release standards
Key Responsibilities
Define the overall AI evaluation strategy and reference architecture across the engineering organization
Build shared evaluation frameworks, harnesses, and reusable libraries for LLM, RAG, and agentic systems
Develop and maintain golden datasets, dataset versioning strategies, and automated evaluation systems
Implement regression detection for non-deterministic AI behavior and establish production monitoring
Evaluate LLM output quality, hallucination, grounding, faithfulness, and citation accuracy
Design and calibrate LLM-as-a-judge systems against human evaluation standards
Integrate AI evaluation into CI/CD pipelines and release processes
Contribute to building the broader evaluation engineering function as the organization scales
Technical Skills Required
Python LLM and Generative AI Evaluation Regression Detection for Non-Deterministic Systems
Benefits & Perks
Tax-free UAE package
Comprehensive relocation support for candidates and families
Competitive compensation
Nice to Have
Experience with Ragas, DeepEval, Promptfoo, Braintrust, LangSmith, or Langfuse
LangChain or LangGraph experience
Vector databases (pgvector, Qdrant)
AI red teaming or adversarial testing
Arabic language AI evaluation expertise
Experience in regulated or highly secure environments

Job Description

Principal Evals Engineer

Location: Abu Dhabi, UAE (relocation support available)

About the opportunity

We are hiring a Principal Evals Engineer to join a major new AI programme in Abu Dhabi. The organization is building a large scale AI platform designed to support next generation AI systems across multiple sectors.The work spans AI assistants, retrieval systems, agentic workflows, voice applications and document intelligence, with a strong focus on building reliable AI systems that can operate at scale in real world environments.

Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources

Similar Jobs

Explore other opportunities that match your interests

Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Mid-Senior level

LevelUP HCS

Emea
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Mid-Senior level

coinspaid dev

Emea

Principal AI Evaluation Engineer

Programming
•
3w ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Mid-Senior level

Discovered MENA

Emea

Subscribe our newsletter

New Things Will Always Update Regularly