Senior QA Engineer - AI Benchmark Quality Assurance

hackajob • India

Remote

This Job is No Longer Active This position is no longer accepting applications

AI Summary

Join a leading AI lab's cutting-edge GenAI team as a Senior QA Engineer. Design test cases, review tasks, and debug complex systems to ensure airtight agentic evaluation benchmarks. Collaborate with researchers and task authors in a tight feedback loop. Full-time, remote within the US.

Key Highlights

Design checks for complex multi-step tasks

Review tasks and reference solutions for ambiguity and gaps

Debug tasks and checks using Python

Shape quality checklists and protect benchmark scores

Interested in remote work opportunities in QA & Testing? Discover QA & Testing Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.

Key Responsibilities

Design checks to confirm each task works as intended, including edge cases

Review tasks and reference solutions before finalization

Debug tasks and checks using Python

Shape simple, repeatable quality checklists

Protect benchmark scores by watching for shortcuts and grading gaps

Technical Skills Required

Python Quality Assurance Test Engineering

Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.

Benefits & Perks

Fully remote within the United States

Approximately 35 hours per week

Job Description

hackajob is collaborating with Mercor to connect them with exceptional professionals for this role.Join a leading AI lab's cutting-edge GenAI team to be at the core of the AI revolution, where your expertise fuels the development of the most advanced AI models.** ## 1\. Overview A leading AI lab is building the next generation of agentic evaluation benchmarks for frontier models, and complex multi-step tasks are only useful if they are airtight: unambiguous, correctly graded, and robust to shortcuts. We are seeking experienced QA and test engineers to be the quality backbone of this benchmark — designing the test cases and review processes that guarantee every task measures what it claims to...

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.