T

Data and Evaluation Systems Engineer

Visa Sponsorship
Apply Now

Build data and evaluation systems that shape how advanced AI models learn and improve. Design data slices, evaluation rubrics, reward signals, and data pipelines across finance, code, and enterprise workflows. Requires 1-4 years of software engineering experience with strong Python and hands-on LLM evaluation or RLHF experience.

Key Highlights
Build data and evaluation systems for advanced AI models
Design data slices, rubrics, and reward signals for RLHF/RLVR
Create and manage real-world and synthetic data pipelines
1-4 years software engineering experience required
On-site in San Francisco, California
Key Responsibilities
Design data slices across finance, code, and enterprise workflows to reveal meaningful model failure modes
Build and refine evaluation rubrics and reward signals for RLHF and RLVR pipelines
Model annotator behavior and run experiments to improve model capabilities
Develop quantitative approaches to measure dataset quality, diversity, and impact on model alignment and capabilities
Create and manage real-world and synthetic data pipelines
Collaborate with research teams to translate training objectives into concrete data and evaluation specifications
Technical Skills Required
Python Machine Learning Data Engineering
Benefits & Perks
Visa sponsorship
Nice to Have
Work in AI safety, benchmarking, model evaluation, or open-source evaluation projects

Job Description

About the Role

Build data and evaluation systems that shape how advanced AI models learn and improve. You will work with research teams to turn training objectives into practical datasets, experiments, and evaluation specifications.

What You'll Do

Want the full job description? Read the complete details on Ashby, the original posting.
Continue on Ashby

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Keep exploring

Similar Jobs

Explore other opportunities that match your interests

Visas, Work Permits & Legal Insights

Stay informed about visa types, work permits, and immigration updates for international professionals.

Browse all articles

Subscribe to our newsletter

New visa sponsorship, relocation and remote tech jobs, delivered to your inbox.