Senior Data Scientist / Data Flywheel Engineer (LLM Evaluation & Improvement)

reflection • United Kingdom

Visa Sponsorship

This Job is No Longer Active This position is no longer accepting applications

AI Summary

Lead the Data Flywheel team at Reflection to bridge benchmark and real-world model performance by designing evaluations, data pipelines, and feedback loops. Collaborate across research, engineering, and partnerships to measure, improve, and deploy LLMs. Requires deep expertise in LLM training, evaluation, and scalable ML systems with a hands-on, impact-driven mindset.

Key Highlights

Bridge the gap between benchmark and real-world LLM performance through data-driven improvements

Design evaluations, graders, and feedback loops to measure and enhance model capabilities

Collaborate with researchers, engineers, and external partners to scale data pipelines and training interventions

Searching for Development & Programming roles that provide visa sponsorship? Connect with international employers through Development & Programming Jobs with Visa Sponsorship opportunities actively seeking talented professionals.

Key Responsibilities

Identify high-value data sources and partnerships, translating real-world use cases into rigorous evaluations

Design and build automated evaluations, graders, and feedback loops to measure priority model behaviors

Analyze model performance and failure modes, then develop targeted datasets and training interventions

Develop human and synthetic data strategies for capabilities lacking sufficient existing data

Build scalable infrastructure and pipelines for data ingestion, validation, versioning, and evaluation

Collaborate with cross-functional teams to integrate new signals into measurable model improvements

Explore our comprehensive directory of visa sponsorship jobs from employers worldwide who are ready to sponsor talented international professionals.

Technical Skills Required

Large Language Model (LLM) Training and Evaluation Data Pipeline and Infrastructure Development Reinforcement Learning and Reward Design

Interested in opportunities specifically in United Kingdom? Discover our dedicated Visa Sponsorship Jobs in United Kingdom page featuring roles from top employers in this location.

Benefits & Perks

Top-tier compensation with salary and equity structured globally

Comprehensive health and wellness benefits (medical, dental, vision, life, wellness allowance)

22 weeks paid parental leave for all new parents, including adoptive and surrogate journeys

Job Description

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.Role OverviewThe Data Flywheel team closes the gap between benchmark performance and useful performance in the real world. We identify and build the signals, data, and feedback loops that turn model usage into rigorous evaluations, targeted training data, and measurable improvements in future generations of models.

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.