T

Senior Site Reliability Engineer (SRE) for Model Post-Training Infrastructure

United States
Visa Sponsorship Relocation
This Job is No Longer Active This position is no longer accepting applications

Lead end-to-end reliability for Model Post-Training workloads (SFT, RL, DPO, distillation) by defining SLAs, observability, and incident response. Shape platform architecture proactively, mentor engineers, and ensure cross-team resilience for distributed ML systems at scale. Requires 7+ years in SRE, production ML operations, and technical authority to drive reliability standards.

Key Highlights
Define and own end-to-end reliability for post-training ML workloads across multiple distributed systems.
Proactively shape platform architecture to prevent production failures, not just react to incidents.
Senior individual-contributor role with technical authority to set reliability standards and lead cross-team incident response.
Key Responsibilities
Define and own end-to-end reliability for Model Post-Training, including CI/CD flows, observability, and incident response.
Develop and enforce Service Level Objectives (SLOs) for distributed training systems, balancing reliability and development velocity.
Design monitoring and observability tools to distinguish infrastructure failures from training/modeling issues.
Lead incident response for platform-wide issues, ensuring rapid recovery and systematic improvements.
Harden multi-tenant isolation and resource scheduling for LoRA-based workloads to maximize utilization without compromising reliability.
Set technical direction for reliability of new post-training methods, collaborating with research teams.
Build and harden checkpointing and recovery systems for long-running distributed training jobs.
Mentor engineers on production ML reliability practices and collaborate with security teams on vulnerabilities.
Technical Skills Required
Distributed Systems Site Reliability Engineering (SRE) Production Machine Learning Systems
Benefits & Perks
Generous health, dental, and vision benefits
Unlimited PTO
Paid parental leave
Nice to Have
Deep experience operating production cloud services at scale (e.g., AWS, GCP)
Working knowledge of post-training methods (SFT, RL, DPO, distillation)
Expertise in distributed training frameworks (PyTorch FSDP/DDP, Megatron, DeepSpeed)
Track record building checkpoint and recovery systems for long-running distributed jobs
Expertise in Kubernetes at scale for heterogeneous GPU workloads

Job Description

About the Role

We're looking for a Site Reliability Engineer (SRE) to drive reliability for Model Post Training end-to-end. Post-training spans several distinct workloads — supervised fine-tuning, reinforcement learning, preference optimization, and distillation — each with its own resource profile, failure signature, and tolerance for interruption or delay. You'll set the reliability strategy across all of them, not just react to what breaks.

Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Keep exploring

Similar Jobs

Explore other opportunities that match your interests

Career Resources & Guides

Latest guides related to this job: Visas, Work Permits & Legal Insights, Relocation Guides.

Browse all articles

Subscribe to our newsletter

New visa sponsorship, relocation and remote tech jobs, delivered to your inbox.