I

RL Infrastructure Engineer

Visa Sponsorship
Apply Now

Join the Institute of Foundation Models as an RL Infrastructure Engineer to extend and scale end-to-end reinforcement learning training systems. You will integrate distributed training frameworks with rollout generation engines, build multi-node launch systems, and ensure training reliability at scale. Requires 5+ years in ML systems with hands-on experience developing LLM RL pipelines and strong Python engineering fundamentals.

Key Highlights
Extend and modify distributed training frameworks (DeepSpeed, FSDP, FairScale, Horovod) for RL workloads
Integrate rollout generation, reward computation, trajectory processing, and policy updates across multi-node clusters
Build robust config, launch, metrics, and monitoring systems for ML infrastructure
Requires 5+ years ML systems experience with hands-on LLM RL pipeline development
Visa sponsorship available
Key Responsibilities
Extend or modify distributed training frameworks such as DeepSpeed and FSDP to support new use cases and architectures
Connect training workers with rollout generation engines including trajectory exchange and policy-weight synchronization
Create and debug multi-node launch scripts with flexible batch sizes, parallelism strategies, and hardware targets
Build systems for experiment tracking, job monitoring, and logging usable by collaborators and researchers
Write production-quality code and tests for ML infrastructure in PyTorch or JAX ensuring reliability and maintainability at scale
Implement reward and verifier integration and trajectory processing, validating log probabilities, token masks, and loss inputs with researchers
Coordinate rollout and training workers including checkpoint/restart and failure recovery, tracking policy versions and sample staleness
Technical Skills Required
Distributed Training Python Multi-node Systems
Benefits & Perks
Comprehensive medical, dental, and vision benefits
Bonus
401K Plan
Generous paid time off, sick leave and holidays
Paid Parental Leave
Employee Assistance Program
Life insurance and disability
Nice to Have
Exposure to mixed-precision training with accuracy validation
Familiarity with performance profiling, kernel fusion, or memory optimization
Open-source contributions or published research in MLSys, ICML, or NeurIPS
CUDA or Triton kernel experience
Experience with large-scale pre-training
Experience building custom training pipelines at scale and modifying them for custom needs
Deep familiarity with training infrastructure and performance tuning
Experience with asynchronous RL, multi-turn or tool-using rollout environments, or custom reward/verifier systems

Job Description

We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy.You will participate in the development of groundbreaking AI solutions that have the potential to reshape entire industries. Strategic and innovative problem-solving skills will be instrumental in establishing MBZUAI as a global hub for high-performance computing in deep learning, driving impactful discoveries that inspire the next generation of AI pioneers.
Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources

Similar Jobs

Explore other opportunities that match your interests

Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Not Applicable

wayve

United State
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Not Applicable

wayve

United State

Senior Distributed ML Infrastructure Engineer

Machine Learning
•
1h ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Not Applicable

institute of foundation models

United State

Subscribe our newsletter

New Things Will Always Update Regularly