P

ML Inference Engineer

Prime Intellect • United State
Remote Visa Sponsorship Relocation
Apply Now
AI Summary

Build and optimize high-performance inference systems for RL research scaling to 1–3T parameter models. Responsibilities include improving rollout throughput, optimizing MoE inference, and reducing weight-update overhead. Requires deep expertise in GPU inference internals, Python, and PyTorch.

Key Highlights
Scaling inference systems for frontier-scale models (1-3T parameters) on NVL72 clusters
Optimizing rollout throughput for long-horizon agents and multi-node MoE inference
Reducing trainer-to-inference weight-update overhead through sharding-aware transfer

Looking to advance your Development & Programming career with relocation support? Explore Development & Programming Jobs with Relocation Packages that include comprehensive packages to help you move and settle in your new role.

Key Responsibilities
Extend prime-rl’s vLLM integration to bring new model architectures into RL training.
Improve rollout throughput for long-horizon agents by tuning concurrency and KV-cache reuse.
Optimize multi-node MoE inference, including expert parallelism and prefill/decode separation.
Reduce trainer-to-inference weight-update overhead through sharding-aware transfer and communication overlap.
Build on FP8 and evaluate lower precision for inference.
Use trainer/sampler mismatch checks to catch numerical regressions.
Collaborate with model researchers on architecture choices to improve inference efficiency.
Technical Skills Required
Python PyTorch GPU Inference

Discover our full range of relocation jobs with comprehensive support packages to help you relocate and settle in your new location.

Benefits & Perks
Cash Compensation Range of $150-350k
Equity incentives
Flexible work arrangements
Visa sponsorship
Relocation assistance
Quarterly team off-sites, hackathons, conferences and learning opportunities

Interested in relocating to United State? Check out our comprehensive Relocation Jobs in United State page with detailed relocation packages and benefits.

Nice to Have
Contributions to vLLM or another inference engine
CUDA or Triton kernel work, particularly for low-precision inference

Job Description

Prime Intellect is building the open superintelligence stack: the infrastructure frontier AI labs build internally, made available to every ambitious AI team.
Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

Apply Now
See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Job Overview
Posted Date Oct 16, 2026
Employment Type Full-time
Experience Level Not Applicable
Location United State
Category Programming
Company Prime Intellect
Mentioned Skills
python ai cloudflare san datadog pytorch

Similar Jobs

Explore other opportunities that match your interests

Remote Senior or Staff Software Engineer - Ruby, Rails, and Agentic AI
Programming
•
15m ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Not Applicable

CyberCoders

United State

Senior Full-stack Software Engineer

Programming
•
15m ago

Premium Job

Sign up is free! Login or Sign up to view full details.

•••••• •••••• ••••••
Job Type ••••••
Experience Level ••••••

Capital One

United State

Forward Deployed Strategist, AI
Programming
•
24m ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Not Applicable

Big Wave Digital

United State

Subscribe our newsletter

New Things Will Always Update Regularly