P

Research Engineer – Distributed AI Training Systems

United States
Full Remote Visa Sponsorship Relocation
This Job is No Longer Active This position is no longer accepting applications

Build scalable, fault-tolerant distributed training systems for Protocol Learning, advancing frontier AI models across heterogeneous hardware and low-bandwidth networks. Implement model-parallel, data, and tensor parallelism while optimizing performance and robustness. Requires hands-on distributed training expertise in PyTorch/DeepSpeed and strong Python engineering skills.

Key Highlights
Develop distributed AI training systems for Protocol Learning, scaling from 8B to frontier models
Optimize communication-efficient training across heterogeneous hardware and unreliable networks
Design fault-tolerant systems with elastic node participation and robust checkpointing
Key Responsibilities
Implement and optimize model-parallel training for large-scale AI models on heterogeneous GPUs
Develop communication-efficient techniques to reduce network overhead in distributed training
Design fault-tolerant systems to handle node churn, checkpointing, and state synchronization
Build monitoring instrumentation to track throughput, bottlenecks, and model quality across devices
Technical Skills Required
Distributed training systems (PyTorch, FSDP, DeepSpeed, Megatron) Model/tensor/data parallelism Python (production-grade)
Benefits & Perks
Equity-heavy compensation package
Remote-first culture with global flexibility
Visa sponsorship (Australia/US)
Nice to Have
Hands-on experience with large language models (e.g., Nemotron, Qwen)
P2P networking and NAT traversal expertise
Post-training and reinforcement learning experience
Experience with inference/serving systems

Job Description

Pluralis Research works on Protocol Learning: training and serving large models in a fully decentralized way on small consumer-grade devices connected via the internet. Despite being dismissed as infeasible, we have made significant advances on this problem, most recently Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs spread over the internet, with no single participant ever holding the full weights (tech report). While many of the core research problems have been solved, Protocol Learning unlocks a series of new challenges. For the mission in full, read A Third Path: Protocol Learning.
Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Keep exploring

Similar Jobs

Explore other opportunities that match your interests

Hidden company

Log in to see location

Premium

Control Systems Engineer

Log in free to see the details

Loop

United States

New

AI Full-Stack Engineer

  • React
  • Typescript
  • Elasticsearch
  • Node.js
Visa

Career Resources & Guides

Latest guides related to this job: Visas, Work Permits & Legal Insights, Relocation Guides, Remote Work Guides.

Browse all articles

Subscribe to our newsletter

New visa sponsorship, relocation and remote tech jobs, delivered to your inbox.