Senior Software Engineer, AI Compute Infrastructure (GPU & Network Virtualization)

Together AI • Netherlands

Remote

Apply Now

AI Summary

Lead the design and development of high-performance GPU and network virtualization for Together AI’s AI-native cloud platform, scaling distributed AI workloads across global data centers. Own end-to-end projects from architecture to production, focusing on fault tolerance, automation, and performance optimization. Collaborate cross-functionally to build scalable infrastructure for inference, fine-tuning, and reinforcement learning.

Key Highlights

Build and own GPU/InfiniBand virtualization stack, including hypervisors, kernels, and SDN libraries for high-performance, isolated compute.

Scale distributed GPU scheduling and global management planes for on-demand/reserved clusters across hundreds of thousands of GPUs.

Harden monitoring, automated remediation, and fault-tolerant control planes for large-scale AI workloads.

Key Responsibilities

Design and implement GPU/InfiniBand virtualization (hypervisors, kernels, SDN) for high-performance, portable, and isolated compute.

Build and maintain in-DC IaaS layer (Kubernetes operators, VMs, parallel filesystems, VPCs, InfiniBand partitions).

Scale distributed GPU scheduling and global management planes for on-demand/reserved clusters across global data centers.

Harden monitoring and automated remediation for fault-tolerant distributed AI workloads (pretraining/inference).

Own end-to-end projects: write design docs, break work into milestones, and improve production reliability.

Mentor junior engineers, conduct code reviews, and provide design feedback.

Develop tooling (testing frameworks, developer tools, documentation) for cross-team robustness.

Interested in remote work opportunities in Devops? Discover Devops Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.

Technical Skills Required

Backend Programming (Golang preferred) Kubernetes (operators, device/network/storage plugins, custom schedulers) Distributed Systems & High-Performance Computing

Benefits & Perks

Equal Opportunity Employer (diversity-focused)

Fully remote work

Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.

Nice to Have

Kubernetes internals (operators, device plugins, custom schedulers)

VM/hypervisor expertise (QEMU/KVM, cloud-hypervisor, VFIO, PCIe passthrough)

DC networking (VLAN, VXLAN, OVS/OVN, VPC, InfiniBand, DPUs/SmartNICs)

Cluster API or similar infrastructure-as-code tools

GPU programming (CUDA, NCCL), high-performance compute/networking/storage

Job Description

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining the fastest LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the [Together GPU Clusters](https://www.together.ai/gpu-clusters) flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products.

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.