N

Senior Distributed Systems Engineer (AI Inference Scheduling & Routing)

United States
Visa Sponsorship
This Job is No Longer Active This position is no longer accepting applications
AI Summary

Design and build the core scheduling and routing layer for NeuroSpark’s high-performance AI inference platform, optimizing request admission, prioritization, and GPU resource allocation across multi-cloud environments. Own end-to-end systems from architecture to production, balancing latency, utilization, and multi-tenant fairness. Requires deep expertise in distributed systems, scheduling theory, and performance-critical languages like Go/Rust.

Key Highlights
Own the foundational scheduling and routing layer for a multi-cloud GPU inference platform, directly impacting latency and cost efficiency
Balance tail latency reduction with fleet utilization under real-world multi-tenant load and SLO commitments
End-to-end ownership: design, implement, deploy, and debug systems in production with architectural decision-making authority

Searching for Development & Programming roles that provide visa sponsorship? Connect with international employers through Development & Programming Jobs with Visa Sponsorship opportunities actively seeking talented professionals.

Key Responsibilities
Design and implement request admission, prioritization, batching, and placement systems for heterogeneous GPU fleets across multi-cloud environments
Apply queueing theory, control theory, and load-shedding principles to ensure predictable performance under bursty multi-tenant traffic
Engineer for concurrent optimization of tail latency and GPU utilization, resolving trade-offs in real-time systems
Build multi-tenant fairness and isolation mechanisms to guarantee SLOs during fleet saturation
Instrument, observe, and debug distributed behavior in production, owning rollout and post-deployment realities
Set technical standards and document foundational architecture decisions for the team
Technical Skills Required
Distributed Systems Scheduling & Load Balancing Performance Optimization (Go/Rust/C++)

Explore our comprehensive directory of visa sponsorship jobs from employers worldwide who are ready to sponsor talented international professionals.

Benefits & Perks
Equity compensation
Medical, dental, and vision coverage
H-1B and work visa sponsorship

Interested in opportunities specifically in United State? Discover our dedicated Visa Sponsorship Jobs in United State page featuring roles from top employers in this location.

Nice to Have
Mandarin proficiency
Kubernetes and multi-cloud operations experience
Open-source contributions to inference/serving/scheduling projects
GPU cluster operations at scale

Job Description

NeuroSpark builds and operates a high-performance AI inference platform that helps enterprises run large language models faster, cheaper, and at scale. Inference infrastructure is the foundation the entire AI application layer runs on — every AI product ultimately depends on how fast, how reliably, and how affordably models can serve their users. Our vision is to make that layer so efficient that compute is never the reason a good AI product fails.

About the Role

Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Job Overview
Posted Aug 03, 2026
Employment Full-time
Experience Mid-Senior level
Location United States
Category Programming
Keep exploring

Similar Jobs

Explore other opportunities that match your interests

Visas, Work Permits & Legal Insights

Stay informed about visa types, work permits, and immigration updates for international professionals.

Browse all articles

Subscribe to our newsletter

New visa sponsorship, relocation and remote tech jobs, delivered to your inbox.