I

Site Reliability Engineer - Cluster & SRECluster

iframe • Canada
Visa Sponsorship
Apply
AI Summary

Own the physical reality of GPU clusters across seven regions, bringing up new racks and validating InfiniBand fabric end-to-end. Drive incident response, capacity planning, and global pager rotations while maintaining runbooks for consistent operations. Requires five-plus years operating large compute clusters with deep InfiniBand experience and strong Linux fundamentals.

Key Highlights
Hands-on hardware work with GPU racks and InfiniBand fabric
Regional incident response with 15-minute page and 4-hour resolution target
Global pager rotation and cross-region secondary coverage
Five-plus years operating large compute clusters required
Key Responsibilities
Bring up new B200/B300/MI300X racks including cabling, ToR configuration, NCCL/RCCL all-reduce validation, and MFU baseline tests
Drive InfiniBand fabric to spec (NDR/XDR) and chase residual bit-error budget to zero
Run capacity planning across seven regions including demand forecasting, power and thermal headroom modeling, and procurement coordination
Own regional incident response with 15-minute page target and 4-hour resolution target
Build and maintain bring-up runbooks for new hires
Carry global pager approximately one week per six weeks alongside runtime and customer engineering
Technical Skills Required
Linux InfiniBand Kubernetes Terraform
Benefits & Perks
Base salary $210,000 - $340,000 USD
Meaningful early-stage equity
On-site pay differential at Cologix regions outside SF/NYC/Bay Area
Nice to Have
DGX H100/H200/B200 bring-up history
Experience with NVIDIA Bright/Base Command Manager
Bare-metal provisioning systems (Tinkerbell, MAAS, Razor)
Procurement/vendor management experience

Job Description


Cluster & SRE

Cluster Site Reliability Engineer

Own the physical reality of the platform in one of our seven regions. You bring up new GPU racks, validate InfiniBand fabric end-to-end, and keep the cluster running at the SLA. This is a hands-on role; you will see the hardware.

Apply by email All open roles

Team Cluster & SRE

Location On-site

  • Cologix region of choice (Toronto, Montréal, Columbus, Vancouver, Ashburn)

Type Full-time

  • IC4–IC6

Stack LinuxInfiniBand (NDR / XDR)NCCL / RCCLKubernetes (host-level)TerraformPrometheus / GrafanaGo or Python

Apply by email

Hiring manager replies within 5 business days.

The team

About The Team

Cluster SRE is six engineers across the seven regions. Each region has a primary and a secondary; you will be one of those for your region. The team coordinates daily, deploys weekly, and rotates a global pager.

Reports to the head of cluster engineering. Primary on a single region; rotates secondary for one neighboring region.

The role

What you'll do

  • 01 Bring up new B200 / B300 / MI300X racks: cabling, ToR config, NCCL/RCCL all-reduce validation, MFU baseline tests.
  • 02 Drive InfiniBand fabric to spec — NDR / XDR depending on the rack — and chase residual bit-error budget down to zero.
  • 03 Run capacity planning across seven regions: forecast demand, model power and thermal headroom, work with procurement on lead times.
  • 04 Own the regional incident response. P1 incidents page within fifteen minutes; resolution target is four hours.
  • 05 Build and maintain the bring-up runbook so the second hire after you can do their first rack solo.
  • 06 Carry the global pager about one week per six, alongside runtime and customer engineering.

The bar

What we're looking for

  • Five-plus years operating large compute clusters — supercomputing centers, hyperscaler infra, or HPC at a national lab count.
  • Deep InfiniBand and Ethernet RoCE experience: subnet manager tuning, fabric debugging, lossless networking.
  • Strong Linux fundamentals: kernel parameters, NUMA topology, kernel cgroup limits.
  • Comfort writing Go or Python for tooling. We are not strict about which.
  • Calm under load. You will be the named person on a $50M-ARR account when something goes wrong.

Bonus

Nice to have, not required

  • DGX H100 / H200 / B200 bring-up history.
  • Experience with NVIDIA Bright / Base Command Manager.
  • Bare-metal provisioning systems: Tinkerbell, MAAS, Razor, or in-house equivalents.
  • Procurement / vendor management experience.

Compensation

In writing, like everything else

We publish bands. We meet them. The number you see on the offer is the same number your future peers got at the same level. We do not negotiate; we level.

Base

$210,000 – $340,000 USD (US Cologix regions) / equivalent in CA.

Equity

Meaningful early-stage equity, refreshed on tenure milestones.

Notes

On-site pay differential at Cologix regions outside SF / NYC / Bay Area is +5–10% to compensate for travel.

How to apply

One email is enough

Send a short note to [email protected] with the role title in the subject line. Include your CV or LinkedIn, one or two links to work you're proud of, and a sentence on why this role specifically. Hiring managers reply within five business days, regardless of outcome.

  • 01

Application

A hiring manager reads every email. Reply within five business days.

  • 02

Manager call

30–45 minutes. Scope, role, mutual fit. We share the comp band on this call.

  • 03

Technical loop

3–4 sessions on the same day. Real problems, no homework, no whiteboard riddles.

  • 04

Offer

Same-week offer at the published band for your level. Start dates are flexible.

Equal opportunity

We hire on the work. Race, gender, age, nationality, religion, sexual orientation, disability, and veteran status do not factor into our decisions. We sponsor visas for senior roles in the US, UK, and EU — bring it up on the manager call.

Need an accommodation for the interview process? Mention it in your application or write [email protected].

Also open

Other roles you might consider

  • Runtime

Inference Acceleration Engineer

Ship the next 2× on the open-source model catalog. Triton, CUDA, ROCm. You will publish what you ship.

View role

  • Research lab

Distributed Training Researcher

Lead a paper / quarter on multi-thousand-GPU pre-training. Co-appointment with a partner university available.

View role

  • Customer engineering

Customer Cluster Engineer

Embedded with a small portfolio of reserved-tier accounts. Distributed training perf, NCCL, kernel tuning.

View role

All open roles

One last thing

If this role isn't quite right but you'd be a fit at iframe.ai, write anyway.

Senior engineers and researchers can apply outside the listed roles. The bar is the same. The reply window is the same.

Apply: Cluster Site Reliability Engineer General application

Similar Jobs

Explore other opportunities that match your interests

Software Developer - Reman Business Unit

Networking
•
2w ago

Premium Job

Sign up is free! Login or Sign up to view full details.

•••••• •••••• ••••••
Job Type ••••••
Experience Level ••••••

Caterpillar Inc.

Canada

HR Specialist (m/f/d)

Networking
•
5h ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Entry level

fox ess

United Kingdom

Junior IT Support

Networking
•
18h ago
Visa Sponsorship Relocation Remote
Job Type Part-time
Experience Level Entry level

inspire early learning journey

Australia

Subscribe our newsletter

New Things Will Always Update Regularly