Senior Data Infrastructure Engineer (ML/Data Systems)

crosstalent • Gibraltar

Remote Visa Sponsorship

Apply Now

AI Summary

Collaborate with researchers and engineers to ensure high-quality, scalable data pipelines for AI model training. Design data quality systems, build distributed processing workflows, and optimize compute/storage efficiency. Bridge research and production by automating tooling and ensuring reproducibility in large-scale ML environments.

Key Highlights

Build and maintain petabyte-scale data infrastructure for AI model training

Define data quality metrics, validation checks, and automated assessment systems

Develop CI/CD tooling and distributed processing pipelines (Spark, Ray, Airflow)

Key Responsibilities

Work with model researchers to define data quality standards, metrics, and acceptance thresholds

Explore and curate open-source/internal datasets for training fundamental World Models

Design algorithms for automated data quality assessment, domain mixture detection, and synthetic-to-real data adaptation

Track dataset provenance, metadata, and versions to ensure experiment reproducibility

Develop and maintain CI/CD pipelines for data infrastructure tooling (GitHub, Python, PyTorch)

Optimize compute, storage, and throughput across large-scale data pipelines

Searching for Development & Programming roles that provide visa sponsorship? Connect with international employers through Development & Programming Jobs with Visa Sponsorship opportunities actively seeking talented professionals.

Technical Skills Required

Python Distributed Processing (Spark, Ray, Airflow) Machine Learning Fundamentals

Benefits & Perks

5 weeks of paid leave

Comprehensive healthcare benefits (including vision and dental)

Visa support (H1B and OPT transfers)

Explore our comprehensive directory of visa sponsorship jobs from employers worldwide who are ready to sponsor talented international professionals.

Nice to Have

Experience with large video datasets

Dataset curation for training

Internal tooling for ML evaluation/analysis

Job Description

In this role, you’ll work closely with model researchers, data infrastructure engineers, and cross-functional partners to make sure our data is high quality and can be produced at petabyte scale in a reliable, efficient way. From understanding how data choices show up in model behavior, to building processing pipelines and running the compute behind them, you’ll help ensure our models are trained on the best data we can get. What you’ll do

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.