Data Acquisition Lead (AI/ML Data Infrastructure)

reflection • United State

Visa Sponsorship

Apply Now

AI Summary

Lead a high-performing team building web-scale data ingestion systems for AI model training, overseeing crawling, extraction, and data lake pipelines while driving technical decisions and collaborating with research and partnerships teams.

Key Highlights

Leadership of a data ingestion engineering team responsible for web crawling, extraction, and data lake pipelines at multi-TB to PB scale

Hands-on technical ownership of distributed systems, orchestration, and storage architectures for AI model pre-training data

Strategic partnership with research, data quality, and partnerships teams to optimize data acquisition for model performance

Searching for IT & Network Engineering roles that provide visa sponsorship? Connect with international employers through IT & Network Engineering Jobs with Visa Sponsorship opportunities actively seeking talented professionals.

Key Responsibilities

Build, mentor, and grow a high-performing team of data ingestion engineers

Oversee end-to-end data ingestion stack including web crawling, extraction pipelines, and data lakes

Make technical and architectural decisions for scalability, reliability, auditability, and cost efficiency

Collaborate with research teams to evaluate crawling strategies and ingestion tradeoffs for model impact

Partner with data partnerships, digitization operations, and external vendors for legal/licensing compliance

Prioritize and execute concurrent data acquisition campaigns in a fast-paced environment

Technical Skills Required

Web-scale data acquisition and ingestion systems Distributed computing (Ray, Beam, Spark) Data lake architectures and storage formats (Parquet, JSONL, WARC)

Explore our comprehensive directory of visa sponsorship jobs from employers worldwide who are ready to sponsor talented international professionals.

Benefits & Perks

Top-tier compensation with salary and equity

Stock options for all contributing employees

Comprehensive health & wellness benefits (medical, dental, vision, life, wellness allowance)

Job Description

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.Reflection's Data team builds the training corpora our frontier models learn from. Before a model can learn anything, the data has to be found, fetched, extracted, and delivered reliably, responsibly, and at enormous scale. The ingestion layer is the machinery that turns the open web, licensed corpora, and other large-scale sources into well-structured, versioned, auditable datasets for pre-training.

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.