Senior Data Engineer - Azure Data Lake & Security Tech

Senior Data Engineer | Azure | Security-Tech

Fully Remote

£80,000 - £100,000 + bonus

We're partnered with a fast-growing technology company building AI-powered security systems. Our data science team is expanding, and we're looking for a Senior Data Engineer to own the data foundation that powers our models.

This is a rare opportunity to shape infrastructure from the ground up, not inherit someone else's technical debt.

What you'll be doing

You'll design and own our Azure-based data lake architecture, build the automated pipelines that bring operational sensor data into it, and create the labelled datasets our data science team depends on for model training.

Concretely, that means:

• Architecting and implementing our data lake in ADLS Gen2, defining schemas, partitioning strategies, and governance from scratch
• Building ingestion pipelines for structured sensor data (radar tracks, RF events, platform classifications) across multiple operational sites
• Developing spatial-temporal matching pipelines that link detection events to ground truth, producing the labelled datasets that drive model improvement
• Replacing fragile manual workflows with reliable, logged, automated pipelines using Azure Event Hubs and Data Factory
• Leading a migration of data from SharePoint, NAS, and physical drives into Azure
• Setting the data quality framework, checks, monitoring, alerting, so the wider team can trust what they're working with
• Integrating the data lake with Azure ML to support annotation and model training workflows

You'll work closely with a Lead Data Scientist, Software Engineering team, and report to the Data Science Manager. There's a junior Data Engineer for you to mentor.

What we're looking for

• Production experience designing and building data pipelines and data lake infrastructure
• Strong Python for data engineering, transformation, pipeline development, schema validation
• Cloud data platform experience (Azure preferred: ADLS Gen2, Data Factory, Event Hubs, Blob Storage, but AWS or GCP equivalents are fine)
• Solid SQL and familiarity with columnar formats like Parquet and Delta Lake
• Experience with both batch and streaming processing
• Understanding of data governance: RBAC, access control, data lifecycle

Nice to have: experience with ASTERIX or other structured sensor/radar data formats; geospatial or time-series processing; MLflow or Azure ML integration.

If you're a data engineer who wants to build something consequential, infrastructure that directly enables AI systems operating in the real world, we'd love to hear from you.

Please apply below if interested.