Databricks Data Engineer

• Location: London customer site – 1/2 days p/week on site
• Salary: Up to £105k DoE plus bonus
• Clearance: Active SC preferred - can sponsor if not currently held (British passport required)
• To apply, email: [email protected]

We’re working with a leading technology consultancy delivering secure, mission-critical digital capabilities across Secure Sector programmes. They’re looking for an experienced Databricks Data Engineer to design and build scalable, resilient data applications that support operational users, analytics platforms, and AI-enabled capabilities.

This is a hands-on engineering role sitting at the intersection of Software Engineering and Data Engineering, focused on building secure, production-grade data services that operate reliably within highly governed Defence environments.

What you’ll be doing:

• Building scalable data applications and pipelines using Python
• Developing streaming, event-driven and orchestrated data solutions
• Deploying and supporting containerised services in Kubernetes environments
• Implementing observability, monitoring and data governance best practices
• Leveraging Infrastructure as Code and CI/CD for secure deployments
• Owning solutions from design and development through to production support
• Delivering resilient data platforms within secure Defence environments

What we’re looking for:

• Strong Python development and software engineering experience
• Experience building scalable data pipelines and distributed data platforms
• Knowledge of Kafka, Spark and modern data processing technologies
• Experience with orchestration tools such as Airflow, Dagster or Prefect
• Hands-on experience with Docker, Kubernetes and containerised deployments
• CI/CD experience with a focus on production ownership and reliability
• Understanding of data quality, testing and lineage best practices

Desirable skills:

• MLOps or production machine learning experience
• Terraform, Ansible or other Infrastructure as Code tools
• PostgreSQL and modern data platforms
• Experience within Defence, National Security or regulated environments
• Familiarity with air-gapped, restricted or secure networks
• Knowledge of Google Distributed Cloud (GDC) or edge cloud technologies

Security Clearance:

Due to the nature of the work, candidates must either hold an active SC Clearance or be eligible to obtain – British passport holder.