L

Senior Data Engineer (GCP, Databricks, BigQuery, Airflow, dbt)

United Kingdom
Full Remote
This Job is No Longer Active This position is no longer accepting applications
AI Summary

Leadtech Group seeks a Senior Data Engineer to design and optimize data infrastructure on Databricks and GCP. Responsibilities include architecting scalable ETL/ELT pipelines using BigQuery, Airflow, dbt, and ensuring data quality with Great Expectations. Requires 3+ years of data engineering experience, with a focus on cloud-native solutions and GCP stack.

Key Highlights
Design, develop, and optimize data infrastructure on Databricks and GCP.
Architect scalable ETL/ELT pipelines using BigQuery, Airflow, dbt, and ensure data quality.
Implement CI/CD best practices and event-driven architectures for microservices.
Key Responsibilities
Define and implement the overall data architecture on GCP, including data warehousing in BigQuery/Databricks, data lake patterns in Google Cloud Storage, and Data Mart (Data Mach) solutions.
Integrate Terraform for Infrastructure as Code to provision and manage cloud resources efficiently.
Establish both batch and real-time data processing frameworks to ensure reliability, scalability, and cost efficiency.
Design, build, and optimize ETL/ELT pipelines using Apache Airflow for workflow orchestration.
Implement dbt (Data Build Tool) transformations to maintain version-controlled data models in BigQuery, ensuring consistency and reliability across the data pipeline.
Use Google Dataflow (based on Apache Beam) and Pub/Sub for large-scale streaming/batch data processing and ingestion.
Automate job scheduling and data transformations to deliver timely insights for analytics, machine learning, and reporting.
Implement event-driven or asynchronous data workflows between microservices.
Employ Docker and Kubernetes (K8s) for containerization and orchestration, enabling flexible and efficient microservices-based data workflows.
Implement CI/CD pipelines for streamlined development, testing, and deployment of data engineering components.
Enforce data quality standards using Great Expectations or similar frameworks, defining and validating expectations for critical datasets.
Define and uphold metadata management, data lineage, and auditing standards to ensure trustworthy datasets.
Implement security best practices, including encryption at rest and in transit, Identity and Access Management (IAM), and compliance with GDPR or CCPA where applicable.
Collaborate with Data Science, Analytics, and Product teams to ensure the data infrastructure supports advanced analytics, including machine learning initiatives.
Maintain Data Mart (Data Mach) environments that cater to specific business domains, optimizing access and performance for key stakeholders.

Interested in remote work opportunities in Data Science? Discover Data Science Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.

Technical Skills Required
Databricks BigQuery Google Cloud Storage Apache Airflow dbt Dataflow Pub/Sub Great Expectations Terraform Docker Kubernetes Python Java Scala RabbitMQ Prometheus Grafana Datadog New Relic Kafka Spark Streaming Vertex AI
Benefits & Perks
Flexible career path
Personalized internal training
Annual budget for external learning opportunities
Flexible schedule with flextime
Option of working full remote or from Barcelona office
Free Friday afternoons
35-hour workweek in July and August
Competitive salary
Full-time permanent contract
Top-tier private health insurance (including dental and psychological services)
25 days of vacation plus your birthday off
Flexible vacation options
Free coffee, fresh fruit, snacks
Game room
Rooftop terrace
Ticket restaurant
Nursery vouchers

Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.

Nice to Have
Familiarity with end-to-end ML workflows and model deployment on GCP (e.g., Vertex AI)
Experience with Prometheus, Grafana, Datadog, or New Relic for system health and performance monitoring
Advanced knowledge of compliance frameworks such as HIPAA, SOC 2, or relevant regulations
Additional proficiency in Kafka, Spark Streaming, or other streaming solutions
GCP-specific certifications (e.g., Google Professional Data Engineer)

Job Description

We are looking for a to design, develop, and optimize our data infrastructure on . You will architect scalable pipelines using BigQuery, Google Cloud Storage, Apache Airflow, dbt, Dataflow, and Pub/Sub, ensuring high availability and performance across our ETL/ELT processes. You will leverage Great Expectations to enforce data quality standards. The role also involves building our Data Mart (Data Mach) environment and implementing CI/CD best practices.A successful candidate has extensive knowledge of cloud-native data solutions, strong proficiency with ETL/ELT frameworks (including dbt), and a passion for building robust, cost-effective pipelines.
Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Job Overview
Posted Feb 24, 2026
Employment Full-time
Experience Mid-Senior level
Location United Kingdom
Category Data Science
Keep exploring

Similar Jobs

Explore other opportunities that match your interests

Haystack

United Kingdom

Data Engineer

  • AWS
  • Cloud
  • Data Warehousing
  • Python
Full Remote

Haystack

United Kingdom

Data Engineer

  • Cloud
  • Data Structures
  • ETL
  • SQL
Full Remote

Remote Work Guides

Practical guides for finding and thriving in remote roles.

Browse all articles

Subscribe to our newsletter

New visa sponsorship, relocation and remote tech jobs, delivered to your inbox.