S

Senior Lead Data Engineer (AI-Augmented Data Platform & Lakehouse Architecture)

seven hills group technologies inc. • United State
Visa Sponsorship
Apply Now
AI Summary

Lead the design and implementation of a scalable, AI-augmented data platform with lakehouse architecture, focusing on data ingestion, transformation, governance, and semantic access. Own end-to-end data pipelines, AI-driven data quality, and conversational data interfaces while mentoring engineers and ensuring platform observability. Requires 12–18 years of hands-on data engineering experience with Python, PySpark, and Azure Databricks.

Key Highlights
Own end-to-end lakehouse architecture (Bronze/Silver/Gold layers) and AI-augmented ingestion frameworks with zero bespoke pipeline code per source
Design and build AI-driven data quality, anomaly detection, and conversational interfaces (GPT-powered text-to-SQL) with fine-grained security
Mentor engineers, set coding standards, and ensure platform self-sufficiency for clients post-engagement
Key Responsibilities
Design and implement a medallion architecture (Bronze/Silver/Gold layers) on ADLS Gen2 with Delta Lake, partitioning, and retention strategies
Build AI-augmented ingestion frameworks that auto-generate metadata, mappings, and validation rules with human-in-the-loop review gates
Develop canonical PySpark/Scala transformations, set coding/testing standards, and debug production incidents across batch/streaming pipelines
Optimize Spark clusters, enforce cost guardrails, and automate CI/CD for Databricks/ADF pipelines using Terraform and Azure DevOps
Create AI-driven data quality frameworks (anomaly detection, reconciliation, synthetic data generation) with differential privacy and masking
Design an ontology-driven semantic layer and GPT-powered conversational interface for natural-language querying with row/column-level security
Implement Unity Catalog for lineage, access control, and PII governance; lead architecture reviews and AI governance forums
Mentor engineers, conduct design reviews, and document reusable patterns for client self-sufficiency

Searching for Data Science roles that provide visa sponsorship? Connect with international employers through Data Science Jobs with Visa Sponsorship opportunities actively seeking talented professionals.

Technical Skills Required
Python PySpark/Scala Azure Databricks (Delta Lake, Unity Catalog)
Benefits & Perks
$140,000–$150,000 annual compensation (H1 transfer up to $150K max)
Hybrid/on-site work in Cary, NC
Benefits package

Explore our comprehensive directory of visa sponsorship jobs from employers worldwide who are ready to sponsor talented international professionals.

Nice to Have
Knowledge graphs (RDF/SPARQL, Neo4j), text-to-SQL systems, or financial domain expertise (reconciliation, actuarial data)
LLMOps/MLOps (model/prompt versioning, cost governance), Databricks certifications (Data Engineer Professional, DP-203/DP-700)
Experience with dbt, Great Expectations, or Workday/Prism accounting systems

Job Description

Lead Data Engineer

| Cary, NC | Hands-On | 12–18 Years of Experience

Role

Lead Data Engineer (Hands-On)

Location

Cary, NC (On-site / Hybrid)

Experience

12–18 Years –

Employment

Full-Time: - $140to 150K per annum plus benefits

H1 transfer – Can go for $145 to $150K MAX ( Candidate should start with Receipt)

 

WHAT YOU WILL OWN

Data Platform Architecture & Engineering

Own the end-to-end lakehouse architecture: Bronze / Silver / Gold layer contracts, zone layout on ADLS Gen2, Delta Lake table design, partitioning, schema evolution, and retention strategy.

Design and build metadata-driven, parameterized ingestion frameworks that onboard new data sources without bespoke pipeline code for every feed.

Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

Ready to apply?

Apply Now
See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Job Overview
Posted Oct 23, 2026
Employment Full-time
Experience Mid-Senior level
Location United State
Category Data Science
Company seven hills group technologies inc.
Keep exploring

Similar Jobs

Explore other opportunities that match your interests

Hidden company

Log in to see location

Premium

Senior Data Engineer (GTM Systems Data Foundations) - AI & Go-To-Market Data Platform

Data Science Full-time 1d ago

Log in free to see the details

Hidden company

Log in to see location

Premium

Principal Data Analyst (Hybrid)

Data Science Full-time 1d ago

Log in free to see the details

BeaconFire Inc.

United State

Data Engineer

Data Science Full-time Entry level 1d ago

  • ETL
  • Numpy
  • Pandas
  • Python
Visa Remote

Visas, Work Permits & Legal Insights

Stay informed about visa types, work permits, and immigration updates for international professionals.

Browse all articles

Subscribe to our newsletter

New visa sponsorship, relocation and remote tech jobs, delivered to your inbox.