Design, build, and maintain scalable data pipelines and modern data lakehouse architectures using Microsoft Fabric. Enable data ingestion, transformation, and governance across OneLake, Synapse, and Power BI. Collaborate with AI Engineers to prepare training datasets for ML/AI use cases.
Key Highlights
Technical Skills Required
Benefits & Perks
Job Description
Data Engineer (Microsoft Fabric & Azure) - Relocation to KL
Position: Data Engineer
Department: Data & AI
Role Overview
The Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and modern data lakehouse architectures using Microsoft Fabric. This role focuses on enabling data ingestion, transformation, and governance across OneLake, Synapse, and Power BI, supporting downstream AI and analytics workloads.
Key Responsibilities
• Design and implement end-to-end data pipelines across Bronze, Silver, and Gold layers within Microsoft Fabric.
• Develop Dataflows Gen2, Spark notebooks, and Synapse pipelines to ingest data from databases, APIs, and files (Excel, CSV, etc.).
• Manage and optimize data storage within OneLake, including partitioning, schema evolution, and delta tables.
• Collaborate with AI Engineers to prepare and curate training datasets for ML/AI use cases.
• Integrate Fabric data outputs with Power BI semantic models and Copilot for Fabric.
• Implement data governance and lineage tracking through Microsoft Purview and Defender for Cloud.
• Monitor pipeline performance using Monitoring Hub, Azure Monitor, and Log Analytics Workspace.
• Automate deployments using CI/CD pipelines for Fabric and Azure Data Factory.
• Enforce data security policies, including role-level security (RLS) and data masking.
Qualifications
• Bachelor’s Degree in Computer Science, Data Engineering, or related field.
• 3+ years of experience in data engineering, preferably on Microsoft Azure or Fabric.
• Strong experience with PySpark, SQL, and Python-based ETL.
• Experience with Microsoft Fabric, Synapse Analytics, or Azure Data Factory.
• Familiarity with Power BI datasets, Purview, and Azure Key Vault.
• Knowledge of delta lakes, metadata management, and data versioning.
• Good understanding of data governance, compliance, and security best practices.
Preferred Skills
• Knowledge of ML data preparation or model output pipelines.
• Exposure to Azure AI Foundry, AI Search, or Copilot for Fabric is a plus.
Location
• Role will be based in KL so have to relocate. Visa will be sponsored
Similar Jobs
Explore other opportunities that match your interests