Senior Databricks Developer for Government Health Sector

ai talent • Australia

Visa Sponsorship

This Job is No Longer Active This position is no longer accepting applications

AI Summary

Join a vital leader in the Government Health sector as a Senior Databricks Developer. Lead the design and development of large-scale data pipelines using PySpark/Scala within the Databricks Lakehouse Platform. Collaborate with government analysts and data scientists to transition analytical models into production-ready pipelines.

Key Highlights

Lead the design and development of large-scale data pipelines using PySpark/Scala within Databricks notebooks and jobs.

Architect and manage the Delta Lake environment for complex public health datasets.

Define and enforce data governance practises within the Lakehouse, utilizing Unity Catalogue for centralized metadata and access control.

Searching for Development & Programming roles that provide visa sponsorship? Connect with international employers through Development & Programming Jobs with Visa Sponsorship opportunities actively seeking talented professionals.

Key Responsibilities

Lead the design and development of large-scale, resilient, and performant ETL/ELT data pipelines using PySpark/Scala within Databricks notebooks and jobs.

Architect and manage the Delta Lake environment, focusing on data ingestion, quality enforcement, and schema evolution for complex public health datasets.

Optimise Databricks clusters, notebooks, and Spark jobs for cost-efficiency and performance, specifically targeting bottlenecks in high-volume batch and streaming workloads.

Define and enforce data governance practises within the Lakehouse, utilizing Unity Catalogue for centralized metadata and access control, adhering to government standards.

Collaborate closely with government analysts and data scientists to transition analytical models and research findings into scalable, production-ready pipelines.

Technical Skills Required

PySpark Scala Delta Lake Azure ADLS Gen2 Terraform Delta Live Tables

Explore our comprehensive directory of visa sponsorship jobs from employers worldwide who are ready to sponsor talented international professionals.

Benefits & Perks

482 visa sponsorship

Job Description

We're partnering with a vital leader in the Government Health sector, dedicated to optimising public health outcomes, resource allocation, and policy analysis through a modern, cloud-native data platform. For the right candidate with the necessary skills and experience, we are pleased to offer 482 visa sponsorship.

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.