Senior Databricks Data Platform Architect

Caylent • Mexico

Remote

Apply Now

AI Summary

Design and harden enterprise-scale Databricks data platforms within client engagements, focusing on guardrails, automation, and standards. Lead architectural design for lakehouse patterns, Unity Catalog integration, and scalable cloud-native systems on AWS. Requires 6+ years in data engineering or architecture with 3+ years of hands-on Databricks and AWS experience.

Key Highlights

Lead architectural design for Databricks-based data platforms and modernization programs.

Design high-volume batch and real-time streaming pipelines using PySpark and Structured Streaming.

Implement automation, CI/CD, and governance patterns to enable safe self-service and agent-driven use cases.

Interested in remote work opportunities in Devops? Discover Devops Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.

Key Responsibilities

Lead architectural design for Databricks-based data platforms or modernization programs, including workspace consolidation, permissioning/persona models, and configuration-as-code patterns.

Ensure Databricks platform designs align with AWS leading-practice architecture patterns.

Design high-volume batch and real-time streaming pipelines leveraging PySpark and Structured Streaming.

Design and integrate Databricks Unity Catalog with third-party metadata repositories such as AWS Glue Catalog and Snowflake Horizon.

Design and implement automation (ConfigOps), CI/CD, and guardrail patterns that allow business teams to safely extend the platform without compromising governance.

Build and maintain certified data products, a semantic layer, and lineage/metadata exposure to support self-service and agent-driven use cases.

Ensure technical viability, scalability, and production readiness of platform components.

Communicate architectural decisions and trade-offs clearly to both technical and business stakeholders; produce architectural documentation, assessments, and roadmaps.

Partner with client business and engineering teams to understand upstream use-case needs and translate them into platform capabilities and guardrails.

Technical Skills Required

Databricks Amazon Web Services Apache Spark

Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.

Benefits & Perks

100% remote work

Medical Insurance for you and eligible dependents

Generous holidays and flexible PTO

Competitive phantom equity

Paid for exams and certifications

Peer bonus awards

State of the art laptop and tools

Equipment & Office Stipend

Individual professional development plan

Annual stipend for Learning and Development

Job Description

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Network, we combine deep expertise in AWS, artificial intelligence, and Anthropic’s Claude platform to help customers modernize their technology, build intelligent products, and move AI from experimentation into production.

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.