Principal Data Engineer - Spark Expertise

Principal Data Engineer

📍 Remote – USA

A VC-backed conversational AI scale-up is expanding its engineering team and is looking for a Principal Data Engineer with deep Spark expertise to help architect and scale the data backbone behind cutting-edge AI-driven systems.

What You’ll Do

• Design and evolve distributed, cloud-based data infrastructure that supports both real-time and batch processing at scale.
• Build high-performance data pipelines that power analytics, AI/ML workloads, and integrations with third-party platforms.
• Champion data reliability, quality, and observability, introducing automation and monitoring across pipelines.
• Collaborate closely with engineering, product, and AI teams to deliver data solutions for business-critical initiatives.

What We’re Looking For

• 5+ years in software development and data engineering with ownership of production-grade systems.
• Proven expertise in Spark/PySpark
• Strong knowledge of distributed computing and modern data modeling approaches.
• Solid programming skills in Python, with an emphasis on clean, maintainable code.
• Hands-on experience with SQL and NoSQL databases (e.g., PostgreSQL, DynamoDB, Cassandra).
• Excellent communicator who can influence and partner across teams.

Bonus Points

• Experience in high-growth, early-stage environments.
• Familiarity with MLOps and deploying ML models into production data workflows.
• A problem-solver at heart, excited by innovation and complex challenges.

Fully remote, great equity and the chance to join a rocketship available here. If you'd like to find out more, don't hesitate to apply!