Lead the architecture and modernization of a unified data fabric and lakehouse platform, designing scalable patterns for access, interoperability, governance, and productization across business and technology teams. Own Apache Iceberg-based data platform capabilities including ingestion, replication, streaming, and lifecycle management while extending platform support for AI/ML ecosystems. Serve as senior engineering lead, mentoring junior engineers and driving technical roadmap execution for firm-wide data and AI initiatives.
Key Highlights
Key Responsibilities
Technical Skills Required
Benefits & Perks
Nice to Have
Job Description
Senior Data Platform Engineer (AI / Data Fabric / Iceberg Lakehouse)
Location: Boston, MA (Onsite 4 days/week) or New York City, NY (Onsite 5 days/week) - relocation assistance available.
About Our Client
Our client is a global investment firm building foundational AI, data, and platform capabilities to enable scale and business transformation. Their platform organization is responsible for evolving technology into robust, reusable, platform-based solutions that increase agility and deliver material business impact across the firm.
The Team
You’ll join a core AI, Data, and Platform Technologies engineering team that architects and supports the firm’s foundational data and AI capabilities. This team is a key enabler for firm-wide services—designing platform primitives that power AI and investment workflows at enterprise scale.
The Role
Our client is hiring a Senior Engineer to take a hands-on technical leadership role in Boston. You will help design and build world-class data engineering capabilities that process massive pipelines, leverage AI-powered insights and document extraction, and integrate across diverse cloud-powered databases.
Looking to advance your Development & Programming career with relocation support? Explore Development & Programming Jobs with Relocation Packages that include comprehensive packages to help you move and settle in your new role.
You’ll define the technical blueprint for how the firm structures, stores, governs, and leverages data to support critical AI and investment platforms—ensuring integrity, performance, and accessibility at scale. This is an onsite role with expectations to be in the Boston office 4 days per week.
What You’ll Do
- Lead the architecture, buildout, and modernization of a unified data fabric, establishing scalable patterns for access, interoperability, governance, and productization across business and technology teams
- Own the design and evolution of Apache Iceberg-based data platform capabilities: ingestion/egress, replication, streaming, performance tuning, lifecycle management, and adoption standards for analytical + operational use cases
- Define and implement compute architecture across stateful, stateless, and distributed processing layers—balancing performance, resiliency, scalability, and cost
- Design and drive adoption of event-driven patterns for real-time ingestion and data movement with low-latency, reliable, observable flows
- Extend platform capabilities to support the firm’s AI/ML ecosystem, including curated datasets, feature-ready pipelines, training/inference data services, and integration points for model development and deployment
- Translate business and engineering priorities into a clear technical roadmap—sequencing platform investments for long-term value
- Serve as a senior engineering lead across strategic platform domains, partnering with app engineering, enterprise architecture, data consumers, and ML stakeholders
- Establish standards for data quality, observability, lineage, governance, security, and operational excellence across batch and streaming environments
- Mentor junior engineers and raise the bar on design rigor, implementation quality, and operational ownership
- Evaluate emerging technologies, run PoCs, and recommend production-ready solutions aligned with target-state architecture
Discover our full range of relocation jobs with comprehensive support packages to help you relocate and settle in your new location.
What Our Client Is Looking For
- Bachelor’s degree in CS/Engineering/IS (advanced degree preferred)
- 4+ years in data engineering, distributed systems, or platform engineering, operating at a senior technical leadership level
- Proven platform mindset: designed/delivered enterprise-scale data platforms or lakehouse ecosystems with emphasis on scalability, reliability, governance, and developer enablement
- Deep hands-on experience with Apache Iceberg and modern open table formats (modeling, partitioning, tuning, metadata management, operational best practices)
- Strong understanding of distributed compute architectures (stateful/stateless processing, orchestration, fault tolerance, performance optimization)
- Experience implementing event-driven/streaming architectures using modern messaging and data movement patterns
- Experience enabling AI/ML platform capabilities (pipeline design, feature/data prep, integration with model development or production ML systems)
- Strong proficiency in Python and SQL; Java/Scala and modern data processing frameworks highly desirable
- Cloud-native/containerized experience, including orchestration, infra automation, and observability tooling
- Ability to operate as a senior technical decision-maker: influence architecture, drive execution through others, and partner across technical + non-technical stakeholders
- Highly desirable: experience in AI Harness engineering
Interested in relocating to United State? Check out our comprehensive Relocation Jobs in United State page with detailed relocation packages and benefits.
Why This Role
- Own core architectural decisions for a firm-wide data fabric and lakehouse platform
- Work at the intersection of high-scale data engineering and AI enablement
- Build real-time and batch systems with strong governance, lineage, and production standards
- High visibility, high leverage platform work that compounds across many business units
Similar Jobs
Explore other opportunities that match your interests
Paid Media Operations Manager
Blizzard Entertainment
Johnson Controls
Cloud-Native Developer Platform Engineer
