Senior Data Engineer (GCP/BigQuery)

ideal folks llc • Japan

Visa Sponsorship

Apply Now

AI Summary

Design and build high-throughput data infrastructure and pipelines for a next-generation telecommunications platform. Architect BigQuery datasets and ETL/ELT systems using Go or Java to process real-time CDR data. Requires 7+ years of experience with GCP, BigQuery, and distributed systems in a global agile environment.

Key Highlights

7+ years of experience with BigQuery and data pipeline orchestration (Airflow/Dataflow)

Strong proficiency in Go or Java for building custom data connectors and micro-transformers

Global distributed team across Japan, India, China, and Singapore with English as the primary language

Key Responsibilities

Design and implement the data infrastructure that powers B2B, B2C, and ISP business intelligence

Build high-performance data pipelines that ingest, transform, and analyze massive volumes of telecommunications data

Architect and optimize BigQuery datasets, including partitioning, clustering, materialized views, and complex SQL query tuning

Build and maintain production-grade ETL/ELT pipelines using orchestration frameworks such as Apache Airflow or Google Cloud Dataflow

Develop custom data connectors, micro-transformers, and integration services using Go or Java

Design efficient data models (star/snowflake schemas) for analytical reporting, including handling nested/repeated structures common in CDR logs

Integrate real-time streaming telecommunications data using message brokers such as Kafka or Pub/Sub

Ensure data consistency between operational and analytical layers, working with distributed databases such as YugabyteDB

Deploy and scale data processing workloads on Kubernetes using containerized services

Collaborate with product managers, QA engineers, DevOps/SRE, and counterpart engineering teams across global offices

Searching for Data Science roles that provide visa sponsorship? Connect with international employers through Data Science Jobs with Visa Sponsorship opportunities actively seeking talented professionals.

Technical Skills Required

BigQuery Go Java

Benefits & Perks

Visa sponsorship available

Explore our comprehensive directory of visa sponsorship jobs from employers worldwide who are ready to sponsor talented international professionals.

Nice to Have

Infrastructure as Code (IaC) proficiency in Terraform or Pulumi

Experience with real-time stream processing frameworks like Apache Flink or Spark Streaming

Data Governance & Quality experience with tools like Data Catalog, Collibra, Great Expectations, or dbt tests

Advanced Telecom Knowledge in billing, charging systems (OCS/CCS), or network protocol analysis (Diameter, GTP, SIP)

CI/CD for Data pipelines experience

Basic familiarity with BigQuery ML or Vertex AI

Experience in agile environments with globally distributed teams

Job Description

[Hiring for a Client]

Position requires strong GCP / Big Query experience

As a Data Engineer, you will design and build the data infrastructure that powers our next-generation platform. You will be responsible for architecting high-throughput data pipelines, managing large-scale distributed data lakes, and ensuring the integrity and accessibility of data across our ecosystem. Your work will focus on transforming raw, high-velocity telecommunications events into actionable insights, enabling real-time analytics and supporting mission-critical decision-making for our platform.

Position Details

・Design and implement the data infrastructure that powers our B2B, B2C, and ISP business intelligence

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.