Big Data Application Developer

Jobgether • United States

Remote

This Job is No Longer Active This position is no longer accepting applications

AI Summary

Design and develop large-scale data solutions supporting advanced analytics, reporting, and machine learning initiatives. Build and optimize high-volume data pipelines across modern big data ecosystems ensuring reliability, scalability, and performance. Requires 5+ years of experience with Apache Spark, Hadoop ecosystem, and distributed data platforms.

Key Highlights

Design and develop scalable data applications and processing pipelines

Strong expertise with Apache Spark, Hadoop ecosystem (Hive, HDFS, Sqoop, HBase), and streaming platforms (Kafka, Spark Streaming, Flink)

Proficiency in Python or Shell scripting, SQL, and workflow orchestration tools (Airflow, Oozie)

Key Responsibilities

Design, develop, and maintain scalable data applications and processing pipelines

Build and maintain data ingestion, transformation, and analytics workflows supporting enterprise applications

Develop high-performance processing solutions using Apache Spark with Scala, Python, or Java

Work with Hadoop technologies including Hive, HDFS, Sqoop, HBase, and related distributed data platforms

Implement and support streaming data solutions using technologies such as Kafka, Spark Streaming, or Flink

Optimize data processing performance through effective use of distributed computing concepts

Develop and maintain workflow automation using orchestration tools such as Airflow or Oozie

Create scripts and automation tools using Python or Shell to improve operational efficiency

Troubleshoot complex data platform issues, perform debugging, and maintain technical documentation

Support cloud-based big data environments and contribute to modernization initiatives

Interested in remote work opportunities in Data Science? Discover Data Science Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.

Technical Skills Required

Apache Spark Hadoop ecosystem Python

Benefits & Perks

Competitive annual salary range of $100,000-$150,000

Fully remote work opportunity within the United States

Full-time direct employment

Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.

Nice to Have

Preferred experience with cloud-based Hadoop platforms such as AWS EMR, Azure HDInsight, or Databricks

Familiarity with modern lakehouse technologies including Delta Lake, Iceberg, or Hudi

Exposure to data governance tools such as Apache Atlas or Collibra

Experience with Kubernetes-based data platforms, CI/CD practices, and infrastructure-as-code solutions

Job Description

This role offers the opportunity to design and develop large-scale data solutions that support advanced analytics, reporting, and machine learning initiatives.You will build and optimize high-volume data pipelines across modern big data ecosystems while ensuring reliability, scalability, and performance.The position combines software engineering expertise with deep knowledge of distributed data platforms and processing frameworks.You will work on complex data challenges involving structured and unstructured information, helping organizations transform data into actionable insights.

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.