E

Senior Data Engineer

encloud services llc United State
Relocation
This Job is No Longer Active This position is no longer accepting applications
AI Summary

Design and implement data ingestion and curation processes using Big Data tools. Develop high-performance, reliable, and maintainable ETL code. Monitor performance of production jobs and advise necessary infrastructure changes.

Key Highlights
Data Ingestion and Curation
ETL Code Development
Performance Monitoring
Key Responsibilities
Build processes that support data transformation, data structures, metadata, dependency, and workload management
Handle ingestion of huge volumes of data from various platforms for Analytics needs
Write high-performance, reliable, and maintainable ETL code
Technical Skills Required
Spark Scala Python Hive HDFS Sqoop HBase Kerberos Sentry Impala SQL Git Shell Scripts
Benefits & Perks
Equal Opportunity Employer
Travel and relocation to various client locations throughout the United States may be required

Job Description


Job Duties And Responsibilities

Build processes that support data transformation, data structures, metadata, dependency and workload management

Build and Implement data ingestion and curation process developed using Big data tools such as Spark (Scala/python), Hive, Spark, HDFS, Sqoop, Hbase, Kerberos, Sentry and Impala etc.

Handle ingestion of huge volumes data from various platforms for Analytics needs and writing high-performance, reliable and maintainable ETL code Strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.

Write shell scripts. Complex SQL queries, Hadoop commands and Git.

Create Database, Schemas, Hive tables (External and Managed) with various file formats (Orc, Parquet, Avro and Text etc.)

Monitor performance of production jobs and advising any necessary infrastructure changes.

Work on Code version using Bitbucket and CI/CD pipeline.

Requirements: Bachelor’s degree in Computer Science, IT, Engg or related with at least 60 months of experience. Send resume to: [email protected] including the JOB ID. Travel and relocation to various unanticipated client locations throughout the United States may be required.Equal Opportunity Employer.

Similar Jobs

Explore other opportunities that match your interests

Senior Geospatial Data Platform Database Engineer (Mission-Critical Systems)

Data Science
12h ago

Premium Job

Sign up is free! Login or Sign up to view full details.

•••••• •••••• ••••••
Job Type ••••••
Experience Level ••••••

Esri

United State

Data Scientist

Data Science
15h ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Entry level

enhance it

United State
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Not Applicable

Caterpillar Inc.

United State

Subscribe our newsletter

New Things Will Always Update Regularly