Senior Software Engineer (Python) - Large-Scale Data Ingestion and Processing

Jobgether • United Arab Emirates

Remote

This Job is No Longer Active This position is no longer accepting applications

AI Summary

Jobgether is seeking a Senior Software Engineer (Python) to design and maintain large-scale data ingestion and processing systems. The ideal candidate will have 5+ years of experience in Python development, particularly in web scraping and data pipeline systems at scale. Strong experience working with REST APIs, search and data technologies, and distributed processing frameworks is required.

Key Highlights

Design and maintain large-scale data ingestion and processing systems

Develop robust data pipelines to extract, process, and normalize data

Implement preprocessing and transformation logic to support ML/NLP models

Key Responsibilities

Design and maintain large-scale data ingestion and processing systems

Develop robust data pipelines to extract, process, and normalize data

Implement preprocessing and transformation logic to support ML/NLP models

Collaborate with ML and data science teams to integrate classification models into production pipelines

Automate workflows using tools such as Apache Airflow and deploy scalable systems using Kubernetes and AWS

Interested in remote work opportunities in Development & Programming? Discover Development & Programming Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.

Technical Skills Required

Python REST APIs ElasticSearch/OpenSearch Apache Airflow Spark (EMR) Kubernetes AWS Tesseract PyMuPDF spaCy Hugging Face TensorFlow PyTorch

Benefits & Perks

Competitive salary

Fully remote flexibility

Comprehensive health coverage

Paid time off

Parental leave

Medical leave

Retirement savings plan

Flexible spending accounts

Health savings accounts

Wellness support

Home office setup support

Continuous learning and professional development support

Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.

Nice to Have

Exposure to ML/NLP concepts, LLMs, or frameworks such as spaCy, Hugging Face, TensorFlow, or PyTorch

Job Description

This role is a high-impact opportunity for an experienced backend/data-focused engineer who enjoys building large-scale, distributed systems that power data extraction, processing, and search at scale. You will work on complex crawling and ingestion pipelines that collect and structure data from diverse sources such as web pages, APIs, PDFs, and documents. The environment is fast-moving, collaborative, and engineering-driven, with a strong emphasis on ownership, scalability, and problem-solving. You will contribute to the design of robust data architectures that support downstream analytics, classification, and machine learning use cases. The role combines Python engineering, distributed systems, and search technologies in a highly production-oriented setting. You will work closely with ML/NLP teams and infrastructure engineers to deliver reliable, high-performance systems in cloud environments. This is an ideal position for someone who enjoys tackling challenging data problems at scale and building systems that evolve continuously.

Want the full job description? Read the complete details on LinkedIn, the original posting.

Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.