M

Research Crawling Engineer

Romania
Full Remote
This Job is No Longer Active This position is no longer accepting applications

Design and operate large-scale web data acquisition systems, providing high-quality inputs for research and model development. Collaborate with research teams to ensure data collection efforts align with modeling requirements. Optimize infrastructure to ensure cost-efficiency, low latency, and reliability.

Key Highlights
Design and operate large-scale web data acquisition systems
Provide high-quality inputs for research and model development
Collaborate with research teams to ensure data collection efforts align with modeling requirements
Key Responsibilities
Construct and maintain large-scale web crawlers across diverse domains
Design high-throughput, fault-tolerant systems for data collection, managing volumes ranging from millions to billions of URLs per day
Navigate anti-bot systems, rate limits, and dynamic, JavaScript-heavy websites
Technical Skills Required
Go Rust Python Java C++
Benefits & Perks
Impactful Opportunity: Contribute to the development of a web-scale crawler and knowledge graph at the forefront of AI data accessibility
High-Performance Culture: Join a lean, low-ego team that prioritizes high output and professional growth
Remote Work: This position is part of a fully remote team, offering flexibility and autonomy

Job Description

Remote - Must have a 6 hour overlap with ESTWe are hiring on behalf of our client who is a technical infrastructure firm specializing in the delivery of massive-scale web data to organizations developing advanced artificial intelligence models. The organization supports high-capacity bandwidth-sharing networks and operates a distributed crawler capable of accessing high-quality public web data at a global scale. Additionally, the team has engineered sophisticated pipelines for the ingestion, segmentation, and annotation of billions of multimedia files, facilitating dataset creation for frontier research labs.
Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Keep exploring

Similar Jobs

Explore other opportunities that match your interests

Remote Work Guides

Practical guides for finding and thriving in remote roles.

Browse all articles

Subscribe to our newsletter

New visa sponsorship, relocation and remote tech jobs, delivered to your inbox.