Senior Ruby Developer - LLM Evaluation & Repository Validation (Part-Time, Remote)

• Role: Ruby Developer (Remote)
• Location: Remote (Work from Anywhere)
• Job Type: Part-Time
• Payout: Competitive, based on experience

Role Overview:

We are hiring for one of our clients, seeking a Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation) to work on a part-time basis. The role involves building verifiable software engineering tasks using public repository histories for LLM training datasets. Candidates will evaluate LLM performance on realistic code problems through hands-on engineering work.

Key Responsibilities:

• Analyze and triage GitHub issues across trending open-source libraries.

• Set up and configure code repositories, including Dockerization and environment setup.

• Evaluate unit test coverage and quality in public repositories.

• Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

• Collaborate with researchers to identify repositories and issues challenging for LLMs.

Required Skills & Qualifications:

• Minimum 3+ years of overall software engineering experience is required.

• Strong Ruby proficiency is required.

• Experience with Git, Docker, and basic software pipeline setup is required.

• Ability to understand and navigate complex codebases is required.

• Comfort with running, modifying, and testing real-world projects locally is required.

More About the Opportunity:

This role offers a unique opportunity to work with a global leader in the Technology, Information and Internet industry, contributing to the advancement of AI-assisted software development. Opportunities to lead junior engineers in collaborative projects are available.

Equal Opportunity Employer:

We hire based on skills and expertise. All qualified candidates are welcome regardless of background, experience, or prior employment history. Applications are reviewed solely on demonstrated technical ability and qualifications.

Apply Now!