N

QA Lead for AI Systems

netrolynx ai • United State
Visa Sponsorship Relocation
Apply Now
AI Summary

Lead the development of QA infrastructure and evaluation frameworks for AI and LLM systems. Design and implement automated testing, CI/CD quality gates, and regression tests for agent workflows. Ensure product reliability, safety, and compliance in a regulated life sciences environment.

Key Highlights
First dedicated QA hire to build the function from the ground up
Focus on evaluating LLMs, agent workflows, and non-deterministic AI systems
Hands-on engineering role requiring Python and Playwright expertise

Looking to advance your QA & Testing career with relocation support? Explore QA & Testing Jobs with Relocation Packages that include comprehensive packages to help you move and settle in your new role.

Key Responsibilities
Develop and implement evaluation frameworks for large language models and agent-based systems to assess performance, safety, and compliance.
Design and maintain regression tests to monitor model updates, prompt modifications, and workflow changes.
Create automated testing infrastructure utilizing Python and Playwright for rapid and reliable test execution.
Own end-to-end testing processes for web applications and APIs, including functional, integration, and performance testing.
Establish and manage CI/CD quality gates, automating release workflows to facilitate smooth deployment cycles.
Conduct manual and exploratory testing on new features to identify unforeseen issues.
Monitor production systems, analyze failures, and develop repeatable tests to proactively address potential issues.
Test agent workflows for failure modes such as tool misuse, looping, hallucinations, and prompt injection.
Collaborate with engineering and product teams to define, implement, and uphold quality standards.
Build and scale the QA function from the ground up, establishing processes, best practices, and documentation.
Technical Skills Required
Python Playwright CI/CD

Discover our full range of relocation jobs with comprehensive support packages to help you relocate and settle in your new location.

Benefits & Perks
Competitive base salary ranging from $160,000 to $300,000
Meaningful equity package
Comprehensive health benefits including medical, dental, and vision coverage
401(k) plan with company matching contributions
Visa sponsorship options available
Relocation support to New York City
Unlimited paid time off
Professional development stipend

Interested in relocating to United State? Check out our comprehensive Relocation Jobs in United State page with detailed relocation packages and benefits.

Nice to Have
Experience working with agent frameworks such as LangChain, LangGraph, or similar tools
Familiarity with evaluation and monitoring tools like LangSmith, Langfuse, Arize, Braintrust, Promptfoo, or OpenAI Evals
Knowledge of TypeScript and React for testing front-end components and integrations
Experience with adversarial testing, red-team assessments, or security-focused testing methodologies
Backend development experience with asynchronous Python services and job queue management
Background working in regulated industries such as healthcare, pharmaceuticals, or finance
MLOps or LLMOps experience, including deployment, monitoring, and management of AI models in production

Job Description

Storm3 is a rapidly expanding Series A AI startup dedicated to revolutionizing the life sciences industry through innovative software solutions. Our platform leverages cutting-edge artificial intelligence to assist enterprise clients in creating, reviewing, and managing complex, regulated content efficiently and accurately. With a focus on integrating advanced language models, agentic systems, and applied AI, Storm3 is at the forefront of technological innovation in this sector. Our collaborations with leading organizations exemplify our commitment to transforming how regulated content is handled, ensuring compliance, quality, and speed. As a company, we foster a dynamic and inclusive environment that encourages innovation, continuous learning, and professional growth, aiming to set new standards in AI-driven solutions for the life sciences industry.
Want the full job description? Read the complete details on LinkedIn, the original posting.
Continue on LinkedIn

This is a short excerpt. All rights to the full description belong to its original publisher.

Apply Now
See Jaabz jobs first on Google 1 tap · free · in AI Overviews Jaabz is on your Google Manage Preferred Sources
Job Overview
Posted Date Oct 06, 2026
Employment Type Full-time
Experience Level Associate
Location United State
Category Testing
Company netrolynx ai
Mentioned Skills
python react typescript artificial intelligence ai continuous integration mlops machine learning

Similar Jobs

Explore other opportunities that match your interests

Network Test Engineer - High-Speed Optical Transceivers
Testing
•
9h ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Associate

aoi

United State

QA Automation Engineer (Playwright & API Testing) – W2 Bench Role
Testing
•
23h ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Mid-Senior level

Baanyan Software Services, Inc...

United State

Program Test Lead Engineer - ADMS
Testing
•
1d ago
Visa Sponsorship Relocation Remote
Job Type Contract
Experience Level Director

ctc

United State

Subscribe our newsletter

New Things Will Always Update Regularly