Lead QA Engineer

Sunflare LLC is seeking a founding QA Engineer to architect, build, and own test automation and evaluation frameworks for an AI-enabled oncology platform on Google Cloud. This role involves hands-on ownership of quality standards, tooling, and release quality across web, API, and AI layers.

Responsibilities

• Hands-on test automation: design and implement maintainable automated suites in Playwright using JavaScript or TypeScript, spanning the web, API, and AI layers of the platform. You own the codebase, not just the strategy.
• CI/CD quality gates: integrate automated tests into GCP-based pipelines (Cloud Build, GitHub Actions, or equivalent) and enforce quality gates at every stage, ensuring coverage for functional, regression, integration, and performance aspects.
• AI and LLM quality: create evaluation frameworks for RAG pipelines and agentic workflows (LangGraph, CrewAI), assessing accuracy, groundedness, hallucination risk, and clinical safety against oncology scenarios; develop repeatable regression tests as models evolve.
• Quality strategy and standards: own the end-to-end test strategy and key KPIs (defect escape rate, automation coverage, cycle time), and represent quality during architecture reviews and release decisions.
• Release quality and compliance: own go/no-go criteria and risk sign-off, and maintain QA documentation and test records suitable for a HIPAA-regulated environment.
• Foundation and influence: promote shift-left quality across product squads from refinement through production, shaping standards, tooling, and mentoring as the QA function expands.

Requirements

• Hands-on QA depth: 7+ years in software QA with active test automation experience; you still write test code and intend to continue doing so.
• Automation mastery: expert with Playwright and/or Selenium WebDriver in JavaScript or TypeScript, able to architect frameworks and uphold high code quality.
• AI/LLM quality comfort: prepared to define testing approaches for non-deterministic LLM outputs, evaluating accuracy, groundedness, and hallucination risk, even if frameworks are not yet in production.
• Strategy ownership: proven track record managing end-to-end test strategy across web, API, and integration layers with automated testing wired into CI/CD gates.
• Foundational mindset: capable of building the QA function from scratch and setting enduring standards as the first QA hire for a period.
• API testing and communication: strong REST API testing experience (Postman or equivalent) and ability to translate quality metrics into clear narratives for engineers and stakeholders.

Technologies

• Playwright, Selenium WebDriver
• JavaScript, TypeScript
• Google Cloud Platform, Cloud Build
• GitHub Actions
• Vertex AI, Cloud Run, GKE
• Postman
• TestRail, Zephyr, Xray
• LangGraph, CrewAI, Claude, OpenAI GPT, Gemini

Benefits

• 401(k)
• Dental insurance
• Flexible schedule
• Health insurance
• Life insurance
• Paid time off
• Vision insurance

NICE TO HAVE

• Hands-on experience building evaluation frameworks for RAG, agentic workflows, or LLM-as-judge in production.
• Healthcare or regulated software experience (HIPAA, EHR integrations, FHIR / HL7).
• Hands-on Google Cloud Platform experience (Vertex AI, Cloud Run, GKE, Cloud Build).
• Test management platforms (TestRail, Zephyr, Xray) and ISTQB Advanced or equivalent certification.
• Genuine interest in oncology and the operational complexity of clinical environments.

DETAILS

• Pay range: USD 85,000 to 100,000 per year
• Travel: up to 25%
• Benefits include 401(k), medical, dental, and vision coverage, flexible schedule, and paid time off

APPLICATION QUESTIONS

• How many years of hands-on QA/test automation experience do you have, and have you personally architected and built test automation frameworks using Playwright or Selenium WebDriver in JavaScript or TypeScript? Describe the scope of a framework you built.
• Have you served as the first or sole QA engineer at a company, building the testing function and standards from scratch? If yes, briefly describe what you built.
• This role involves testing non-deterministic LLM/AI outputs for accuracy, groundedness, and hallucination risk. Do you have direct experience with this, or a strong conceptual plan for how you would approach it?
• Describe your experience integrating automated test suites into CI/CD pipelines (e.g., GitHub Actions, Cloud Build) and your experience with REST API testing tools like Postman.

LOCATION

Orlando, FL 32819 (Preferred)

ABILITY TO COMMUTE

Orlando, FL 32819 (Required)

ABILITY TO RELOCATE

Orlando, FL 32819: Relocate before starting work (Preferred)

WILLINGNESS TO TRAVEL

25% (Preferred)

WORK LOCATION

In person