AI Engineer

Momentic is hiring a hands-on AI Engineer to advance the models and agents powering its AI-native testing platform.

Responsibilities

• Fine-tune models for long-running agents, including agents that determine whether test failures originate from infrastructure, recent code changes, or test authorship
• Build and curate training and evaluation datasets from production workflows, using failure patterns to surface improvement opportunities
• Create LLM evaluations and observability so offline results map to production behavior and teams can validate improvements
• Tune prompts, context, and model behavior to improve task success, reliability, latency, token usage, and cost
• Design repeatable experiments and help publish benchmarks demonstrating agent performance on testing and browser workflows
• Own technical projects end-to-end, from problem definition through delivery, partnering with engineering to make practical tradeoffs and ship enhancements

Requirements

• 3+ years of relevant software engineering experience with hands-on contributions to AI-powered capabilities used in production
• Hands-on experience with supervised fine-tuning, including preparing training data and evaluating model behavior
• Experience with prompt and context tuning, evaluation datasets, and LLM observability to diagnose failures and measure improvements
• Strong engineering fundamentals; code efficiently and have clarity on strengths and growth areas
• Ability to use AI agents to build software by breaking down work, tracking progress, checking results, and redirecting agents when they get stuck
• Comfort with autonomy and ambiguity: turn unclear problems into a plan, manage your work, and follow through to delivery

Benefits

• Medical, dental, and vision insurance
• 401(k)
• Unlimited PTO and sick days, plus company-observed US holidays
• Daily lunch and a fully stocked kitchen
• Commuter benefits
• Team offsites and events

Nice to Haves

• Experience with reinforcement learning fine-tuning, RL environments, or LLM-as-judge evaluators
• Experience with long-running agents, recovery from failed steps, or coordinating multiple agents
• Experience publishing benchmarks or running rigorous experiments on AI systems

Location: San Francisco, CA (onsite)

Compensation: USD 175,000 - 250,000 per year