Applied AI Engineer — Conversational Agents (Senior/Staff)

Revin is hiring a Senior/Staff Applied AI Engineer to advance conversational agents with a major focus on voice, owning the improvement cycle from real-customer failures to validated production outcomes.

Responsibilities

• Review production conversations by listening to calls, inspecting traces, and partnering with customer-facing teams to spot failure patterns and prioritize fixes.
• Set quality dimensions and build representative evaluation datasets; combine human judgment, automated graders, and production metrics while validating they reflect what customers actually experience.
• Design reproducible experiments to compare approaches using offline evaluation and controlled production rollouts; identify what works, where it works, and the tradeoffs.
• Investigate issues in turn-taking, interruptions, response latency, transcription errors, speech quality, and spoken naturalness using audio plus system telemetry to diagnose root causes and verify changes.
• Implement improvements, monitor production impact, and take end-to-end responsibility for reliability, cost, regressions, and quality gains.
• Convert production failures and experimental learnings into reusable datasets, evaluations, debugging tools, and release checks to accelerate the team.
• At the Staff level: shape evaluation and experimentation strategy across agent capabilities, identify highest-impact opportunities, and raise team capability to diagnose and improve agent behavior while staying hands-on.

Requirements

• Proven record improving substantial AI agent systems deployed to external customers, including explaining system scale and complexity, problems found, hypotheses and experiments, what you personally implemented, and how you validated production improvement outcomes.
• Strong experimental and analytical judgment: define useful baselines, run informative comparisons, reason about noisy or confounded results, and extract lessons from experiments that do not confirm hypotheses.
• Strong software engineering skills: use code and SQL to analyze conversational data, debug complex agent behavior, and deliver maintainable production software from experimental work.
• Working understanding of agent systems: reason about how models, instructions, context, tools, and orchestration interact; balance quality with latency, reliability, and cost.
• Ownership and clear communication: move forward through ambiguity, communicate findings and tradeoffs, and collaborate with engineers and product to ship improvements to customers.
• Motivation for an early-stage startup: enjoy broad responsibility, rapid iteration, high standards, use AI programming tools heavily, and deliver high-quality outcomes with effective use of those tools.

Technologies

• SQL

Benefits

• Meaningful early equity
• Excellent health coverage: $0 employee premiums for individual medical coverage on most plans, plus dental and vision insurance
• 401(k) matching up to 4% of salary (contribute 5% to receive the full match)
• Unlimited PTO
• Commuter benefits
• Free lunches and dinners in the office
• $100/month gym membership stipend
• Tech equipment of your choice

Voice Experience

• Hands-on experience improving production voice agents is strongly preferred.
• Ability to diagnose conversational timing, speech recognition and generation, audio quality, and end-to-end latency across any provider or stack.
• Open to engineers with exceptional improvement track records on other customer-facing agent types, who can demonstrate strong judgment on voice issues and want to build deeper voice expertise.

Nice to Have

• Experience building simulation, replay, or evaluation tools for multi-turn agents
• Experience developing human review workflows or calibrating automated evaluations against human judgment
• Experience at a strong startup or on a small team with substantial ownership

Location: New York, NY (hybrid)

Compensation: USD 225,000 - 325,000 per yearly