EngineerJobs.io
← Back to all jobs

Job Description

Own evaluation systems and drive model-side quality improvements for a generative content platform producing AI-generated titles, bullets, A+ modules, and imagery.

  • Build and maintain datasets, rubrics, and automated judges to evaluate changes to the content engine
  • Convert brand rejection reasons into structured, labeled training data for the next model improvement cycle
  • Identify approaches to quantify qualitative improvements in generated content
  • Decide and defend approval thresholds for generated content in partnership with data science and brand teams
  • Create quality gates to detect problematic outputs before they reach brand review, reducing rework across the pipeline

Requirements

  • Strong code and system design experience in any programming language or stack
  • 3+ years owning production software services end to end
  • Formal statistics or machine learning training, or a defensible equivalent depth developed through hands-on work
  • Experience engineering systems with non-deterministic outputs, where correctness is measured rather than assumed
  • Nice to have: Fine-tuning experience (LoRA/PEFT), hands-on LLM or generative media production work, evaluation-system ownership, multimodal evaluation, e-commerce domain knowledge, human-labeling operations, or A/B testing infrastructure

Location and role details

  • Location: Lehi, UT (hybrid)
  • Experience: 3+ years
  • Employment: Full-time

Benefits

  • Unlimited PTO
  • Paid Holidays
  • Onsite Fitness Center
  • Company Paid Life Insurance
  • Casual Dress Code
  • Competitive Pay
  • Health, Vision, and Dental Insurance
  • 401(k) match: Pattern matches 100% of the first 3% and 50% of the next 2% (eligible compensation deferred)

Career growth

  • Pattern prioritizes internal mobility and professional development
  • This role sits at the intersection of software engineering and data science on one of Pattern’s most visible AI systems
  • Work includes deep expertise in evaluation design, fine-tuning, and production ML
  • Experience supports senior IC or technical leadership tracks across Pattern’s broader AI and generative content initiatives

First 30/60/90 days

  • 30 Days: Complete onboarding, learn the generative content pipeline plus existing evaluation datasets and rubrics, contribute to an existing regression suite
  • 60 Days: Own a defined slice of the evaluation system end to end (example: judges and thresholds for a specific content type), start converting brand rejection reasons into labeled training data
  • 90 Days: Independently drive a fine-tuning or retrieval experiment from hypothesis to validated result, with at least one quality gate live in production catching issues before brand review

What Pattern values

  • Game Changers: Open-minded problem solving, new idea sharing, reassessing plans with realistic timelines, productive and innovative contributions, continuous process and outcome improvements
  • Data Fanatics: Use data to understand problems, draw unbiased conclusions, deliver actionable solutions, track results with data
  • Partner Obsessed: Clear status communication to partners, constructive feedback loops, active listening to expectations, delivering results that exceed expectations
  • Team of Doers: Uplift teammates, recognize contributions, take initiative to help, support improvements, and maintain accountability to both team and partners

Hiring process

  • Initial phone interview with Pattern talent acquisition team
  • Video technical interview
  • Onsite interview with hiring manager and a panel of department leaders
  • Professional reference checks
  • Executive review
  • Offer

How to stand out

  • Share professional accomplishments with specific, quantified examples
  • Explain how you will add value and why you are a strong team fit
  • Highlight how you would be partner obsessed at Pattern
  • Include experience from side projects related to data and analytics

Similar Jobs