Machine Learning Engineer, Evaluation - Responsible AI and Safety

Build safer, more inclusive generative AI experiences at scale. Apple is looking for a Senior Machine Learning Engineer to lead Responsible AI Evaluation efforts for global generative AI products. In this onsite role in Cambridge, MA, you will help turn policy and risk assessments into measurable evaluation criteria that inform ship or no-ship decisions.

How you will make an impact

• Operationalize policy and risk assessments into measurable safety and risk criteria by designing datasets, creating LLM- and human-in-the-loop analysis methods, and implementing robust data pipelines.
• Run pre- and post-launch evaluation analysis, perform failure investigations, and use human and automated feedback to improve feature safety and strengthen user trust.
• Apply technical expertise to identify and solve complex engineering roadblocks, while empowering leadership with data-driven safety recommendations.
• Defend and communicate safety metrics to stakeholders and decision-makers.

Key responsibilities

• Develop metrics to evaluate safety and fairness risks in generative models and Gen-AI features.
• Design datasets, determine data needs, and create scalable data coverage solutions using human and synthetic generation.
• Create sampling strategies and combine human annotation with auto-grading to produce high-confidence insights at speed and at large scale.
• Design, implement, and optimize data pipelines, collaborating cross-functionally to deliver end-to-end safety evaluations.
• Drive operational excellence by automating manual workflows and optimizing evaluation velocity for rapid product release cycles.
• Distill evaluation findings into actionable insights for product engineering teams and safety policy development.
• Work with highly sensitive content, including exposure to offensive and controversial material.

What you bring

• Experience developing safety and fairness evaluation metrics for generative models and Gen-AI features.
• Experience designing datasets, scaling data coverage with human and synthetic generation, and creating sampling and annotation workflows.
• Experience building and optimizing data pipelines for end-to-end safety evaluations.
• Capability to automate manual workflows and improve evaluation throughput for fast-moving product timelines.
• Ability to communicate complex findings into actionable guidance for technical and policy stakeholders.

Preferred qualifications

• Experience with multimodal models, agentic systems, and/or diverse languages and locales.
• Experience with AI compliance and regulatory needs, specifically child safety, clinical or health applications, or companionship chatbots.
• Experience conducting large-scale benchmarking across models and capabilities.

Compensation & benefits

• Base pay range: USD 165,800 - 292,400 per year, depending on skills, qualifications, experience, and location.
• Opportunity to become an Apple shareholder through Apple’s discretionary employee stock programs, including discretionary restricted stock unit awards, and an ability to purchase Apple stock at a discount via the Employee Stock Purchase Plan.
• Comprehensive medical and dental coverage.
• Retirement benefits.
• Discounted products and free services.
• Reimbursement for certain educational expenses, including tuition, for formal education related to advancing your career.
• Additionally, this role might be eligible for discretionary bonuses or commission payments, as well as relocation.
• Note: Apple benefit, compensation, and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.