Senior Machine Learning Engineer, Conversational AI - Services Special Projects
Job Description
Apple is seeking a Senior Machine Learning Engineer to help build a real-time conversational AI system for Services Special Projects. This role focuses on the conversational intelligence layer, including agent reasoning, tool-calling, low-latency LLM inference, and knowledge services that keep outputs accurate and grounded.
Role Focus
Own core components of a multi-turn conversational experience, translating user requests into structured intent, enabling agent planning and tool usage, and delivering timely, efficient model inference. The role also supports knowledge graph driven services and evaluation pipelines that improve conversational quality within the development workflow.
Responsibilities
- Design multi-turn conversational experiences
- Build agent logic that plans and calls tools
- Develop query understanding models that convert natural language into structured intent
- Optimize large and small models to meet real-time latency and cost requirements
- Support accuracy and grounding to ensure responses are reliable
- Build and scale knowledge graph services to resolve, enrich, and expand user requests with low latency
- Connect the system to real-time data sources so responses remain current
- Create automated and LLM-as-judge evaluation for multi-turn conversations, integrated into the development workflow
Requirements
- Master's degree in Computer Science, Machine Learning, NLP, or a related field
- 7+ years of experience as a software or machine learning engineer, with a significant focus on conversational AI, search, or LLM-powered systems
- Strong programming skills in Go and Python, plus at least one of Java or Scala
- Hands-on experience building and shipping conversational AI or dialogue systems, including intent and slot understanding, multi-turn context management, clarification, and disambiguation
- Production experience building LLM-powered applications, including prompt design, tool/function calling, structured output generation, and agentic orchestration patterns
- Demonstrated experience optimizing LLM inference, including distillation, quantization, batching, KV-cache tuning, caching, and profiling to identify bottlenecks
- Hands-on experience deploying and serving LLMs or ML models directly in the production request path using serving frameworks such as vLLM, SGLang, Triton, or TorchServe
- Experience with knowledge graphs and low-latency lookup and retrieval services
- Fluency with PyTorch and the Hugging Face ecosystem (transformers, PEFT)
- Familiarity with cloud environments, including AWS, and containerization using Docker and Kubernetes
- Excellent communication skills and a collaborative mindset
Preferred Qualifications
- Ph.D. in Computer Science, Machine Learning, NLP, or a related field
- Experience fine-tuning and distilling LLMs for task-specific use, such as query parsing, classification, and response generation, using techniques including LoRA/adapters and teacher/student setups
- Experience designing knowledge graph schemas, building graph compilation and ingestion pipelines, and working with graph databases such as TigerGraph or Neo4j
- Deep experience with KV stores and in-memory serving for low-latency lookups (e.g., Cassandra, Redis, SSTables)
- Experience calibrating LLM judges against human judgments for multi-turn conversations, including inter-annotator agreement and judge reliability tracking
- Experience with personalization and memory in conversational systems, applying privacy principles such as data minimization and on-device processing
- Experience implementing safety guardrails for LLM-powered agents, including prompt-injection defenses and adversarial evaluation
- Familiarity with experiment tracking and data/model versioning tools such as MLflow, Weights & Biases, and DVC
- Deep understanding of distributed systems and service design, including architecting and leading projects with cross-functional stakeholders
- Published work or patents in conversational AI, information retrieval, knowledge graphs, or efficient inference
Technologies
- Go, Python, Java, Scala
- PyTorch, Hugging Face, transformers, PEFT
- vLLM, SGLang, Triton, TorchServe
- AWS, Docker, Kubernetes
- KV-cache tuning
- Cassandra, Redis, SSTables
- TigerGraph, Neo4j
- LoRA, adapters
- MLflow, Weights & Biases, DVC
Location and Salary
- Location: Cupertino, CA (onsite)
- Salary Range: USD 184,700 - 324,800 per year
Benefits
- Comprehensive medical and dental coverage
- Retirement benefits
- A range of discounted products and free services
- Reimbursement for certain educational expenses, including tuition
- Discretionary bonuses or commission payments may be available, and relocation may be supported
- Participation in Apple’s discretionary employee stock programs may allow employees to become Apple shareholders
- Eligibility for discretionary restricted stock unit awards
- Option to purchase Apple stock at a discount through voluntary participation in the Employee Stock Purchase Plan
Similar Jobs
F