Software Engineer, ML Serving Platform

DoorDash is hiring a Software Engineer for the ML Serving Platform to deliver self-serve infrastructure that connects request routing and online feature retrieval to CPU and GPU model inference, optimized for production scale.

Responsibilities

• Build the next generation ML serving platform by connecting request routing, online feature retrieval, and model inference on CPU and GPU infrastructure.
• Solve infrastructure challenges across latency, reliability, resource efficiency, and scale.
• Expose platform capabilities through self-serve tools and workflows for internal teams.
• Own projects end to end: technical design, implementation, testing, rollout, and production support.
• Collaborate with internal teams to understand emerging modeling needs, remove adoption blockers, and translate open source advances into measurable production improvements.
• Develop tools, APIs, and workflows that help teams independently deploy, configure, validate, and operate models.
• Assess and integrate evolving inference frameworks and enable open source models, turning promising capabilities into reliable, efficient services at scale.
• Improve the end-to-end path for prediction: routing prediction requests, retrieving online features, and executing models on a platform serving tens of millions of predictions per second.
• Build modular components so routing, feature retrieval, and model execution can evolve and scale independently as workloads and modeling requirements change.
• Design systems to handle changing traffic while meeting latency and availability goals and using CPU and GPU resources efficiently.
• Create integrations, validation, and migration tooling to support adoption of new serving capabilities and a unified global platform.
• Use benchmarking, profiling, metrics, and tracing to find bottlenecks; participate in on-call and improve automation and runbooks.

Requirements

• 2+ years of software engineering experience building and maintaining production services or infrastructure.
• Strong computer science fundamentals and backend/system programming proficiency in Java, Kotlin, Go, C++, or Python.
• Understanding of distributed systems, including concurrency, networking, timeouts, failure handling, and performance trade-offs.
• Ability to take a defined problem through technical design, tested implementation, and safe production rollout.
• Experience debugging production systems using operational data to improve reliability, performance, or cost.
• Collaboration mindset focused on the engineers using the platform; work effectively with partners and customers to reduce complexity in adoption.
• A degree in Computer Science (or related field) or equivalent practical experience.

Technologies

• Java, Kotlin, Go, C++, Python
• Kubernetes
• CPU, GPU

Nice to Have

• Experience with ML inference infrastructure, online feature retrieval, or other latency-sensitive distributed services.
• Experience operating containerized workloads on Kubernetes, including deployment, resource management, or autoscaling.
• Experience building self-serve developer platforms, APIs, or automation to help other engineers move faster.
• Experience integrating open source infrastructure or models and validating behavior under production workloads.
• Familiarity with CPU/GPU performance profiling, inference runtimes, or model deployment workflows.

Location: San Francisco, CA (onsite)

Compensation: USD 130,600 - 192,000 per year