Ray Serve Engineer Jobs

1 job found

Today

Sr. Machine Learning Engineer

Workday Boulder, CO

Boulder, CO

Today

$163k - $257k/yr Full time Ai Ml Artificial Intelligence Automation

Apply Now

Job Description

Workday is hiring a Senior Machine Learning Engineer on the AI Model Serving team in a hybrid role based in Boulder, CO.

Responsibilities

Provide technical leadership by driving critical design decisions that improve performance, reliability, and scalability across the platform
Design, implement, and maintain large-scale systems that move ML models into production
Create design documents to align stakeholders on new components and enhancements
Evaluate and adopt new technologies introduced within Workday and across the industry
Troubleshoot, improve, and scale continuous integration pipelines
Collaborate across partner teams with software engineers, machine learning engineers, and data scientists
Respond to alerts and debug production incidents to maintain platform health
Review pull requests and enforce standards for consistency, performance, readability, and security
Develop documentation to share knowledge with other engineers

Requirements

6+ years of related experience in software development, focused on building and operating large-scale distributed systems
Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
Deep hands-on experience deploying and scaling workloads on Kubernetes, with a focus on GPU resource management
Knowledge of optimizing GPU utilization for hosting and tuning smaller open-weight LLMs using inference engines such as vLLM and SGLang
Familiarity with GPU memory constraints, serving tuned models such as LoRA, and autoscaling hardware metrics
Deep experience designing, building, and scaling production-grade distributed systems
Understanding of the full software development lifecycle, including coding standards, testing, code reviews, source control, and deployment for complex, high-throughput platforms
Deep proficiency in Python, including production-level code and Python-based framework systems
Familiarity with both large language models and traditional ML models, including serving, scaling, and production monitoring
Ability to understand operational differences and design abstractions that support both model types effectively
Ability to design and maintain monitoring strategies that provide insight into health, performance, and cost
Excellent written and verbal communication skills, including writing design documents and articulating complex technical ideas to build consensus
Collaborative engineering approach, including mentoring and fostering an inclusive team environment

Technologies

Ray Serve
Kubernetes
GPU
vLLM
SGLang
LoRA
Python
Continuous integration

About the Team

The AI Model Serving team powers every production Workday agent and machine learning use case
Own the services that enable production AI workloads
Act as a gateway to vendor-hosted LLMs (GCP, AWS Bedrock, Gemini) and the primary platform for Workday-hosted internal models
Host thousands of traditional ML models across sharded Ray Serve clusters and maintain the production model registry
Platform scale includes ~2,000 requests per second, peaking at over 10,000 RPS in largest clusters
Scaling architecture supports upgrades to support 20+ new AI agents going into production
Open-weight LLM hosting includes designing infrastructure to host and tune open-source LLMs within the stack
Performance and reliability focus on reducing core latency while maintaining stability
Enterprise governance includes hardening a unified vendor interface and implementing cost-governance controls

Role Focus

Provide technical leadership that shapes the platform vision and direction
Make central design decisions and drive team outcomes
Help set a positive and inclusive engineering culture
Impact Workday’s ability to serve AI at scale, spanning traditional ML models through the latest large language models for Workday agents

Compensation

Salary (USD): $163,000 - $257,400 per year
Primary Location: USA.CO.Boulder
Primary Location Base Pay Range: $171,600 - $257,400
Additional US Location(s) Base Pay Range: $163,000 - $288,000
Total compensation may include eligibility for the Workday Bonus Plan or role-specific commission/bonus, plus annual refresh stock grants

Location and Work Model

Location: Boulder, CO (hybrid)
Flex Work: spend at least half (50%) of your time each quarter in the office or in the field with customers, prospects, and partners (depending on role)

Application Details

Application deadline: 10/30/2026

Equal Opportunity and Legal Notices

Workday will consider qualified applicants with arrest and conviction records pursuant to applicable Fair Chance law
Workday is an Equal Opportunity Employer including individuals with disabilities and protected veterans

Accommodations and AI Use

Workday provides reasonable accommodations for qualified individuals during the application process
Workday supports use of AI tools for employment decision-making with appropriate commitments
Accommodations contact: accommodations@workday.com

Recruiting Fee Statement

Workday will never ask candidates to apply through websites that are not Workday Careers
Workday will never ask candidates to pay a recruiting fee or pay for consulting/coaching services to apply for a job

View full job page →

Get Ray Serve Engineer Jobs alerts

New Ray Serve Engineer Jobs, daily.