Senior Machine Learning Engineer

Cloudflare is building machine learning inference that can run across its global network, supporting everything from frontier open LLMs and real-time voice models to customer-deployed models served on heterogeneous GPUs and next-generation accelerators. In this role, you will help define how models operate at scale, with an emphasis on low latency, reliability, and efficient resource use.

Working on Cloudflare’s serverless inference platform, you will develop production-grade ML inference and the evaluation tooling that measures model quality and serving performance across multiple model families, including LLMs, speech, and vision.

Responsibilities

• Develop, optimize, and productionize machine learning models for Cloudflare’s serverless inference platform, focusing on performance, reliability, and model quality.
• Build benchmarking and evaluation frameworks to measure latency, throughput, cost efficiency, and model behavior across LLMs, speech, vision, and other model families.
• Improve inference performance using quantization, batching, caching, model compilation, runtime tuning, and accelerator-aware optimization.
• Collaborate with systems engineers to integrate models into Cloudflare’s distributed inference infrastructure across a heterogeneous fleet of GPUs and next-generation accelerators.
• Advance model deployment workflows, including validation, rollout safety, observability, regression testing, and operational readiness.
• Work with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
• Mentor engineers, contribute to technical direction, and raise the quality bar for production ML engineering practices across the team.

Requirements

• Experience building, optimizing, and operating machine learning models in production environments.
• Strong proficiency with Python and modern ML frameworks such as PyTorch, TensorFlow, JAX, or equivalents.
• Hands-on inference optimization experience for large-scale models, including quantization, batching, caching, compilation, and serving runtime tuning.
• Experience with large-scale inference serving frameworks or runtimes such as SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, or similar.
• Familiarity with LLMs, speech models, vision models, embeddings, multimodal models, retrieval-augmented generation, or other modern deep learning architectures.
• Experience optimizing models for GPUs or specialized accelerators.
• Strong understanding of production ML concerns, including evaluation, monitoring, model regressions, rollout safety, and reliability.
• Ability to collaborate across ML and systems boundaries, including familiarity with distributed systems, networking, or serverless platforms.
• Track record of leading complex technical projects and mentoring other engineers.

Technologies

• Python
• PyTorch
• TensorFlow
• JAX
• SGLang
• vLLM
• TensorRT-LLM
• ONNX Runtime
• Triton
• llama.cpp
• Workers AI

Benefits

• Medical/Rx Insurance
• Dental Insurance
• Vision Insurance
• Flexible Spending Accounts
• Commuter Spending Accounts
• Fertility & Family Forming Benefits
• On-demand mental health support and Employee Assistance Program
• Global Travel Medical Insurance
• Short and Long Term Disability Insurance
• Life & Accident Insurance
• 401(k) Retirement Savings Plan
• Employee Stock Participation Plan
• Flexible paid time off covering vacation and sick leave
• Leave programs, including parental, pregnancy health, medical, and bereavement leave

Bonus Points

• Experience contributing to open source ML tooling, model serving frameworks, or inference runtimes.

Compensation

• Compensation may be adjusted depending on work location.
• For Bay Area based hires: Estimated annual salary of $194,000 - $266,000.
• For New York City, New Jersey, Washington, Washington DC, and California (excluding Bay Area) based hires: Estimated annual salary of $185,000 - $254,000.

Equity

• This role is eligible to participate in Cloudflare’s equity plan.

Location

• New York, NY (hybrid)