Llm Performance Engineer Jobs

1 job found

Today

Senior Machine Learning Engineer

Cloudflare New York, NY

New York, NY

Today

$194k - $266k/yr Full time Artificial Intelligence Generative AI Gpu Inference

Apply Now

Job Description

Cloudflare is building machine learning inference that can run across its global network, supporting everything from frontier open LLMs and real-time voice models to customer-deployed models served on heterogeneous GPUs and next-generation accelerators. In this role, you will help define how models operate at scale, with an emphasis on low latency, reliability, and efficient resource use.

Working on Cloudflare’s serverless inference platform, you will develop production-grade ML inference and the evaluation tooling that measures model quality and serving performance across multiple model families, including LLMs, speech, and vision.

Responsibilities

Develop, optimize, and productionize machine learning models for Cloudflare’s serverless inference platform, focusing on performance, reliability, and model quality.
Build benchmarking and evaluation frameworks to measure latency, throughput, cost efficiency, and model behavior across LLMs, speech, vision, and other model families.
Improve inference performance using quantization, batching, caching, model compilation, runtime tuning, and accelerator-aware optimization.
Collaborate with systems engineers to integrate models into Cloudflare’s distributed inference infrastructure across a heterogeneous fleet of GPUs and next-generation accelerators.
Advance model deployment workflows, including validation, rollout safety, observability, regression testing, and operational readiness.
Work with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
Mentor engineers, contribute to technical direction, and raise the quality bar for production ML engineering practices across the team.

Requirements

Experience building, optimizing, and operating machine learning models in production environments.
Strong proficiency with Python and modern ML frameworks such as PyTorch, TensorFlow, JAX, or equivalents.
Hands-on inference optimization experience for large-scale models, including quantization, batching, caching, compilation, and serving runtime tuning.
Experience with large-scale inference serving frameworks or runtimes such as SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, or similar.
Familiarity with LLMs, speech models, vision models, embeddings, multimodal models, retrieval-augmented generation, or other modern deep learning architectures.
Experience optimizing models for GPUs or specialized accelerators.
Strong understanding of production ML concerns, including evaluation, monitoring, model regressions, rollout safety, and reliability.
Ability to collaborate across ML and systems boundaries, including familiarity with distributed systems, networking, or serverless platforms.
Track record of leading complex technical projects and mentoring other engineers.

Technologies

Python
PyTorch
TensorFlow
JAX
SGLang
vLLM
TensorRT-LLM
ONNX Runtime
Triton
llama.cpp
Workers AI

Benefits

Medical/Rx Insurance
Dental Insurance
Vision Insurance
Flexible Spending Accounts
Commuter Spending Accounts
Fertility & Family Forming Benefits
On-demand mental health support and Employee Assistance Program
Global Travel Medical Insurance
Short and Long Term Disability Insurance
Life & Accident Insurance
401(k) Retirement Savings Plan
Employee Stock Participation Plan
Flexible paid time off covering vacation and sick leave
Leave programs, including parental, pregnancy health, medical, and bereavement leave

Bonus Points

Experience contributing to open source ML tooling, model serving frameworks, or inference runtimes.

Compensation

Compensation may be adjusted depending on work location.
For Bay Area based hires: Estimated annual salary of $194,000 - $266,000.
For New York City, New Jersey, Washington, Washington DC, and California (excluding Bay Area) based hires: Estimated annual salary of $185,000 - $254,000.

Equity

This role is eligible to participate in Cloudflare’s equity plan.

Location

New York, NY (hybrid)

View full job page →

Get Llm Performance Engineer Jobs alerts

New Llm Performance Engineer Jobs, daily.