Machine Learning Inference

Machine Learning Inference Engineer Jobs

27 jobs found

1d ago

Machine Learning Engineer 5 (Senior Manager, IC)

Capital One New York, NY

New York, NY

1d ago

$251k - $286k/yr Full time Manager Artificial Intelligence Big Data

Apply Now

Job Description

Capital One is seeking a Machine Learning Engineer to design, build, and scale machine learning models and platforms that move from development into production. In this role, you will help deliver ML solutions across modeling, data engineering, and operations, including deployment, monitoring, and retraining at scale.

This position is based in New York, NY (onsite) and targets an experienced engineer with a strong background in ML engineering, distributed systems, and production software practices. The salary range for this role is USD 250,800 - 286,200 per year.

What you’ll do

Design, build, and/or deliver ML models and components that address real-world business problems in collaboration with Product and Data Science.
Build and scale massive multi-tenant platforms for large footprint ML model training and/or serving.
Apply ML modeling knowledge to influence infrastructure decisions, including model and data/feature selection, training, hyperparameter tuning, dimensionality, and validation, with attention to bias/variance.
Solve complex problems by writing and testing application code, developing and validating ML models, and automating tests and deployment.
Work within a cross-functional Agile team to create and improve software powering state-of-the-art big data and ML applications.
Retrain, maintain, and monitor models in production.
Use or build cloud-based architectures and platforms to deliver optimized ML models at scale.
Construct optimized data pipelines that feed ML models.
Use CI/CD best practices, including test automation and monitoring, to support reliable deployment of ML models and application code.
Help ensure code is well-managed to reduce vulnerabilities, models are well-governed from a risk perspective, and ML systems follow Responsible and Explainable AI best practices.
Use programming languages such as Python, Scala, or Java.

Required qualifications

Bachelor’s Degree or higher in Computer Science, Machine Learning, or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering).
6+ years programming with Python, Java, Golang, or C++.
6+ years Machine Learning experience using PyTorch or TensorFlow and libraries such as Pandas, NumPy, Scikit-learn.
6+ years experience using and operating large scale distributed systems (including Spark and Ray) to prepare AI/ML data.
4+ years deploying and operating ML solutions in production and operating production services in the cloud (AWS, GCP, Azure), including using Kubernetes to manage containerized ML systems at scale.
Master’s or Doctoral Degree in Computer Science, Electrical Engineering, Mathematics, or related field.
5+ years optimizing ML algorithms, configurations, and infrastructure.
5+ years following software development best practices including source control, testing, code reviews, and CI/CD.
5+ years building resilient software with pre-production testing, advanced deployment techniques (one-box, blue/green, gradual dial-up), monitoring and alarms, and incident response plan preparation.
5+ years applying ML techniques (Supervised, semi-supervised, and unsupervised learning, reinforcement learning, etc.) across model types and architectures including Regression, Classification, Clustering, and RNNs, CNNs, LSTMs, Transformers, along with concepts for training (loss function, hyperparameters, regularization) and model evaluation to diagnose issues such as underfitting and overfitting.
5+ years designing, implementing, and scaling production-ready data pipelines for training and evaluating ML models.
Demonstrated ML industry impact through conference presentations, papers, blog posts, open source contributions, or patents.
Ability to communicate complex technical and machine learning concepts clearly to a variety of audiences.

Technologies

Python, Scala, Java, Golang, C++, PyTorch, Tensorflow, Pandas, NumPy, Scikit-learn
Spark, Ray, AWS, GCP, Azure, Kubernetes
CI/CD
RNNs, CNNs, LSTMs, Transformers

Compensation and benefits

Eligible for performance-based incentive compensation, which may include cash bonus(es) and/or long-term incentives (LTI).
Comprehensive, competitive, and inclusive health, financial, and other benefits supporting total well-being.

Salary information by location

Cambridge, MA: $229,900 - $262,400
McLean, VA: $229,900 - $262,400
New York, NY: $250,800 - $286,200
San Francisco, CA: $250,800 - $286,200
San Jose, CA: $250,800 - $286,200

Recruiting support

If you visited the website seeking information on employment opportunities or to apply and require accommodation: 1-800-304-9102 or RecruitingAccommodation@capitalone.com.
For technical support or questions about Capital One’s recruiting process: Careers@capitalone.com.

View full job page →

4d ago

Machine Learning Engineer

Glidewell Dental Irvine, CA

Irvine, CA

4d ago

$113k - $115k/yr Full time Artificial Intelligence AWS Aws Sagemaker

Apply Now

Job Description

In this onsite Machine Learning Engineer role in Irvine, CA, you will build and operationalize ML models and automated training and inference pipelines to improve business workflows and customer experiences.

Build, deploy, and optimize ML systems by working with the team on model training, deployment, monitoring, and continuous optimization
Develop automated ML pipelines for both model training and inference
Drive ML research to production across ML algorithms, feature engineering, analysis, modeling, and production deployment
Collaborate across ML engineering to help mature the use of machine learning within the organization
Learn the business context by building a deep understanding of products, services, data, and customers to support personalized experiences
Identify ML opportunities to improve business workflows and processes using machine learning technology
Support additional initiatives by performing other related duties and projects as directed by management

Requirements

Education: Master’s degree in Machine Learning, Deep Learning, or a computer science-related field
PhD: PhD preferred
Experience: Previous relevant working experience required
ML fundamentals: Understanding of fundamental concepts, practices, and procedures in the ML field
Data and SQL: Prior experience in data discovery, data aggregation, and feature engineering with strong SQL query writing skills preferred
Model lifecycle: Experience training, evaluating, optimizing, deploying, and maintaining ML models on production systems preferred
Evaluation and experimentation: Experience with logging, tracking, A/B testing, evaluation, and analysis of ML algorithm/model performance on production systems preferred
Python development: Strong development skills in Python and experience building data-driven, scalable, reliable applications; AWS preferred
Optimization and applied ML: Experience applying ML algorithms to optimization problems preferred, including examples such as customer sales prediction, recommendation engines, sentiment analysis, deep learning for image and natural language, customer segmentation/clustering, and object detection
ML libraries: Experience with open source ML/deep learning libraries preferred, including MxNet/Gluon, Tensorflow, scikit-learn, pandas, pyTorch, Keras, xgboost, LightGBM, Turi Create, and Nvidia Digits
Data platforms: Experience with relational, non-relational, and high-scale data processing and storage frameworks preferred, including SQL, AWS RedShift, Aurora, S3, DynamoDB, MySQL, and PostgreSQL
AWS services and serverless: Experience with AWS Serverless architecture and AWS native services preferred, including EC2, Lambda, Step Functions, SageMaker, DeepLens, Rekognition, IoT, GreenGrass, Comprehend, Lex/Polly, and Transcribe

Technologies

Python
SQL
AWS
MxNet/Gluon
Tensorflow
scikit-learn
pandas
pyTorch
Keras
xgboost
LightGBM
Turi Create
Nvidia Digits
AWS RedShift
Aurora
S3
DynamoDB
MySQL
PostgreSQL
EC2
Lambda
Step Functions
SageMaker
DeepLens
Rekognition
IoT
GreenGrass
Comprehend
Lex/Polly
Transcribe

Pay Range

$113,000 – $115,000.00 per year

Location: Irvine, CA (onsite)

Education: Master’s degree in Machine Learning, Deep Learning, or a computer science-related field

View full job page →

7d ago

Machine learning engineer II, AWS Inference Optimization

Amazon Development Center U.S., Inc. Santa Clara, CA

Santa Clara, CA

7d ago

$165k - $224k/yr Full time Artificial Intelligence AWS Aws Bedrock

Apply Now

Job Description

Machine learning engineer on an AWS inference optimization team focused on improving large language model inference performance.

Responsibilities

Develop model servers for large language OSS models (examples include Qwen, Gemma, gptoss, and others)
Build inference stack components, including CUDA kernel development in vLLM and SGLang
Create and iterate on services for GenAI optimization and benchmarking
Create and modify a mid-size application or service
Apply inference optimization techniques to improve performance, including:
- Speculative decoding
- Disaggregated Inference
- CUDA kernel tuning
Own price-performance for Amazon LLM inference on SageMaker, Hyperpod, and Bedrock
Deliver production-ready solutions for customer-facing products, using existing academic/industrial research and practical problem solving
Optimize inference performance using AWS inference stacks including vLLM and SGLang
Tune GenAI inference performance on Amazon SageMaker, Amazon Bedrock, and Hyperpod

Requirements

3+ years of non-internship professional software development experience
2+ years of non-internship experience designing or architecting new and existing systems (design patterns, reliability, scaling)
1+ years of software development engineer or related occupational experience
1+ years building large-scale, multi-tiered, multi-threaded, embedded, or distributed software applications, tools, systems, and services using C#, C++, Java, or Perl
Bachelor’s degree (Computer Science, Engineering, Mathematics, or related field) or foreign equivalent
Knowledge of Machine Learning and LLM fundamentals, including:
- Transformer architecture
- Training and inference lifecycles
- Optimization techniques

Technologies

vLLM, SGLang
Amazon SageMaker, Amazon Bedrock, Hyperpod
CUDA
C#, C++, Java, Perl
Transformer architecture

Preferred Qualifications

3+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations
Bachelor’s degree in computer science or equivalent

Benefits

Sign-on payments
Restricted stock units (RSUs)
Health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line)
Flexible Spending Accounts
Adoption and Surrogacy Reimbursement coverage
401(k) matching
Paid time off
Parental leave
Workplace accommodation or adjustment during the application and hiring process (including support for interview or onboarding)

Additional Information for Los Angeles County Applicants

Work safely and cooperatively with employees, supervisors, and staff
Adhere to standards of excellence despite stressful conditions
Communicate effectively and respectfully to ensure exceptional customer service
Follow all federal, state, and local laws and company policies
Criminal history may have a direct, adverse, and negative relationship with some material job duties, including the abilities to follow policies, exercise sound judgment, manage stress, work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the company’s reputation
Under the Los Angeles County Fair Chance Ordinance, qualified applicants with arrest and conviction records will be considered

Location: Santa Clara, CA (onsite)

Compensation: USD 165,200 - 223,600 per year

View full job page →

10d ago

Senior Machine Learning Engineer

Cloudflare New York, NY

New York, NY

10d ago

$194k - $266k/yr Full time Artificial Intelligence Generative AI Gpu Inference

Apply Now

Job Description

Cloudflare is building machine learning inference that can run across its global network, supporting everything from frontier open LLMs and real-time voice models to customer-deployed models served on heterogeneous GPUs and next-generation accelerators. In this role, you will help define how models operate at scale, with an emphasis on low latency, reliability, and efficient resource use.

Working on Cloudflare’s serverless inference platform, you will develop production-grade ML inference and the evaluation tooling that measures model quality and serving performance across multiple model families, including LLMs, speech, and vision.

Responsibilities

Develop, optimize, and productionize machine learning models for Cloudflare’s serverless inference platform, focusing on performance, reliability, and model quality.
Build benchmarking and evaluation frameworks to measure latency, throughput, cost efficiency, and model behavior across LLMs, speech, vision, and other model families.
Improve inference performance using quantization, batching, caching, model compilation, runtime tuning, and accelerator-aware optimization.
Collaborate with systems engineers to integrate models into Cloudflare’s distributed inference infrastructure across a heterogeneous fleet of GPUs and next-generation accelerators.
Advance model deployment workflows, including validation, rollout safety, observability, regression testing, and operational readiness.
Work with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
Mentor engineers, contribute to technical direction, and raise the quality bar for production ML engineering practices across the team.

Requirements

Experience building, optimizing, and operating machine learning models in production environments.
Strong proficiency with Python and modern ML frameworks such as PyTorch, TensorFlow, JAX, or equivalents.
Hands-on inference optimization experience for large-scale models, including quantization, batching, caching, compilation, and serving runtime tuning.
Experience with large-scale inference serving frameworks or runtimes such as SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, or similar.
Familiarity with LLMs, speech models, vision models, embeddings, multimodal models, retrieval-augmented generation, or other modern deep learning architectures.
Experience optimizing models for GPUs or specialized accelerators.
Strong understanding of production ML concerns, including evaluation, monitoring, model regressions, rollout safety, and reliability.
Ability to collaborate across ML and systems boundaries, including familiarity with distributed systems, networking, or serverless platforms.
Track record of leading complex technical projects and mentoring other engineers.

Technologies

Python
PyTorch
TensorFlow
JAX
SGLang
vLLM
TensorRT-LLM
ONNX Runtime
Triton
llama.cpp
Workers AI

Benefits

Medical/Rx Insurance
Dental Insurance
Vision Insurance
Flexible Spending Accounts
Commuter Spending Accounts
Fertility & Family Forming Benefits
On-demand mental health support and Employee Assistance Program
Global Travel Medical Insurance
Short and Long Term Disability Insurance
Life & Accident Insurance
401(k) Retirement Savings Plan
Employee Stock Participation Plan
Flexible paid time off covering vacation and sick leave
Leave programs, including parental, pregnancy health, medical, and bereavement leave

Bonus Points

Experience contributing to open source ML tooling, model serving frameworks, or inference runtimes.

Compensation

Compensation may be adjusted depending on work location.
For Bay Area based hires: Estimated annual salary of $194,000 - $266,000.
For New York City, New Jersey, Washington, Washington DC, and California (excluding Bay Area) based hires: Estimated annual salary of $185,000 - $254,000.

Equity

This role is eligible to participate in Cloudflare’s equity plan.

Location

New York, NY (hybrid)

View full job page →

10d ago

Sr Staff Machine Learning Engineer- Consumer Incentives

Uber San Francisco, CA

San Francisco, CA

10d ago

$267k - $297k/yr Full time Ai Ml Artificial Intelligence Data Processing

Apply Now

Job Description

Uber’s Consumer Incentives team is seeking a Senior Staff Machine Learning Engineer to shape the long-term machine learning strategy and lead hands-on technical execution. This onsite role in San Francisco focuses on personalization, optimization, experimentation, and scalable machine learning systems that translate into measurable business impact.

Role Responsibilities

Define the long-term ML vision, architecture, and technical roadmap for Consumer Incentives, aligning investments with consumer needs and business priorities.
Convert ambiguous problems into actionable strategies, driving end-to-end execution from problem formulation and offline evaluation through experimentation, production deployment, and ongoing improvements.
Design models and algorithms to improve incentive relevance, estimate incremental impact, and optimize decisions within business and operational constraints.
Guide and contribute directly to end-to-end ML system design and implementation across data and feature pipelines, model training, inference, and monitoring, with a focus on reliability, scalability, and maintainability.
Develop strong approaches to model evaluation, experimental design, and causal measurement, balancing short-term gains with long-term consumer value and sustainable growth.
Assess where reinforcement learning, foundation models, and generative AI can improve product outcomes, and lead promising approaches from exploration through production.
Partner with product management, data science, operations, and engineering leaders to prioritize opportunities, make technical tradeoffs, and deliver shared outcomes.
Mentor engineers and technical leaders, elevate the quality of technical decision-making, and build reusable capabilities to accelerate work across teams.

Required Qualifications

Masters degree or Ph.D in Computer Science, Engineering, Mathematics.
12+ years of machine learning experience, including significant work on large-scale deep learning systems.
Demonstrated ownership of high-impact ML systems such as search, recommendations, personalization, and targeting.
Proven ability to set technical strategy for a large organization and influence product roadmaps at the executive level.
Strong product intuition with the ability to connect model improvements to business outcomes.

Preferred Qualifications

Track record of successfully launching multi-year, multi-organization ML initiatives that delivered step-change business outcomes.
Experience championing and driving adoption of multi-year technical roadmaps across multiple large engineering organizations.
Demonstrated impact on engineering standards through mentorship and technical leadership, including the establishment of org-wide best practices.

About the Role and Team

Operating across key business areas like grocery and food, the Consumer Incentives team supports Uber’s growth and long-term profitability. The team delivers products that help create seamless, affordable, and enjoyable customer experiences through AI/ML-powered intelligence and sophisticated distributed systems at scale for hundreds of millions of global users.

Compensation and Eligibility

Location: San Francisco, CA (onsite)
Base salary range: USD 267,000 - 297,000 per year
Eligible to participate in Uber’s bonus program
May be offered an equity award and other types of compensation
Eligible to participate in a 401(k) plan
Eligible for various benefits

Work Location Expectations

Offices remain key to collaboration and Uber’s culture. Unless approved for full remote work, employees must spend at least 50% of their time in-office. Some roles (for example, those at greenlight hubs) require full-time in-office presence. Ask your Recruiter for details about this role’s requirements.

View full job page →

13d ago

Sr Machine Learning Engineer

Disney Entertainment & Sports LLC New York, NY

New York, NY

13d ago

$144k - $202k/yr Full time Artificial Intelligence Machine Learning Deployment Machine Learning Engineering

Apply Now

Job Description

Disney Entertainment & Sports LLC is seeking a Senior Machine Learning Engineer for ESPN to help build and productionize advanced deep learning models that power large-scale recommendation and ranking experiences. This onsite role in New York, NY will drive key architecture and training decisions, then partner with platform and infrastructure teams to bring models into reliable online serving.

The position focuses on end-to-end ownership across model design, offline evaluation, inference optimization, and online integration, with an emphasis on consistency between training and production behavior.

Responsibilities

Lead the design and implementation of advanced deep learning models for retrieval, ranking, and personalization, including embedding-based, sequence, transformer, and multi-task architectures.
Convert product and personalization goals into model formulations, loss functions, sampling strategies, feature representations, and evaluation plans grounded in machine learning theory.
Build production-grade training and inference code with standards for correctness, modularity, testing, reproducibility, and maintainability.
Optimize models for online performance, targeting latency, throughput, memory efficiency, and cost, using approaches such as distillation, quantization, pruning, batching, compilation, and efficient architectural design when appropriate.
Own the productionization path from prototype through offline validation and online integration, ensuring training and inference behavior remain aligned.
Create rigorous offline experimentation plans, ablation studies, and evaluation frameworks, and collaborate with data science and product teams to validate impact through online experiments.
Diagnose model behavior and failure modes, including bias, calibration, cold start, feedback loops, delayed labels, distribution shift, and relevance-quality tradeoffs.
Partner with MLOps, platform, data, and serving teams to define model interfaces, runtime requirements, observability signals, and safe release criteria, without owning general ML infrastructure.
Contribute to technical design reviews, establish model engineering standards, and mentor engineers on deep learning, recommender systems, and production-quality model development.

Requirements

Strong theoretical and applied knowledge of deep learning, including optimization, representation learning, regularization, generalization, and modern neural network architectures.
Experience designing and launching deep learning models in production with ownership that goes beyond notebook experimentation.
Hands-on experience with recommendation, ranking, retrieval, search, or personalization models at meaningful scale.
Expert proficiency in PyTorch, TensorFlow, or a comparable deep learning framework, along with strong Python software engineering skills.
Experience improving inference performance and efficiency, including practical understanding of latency, throughput, memory, hardware utilization, and quality tradeoffs.
Experience with large-scale training, distributed data processing, and production model evaluation.
Solid understanding of experimentation and evaluation, including metric selection, offline-online correlation, ablation analysis, and statistical validation.
Ability to communicate complex modeling decisions clearly and collaborate with product, data science, data engineering, platform, and infrastructure teams.
Experience with multi-stage recommender systems, including candidate generation, retrieval, ranking, re-ranking, and embedding-based approaches.
Deep expertise in one or more areas such as transformers, sequence modeling, multi-task learning, graph neural networks, self-supervised learning, contextual bandits, or reinforcement learning for recommendations.
Experience optimizing neural models with ONNX, TensorRT, CUDA, custom kernels, or comparable inference acceleration technologies.
Experience designing models under strict real-time serving constraints and measuring p95/p99 latency, throughput, memory footprint, and cost-to-serve.
Knowledge of recommender-system challenges including implicit feedback, negative sampling, selection bias, cold start, exploration versus exploitation, and delayed outcomes.
Research publications, patents, open-source contributions, or equivalent evidence of depth in deep learning or recommender systems.
Experience operating in fast-paced or 0-to-1 product development settings with ambiguous requirements.
5+ years of hands-on experience developing machine learning systems, including substantial deep learning model design and productionization experience.
Bachelor’s or Master’s degree in Computer Science, Machine Learning, Data Science, or related field, or equivalent practical experience.

Technologies

PyTorch
TensorFlow
Python
ONNX
TensorRT
CUDA

Compensation and Pay Transparency

The hiring range for this position in New York, NY is $150,900 - $202,400 and for Glendale, CA is $144,100 - $193,200.
The base pay actually offered will take internal equity into account and may vary based on candidate geographic region and job-related knowledge, skills, and experience.
A bonus and/or long-term incentive units may be provided in addition to medical, financial, and/or other benefits, depending on the level and position offered.

View full job page →

16d ago

Senior Staff Machine Learning Engineer

ValueBase Consulting San Jose, CA

San Jose, CA

16d ago

$70 - $80/hr Contract Senior Ai Accelerators Ai Ml

Apply Now

Job Description

ValueBase Consulting is seeking a Senior Staff Machine Learning Engineer to support an AI-enhanced security processor effort. This contract role spans the full lifecycle, from model development and MLOps to silicon/NPU enablement, inference runtime optimization, and embedded integration with end-to-end deployment and monitoring.

Key Responsibilities

Optimize training and inference performance across GPU and AI-accelerator infrastructure, including MLOps pipelines
Design, train, and evaluate ML models across deep learning, LLM, computer vision, or recommendation systems, and take models into production
Harden and extend NPU cores (for example, building on an open RVV/tensor core such as CoralNPU) into production silicon
Build or optimize inference engines and serving runtimes with attention to real hardware constraints including latency, memory, and power
Work below the application layer as needed, including BMC firmware, embedded Linux, or RTOS (for example, Zephyr) to ensure AI features run reliably on actual systems
Create automated test and verification harnesses that close the loop for AI-assisted RTL/DV, hardware bring-up, or manufacturing testing
Apply ML to security use cases such as AI-driven log or intrusion analysis, AI-assisted penetration testing, or firmware and hardware security work
Collaborate with RTL/hardware, firmware, and QA teams to deliver AI features end-to-end from training through deployment and monitoring

Required Qualifications

8-10+ years of hands-on AI/ML experience; Master’s degree required, PhD preferred
Hands-on experience with AI/ML infrastructure and performance, including GPU clusters, distributed training, inference-serving optimization, and MLOps pipelines
Experience in model and algorithm development, including designing, training, and evaluating ML models
Experience taking models into production, including feature engineering, data pipelines, and deployment
AI chip and hardware-aware ML experience, such as optimizing inference engines for a specific chip or adapting model architecture and quantization to chip constraints
Deep, hands-on expertise in at least 2 of the following specialty areas:
- NPU / AI-ACCELERATOR: hardening or extending an NPU core into production silicon, mapping models onto MAC or tensor-engine constraints, or NPU-aware RTL/DV work
- SYSTEMS / SYS-LEVEL SOFTWARE: BMC firmware, embedded Linux, RTOS (for example, Zephyr), or other low-level system software
- INFERENCE ENGINE / RUNTIME: built or materially optimized an inference engine or serving runtime against real hardware constraints
- TEST / VERIFICATION HARNESS: built an automated harness that closes a loop, such as an agent-driven RTL/DV test runner or a hardware bring-up or manufacturing test harness
- CYBER SECURITY: AI-driven log or intrusion analysis, AI-assisted penetration testing, or firmware or hardware security