Machine Learning Engineer 4 - Manager IC

Build and scale enterprise machine learning on Capital One’s Enterprise ML Platform team. In this Machine Learning Engineer 4 (Manager IC) role, you will help evolve a high-throughput, multi-tenant training platform and own Kubeflow Pipelines v2 on Amazon EKS. You will also integrate agentic AI into engineering lifecycles, supporting upstream open-source Kubeflow while maintaining enterprise-grade stability, security, and governance.

What you’ll do

• Design, build, and deliver ML models and components to address real business problems in partnership with Product and Data Science
• Drive ML infrastructure decisions using practical experience across model development topics such as model selection, data and feature selection, training, hyperparameter tuning, dimensionality, and bias/variance and validation
• Solve complex problems by writing and testing application code, developing and validating ML models, and automating tests and deployment
• Collaborate within a cross-functional Agile team to build and improve software enabling large-scale big data and ML applications
• Retrain, maintain, and monitor models in production
• Leverage or develop cloud-based architectures, technologies, and platforms to deliver optimized ML models at scale
• Construct optimized data pipelines that feed ML models
• Apply CI/CD best practices, including test automation and monitoring, to support reliable deployment of models and application code
• Ensure code quality to reduce vulnerabilities, support risk-based governance for models, and follow best practices for Responsible and Explainable AI
• Use programming languages such as Python, Scala, or Java

What you bring

• Bachelor’s Degree or higher in Computer Science, Machine Learning, or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
• 4+ years programming with Python, Java, Golang, or C++
• 4+ years Machine Learning experience using industry-standard frameworks PyTorch or Tensorflow and libraries (Pandas, NumPy, Scikit-learn)
• 4+ years operating large-scale distributed systems (Spark, Ray) to prepare AI or ML data
• 2+ years deploying and operating ML solutions in production using cloud platforms (AWS, GCP, Azure) and using Kubernetes to manage large-scale containerized ML software systems

Technologies you may work with

• Python, Scala, Java, Golang, C++
• PyTorch, Tensorflow, Pandas, NumPy, Scikit-learn
• Spark, Ray
• Kubeflow Pipelines v2, Amazon EKS, Kubernetes, AWS, GCP, Azure
• Docker, CI/CD, Blue/green, Canary, Progressive rollout
• Transformers, CNNs, RNNs
• Supervised learning, Unsupervised learning, Reinforcement learning

Preferred qualifications

• Master’s or Doctoral Degree in Computer Science, Electrical Engineering, Mathematics, or related field
• 3+ years designing, training, optimizing, and evaluating modern ML models across architectures (Transformers, CNNs, RNNs) and paradigms (supervised, unsupervised, reinforcement learning), including loss design, hyperparameter tuning, and diagnostic debugging
• 3+ years architecting and scaling distributed end-to-end ML systems, from production-ready training and evaluation data pipelines to algorithm and infrastructure optimization
• 3+ years managing ML workloads using Docker, Kubernetes, and pipeline orchestration tools (Kubeflow is a strong plus)
• 3+ years applying software best practices (CI/CD, version control, automated testing, code reviews) and deploying resilient systems with progressive rollout strategies (blue/green, canary/one-box), including monitoring, alerting, and incident response planning
• 1+ years experience as a technical lead developing ML solutions using industry best practices, patterns, and automation
• Authored or co-authored a paper on a ML technique, model, or proof of concept

Compensation and benefits

Location: New York, NY (onsite)

Salary: USD 215,200 - 245,600 per year

• This role is also eligible for performance-based incentive compensation, which may include cash bonus(es) and/or long-term incentives (LTI)

Benefits: Capital One offers a comprehensive, competitive, and inclusive set of health, financial and other benefits.

Additional information

• Expected minimum application acceptance: 5 business days
• No agencies
• Capital One is an equal opportunity employer committed to non-discrimination in compliance with applicable laws
• Capital One promotes a drug-free workplace
• Capital One will consider qualified applicants with criminal history in a manner consistent with applicable laws
• No employment authorization sponsorship: Capital One will not sponsor a new applicant for employment authorization, or offer any immigration-related support for this position
• Accommodation: If you require an accommodation, contact Capital One Recruiting at 1-800-304-9102 or via [email protected]
• Technical support/questions: [email protected]