Machine Learning Engineer 4 - Manager IC

Capital One is hiring a Machine Learning Engineer 4 (Manager IC) to help build and evolve an enterprise ML platform for high-throughput, multi-tenant training and production model workflows.

Responsibilities

• Design, build, and deliver ML models and components to address real-world business problems in collaboration with Product and Data Science teams
• Guide ML infrastructure decisions using ML modeling fundamentals, including model and data/feature selection, training, hyperparameter tuning, dimensionality, bias/variance considerations, and validation approach
• Develop and test application code to solve complex problems through ML model development, validation, and automation of tests and deployment
• Work within a cross-functional Agile team to create and enhance software for big data and ML applications
• Retrain, maintain, and monitor models in production
• Use cloud-based architectures, technologies, and platforms to deliver optimized ML models at scale
• Construct optimized data pipelines to supply ML models
• Apply continuous integration and continuous deployment practices, including test automation and monitoring, to support successful model and code deployments
• Manage code to reduce vulnerabilities, support risk-governed model practices, and follow Responsible and Explainable AI best practices
• Use programming languages such as Python, Scala, or Java for ML engineering and platform work
• Own and evolve Kubeflow Pipelines v2 on Amazon EKS as part of the enterprise ML platform
• Contribute to agentic AI integration, including work connected to platform stability and security

Requirements

• Bachelor’s Degree or higher in Computer Science, Machine Learning, or related quantitative fields (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
• At least 4 years programming with Python, Java, Golang, or C++
• At least 4 years ML experience using industry-standard frameworks PyTorch or Tensorflow, with libraries including Pandas, NumPy, and Scikit-learn
• At least 4 years operating large-scale distributed systems such as Spark and Ray to prepare AI/ML data
• At least 2 years deploying and operating ML solutions in production services on cloud platforms (AWS, GCP, Azure) and using Kubernetes to manage containerized ML systems

Team / Platform Context

• Enterprise ML Platform team builds and evolves a high-throughput, multi-tenant training platform powering model workflows across Capital One, including classical ML and GenAI fine-tuning
• Platform work includes distributed computing capabilities using Ray, Spark, and Dask
• Role includes owning the Kubeflow Pipelines v2 application layer on Amazon EKS, plus developing internal Python and Go SDKs, custom Kubernetes operators, and developer tooling
• Contribute to upstream open-source Kubeflow while balancing feature delivery with enterprise-grade platform stability and security
• Integrate agentic AI into modern engineering lifecycles

Technologies

• Python, Scala, Java, Golang, C++, Go
• PyTorch, Tensorflow
• Pandas, NumPy, Scikit-learn
• Spark, Ray, Dask
• Kubeflow Pipelines v2, Kubeflow
• Amazon EKS, Kubernetes
• AWS, GCP, Azure
• Docker
• Python SDKs

Preferred Qualifications

• Master’s or Doctoral Degree in Computer Science, Electrical Engineering, Mathematics, or a related field
• 3+ years designing, training, optimizing, and evaluating modern ML models across architectures (e.g., Transformers, CNNs, RNNs) and paradigms (supervised, unsupervised, reinforcement learning), including loss design, hyperparameter tuning, and diagnostic debugging
• 3+ years architecting and scaling distributed end-to-end ML systems, from production-ready training/evaluation data pipelines to algorithm and infrastructure optimization
• 3+ years managing ML workloads using Docker, Kubernetes, and pipeline orchestration tools (Kubeflow preferred)
• 3+ years applying modern software best practices (CI/CD, version control, automated testing, code reviews) and deploying resilient systems using progressive rollout strategies (blue/green, canary/one-box), including monitoring, alerting, and incident response planning
• 1+ years of experience as a technical lead developing ML solutions using industry best practices, patterns, and automation
• Authored or co-authored a paper on a ML technique, model, or proof of concept

Benefits

• Comprehensive, competitive, and inclusive set of health, financial, and other benefits to support total well-being

Location: McLean, VA (onsite)

Salary: USD 197,300 - 225,100 per yearly

Minimum experience: 2 years