Senior Lead AI Engineer (LLM Gateway, FM Hosting)

Overview

Capital One is building responsible and reliable AI systems to transform banking. The company has long led in applying machine learning to deliver real-time, personalized customer experiences, backed by strong investments in infrastructure and top AI talent. These efforts position Capital One at the forefront of enterprises adopting AI.

Team

The Intelligent Foundations and Experiences (IFX) team sits at the center of Capital One's AI strategy. We collaborate with partners across the organization to advance AI science and engineering, building proprietary solutions that scale to millions of customers and empower product teams to integrate AI into their offerings.

Responsibilities

• Collaborate with a cross-functional squad of engineers, researchers, technical program managers, and product managers to deliver AI powered products that transform how associates work and how customers interact with Capital One.
• Design, develop, test, deploy, and support AI software components including foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Leverage a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and more.
• Invent and apply state-of-the-art LLM optimization techniques to improve scalability, cost, latency, and throughput of large-scale production AI systems.
• Contribute to the technical vision and long-term roadmap of Capital One's foundational AI systems.

Requirements

• You enjoy building systems, take pride in quality, and are motivated by problems that can change banking for good.
• Passion for staying current with AI research and the ability to translate scientific publications into production-ready techniques.
• Adaptable with a knack for clarifying large, undefined problems, asking deep questions, and clearly communicating findings; you share new ideas even if unproven.
• Deep technical depth in engineering and mathematics, with the ability to spot optimization opportunities across hardware, software, and AI.
• Resilient and capable of forging new paths to meet business goals when the route is unclear.
• Bachelor's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 6 years of AI/ML experience, or Master's degree plus at least 4 years of AI/ML experience.
• At least 6 years of programming experience in Python, Go, Scala, or Java.
• 7 years of experience deploying scalable and responsible AI solutions on cloud platforms (AWS, Google Cloud, Azure, or equivalent private cloud).
• Experience designing, developing, integrating, delivering, and supporting complex AI systems.
• Proven ability to lead and mentor an engineering team and influence cross-functional stakeholders.
• Experience developing AI and ML algorithms or technologies (e.g., LLM Inference, Similarity Search and VectorDBs, Guardrails, Memory) using Python, C++, C#, Java, or Golang.
• Experience optimizing training and inference software to improve hardware utilization, latency, throughput, and cost.
• Strong interest in AI research and practical application of novel techniques in production environments.
• Excellent communication and presentation skills for explaining complex AI concepts to peers.

Technologies

• Python, Go, Scala, Java
• PyTorch, Huggingface
• AWS Ultraclusters, VectorDBs
• Nemo Guardrails

Benefits

• Performance-based incentive compensation (cash bonuses and/or long-term incentives)
• Health, financial and other benefits that support total well-being

Salary and location

Location: McLean, VA (onsite). Salary: USD 229,900 - 262,400 per year.

Note: Salaries by location vary. Other listed locations (Cambridge, MA; New York, NY; San Jose, CA) have the same range for Sr. Lead AI Engineer, while hires in other locations will receive an offer based on the pay range for that location as shown in the offer letter.