Senior Lead AI Engineer (LLM Gateway, FM Hosting)

Overview

Capital One is committed to building reliable and responsible AI systems that transform banking for good. The company has a history of leveraging machine learning to provide real time, personalized experiences, supported by strong infrastructure and top talent to stay at the forefront of AI in enterprise settings.

Team Description

The Intelligent Foundations and Experiences team sits at the heart of turning AI visions into reality at Capital One. We collaborate with partners across the organization to push the boundaries of AI engineering, developing proprietary solutions that drive value for millions of customers. Our models and platforms empower product teams to incorporate AI into their products and experiences.

Responsibilities

• Collaborate with a cross functional group of engineers, researchers, program managers, and product managers to deliver AI driven products that transform how associates work and how customers engage with Capital One.
• Design, build, test, deploy, and maintain AI software components such as foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Utilize a broad mix of Open Source and SaaS AI technologies including AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and more.
• Develop and apply advanced LLM optimization techniques to enhance performance, scalability, cost, latency, and throughput of large scale production AI systems.
• Contribute to the technical vision and long term roadmap for foundational AI systems at Capital One.

Technologies

• AWS Ultraclusters
• Huggingface
• VectorDBs
• Nemo Guardrails
• PyTorch
• Python
• Go
• Scala
• Java

Benefits

• Health benefits
• Performance-based incentive compensation including cash bonuses and long term incentives

The Ideal Candidate

• Enjoys building systems with a strong focus on quality and has a desire to do the right thing while improving banking through technology.
• Keeps up with the latest AI research and can translate scientific findings into practical production techniques.
• Works well in ambiguous situations, asking questions to uncover root causes and clearly communicating findings; willing to share new ideas even when unproven.
• Is deeply technical with a solid foundation in engineering and mathematics, and can spot optimization opportunities across hardware, software, and AI.
• Actively pursues new paths to achieve business goals when the route is not obvious and remains resilient through challenges.

Basic Qualifications

• Bachelor's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or a related field plus at least 6 years of AI/ML development experience, or a Master’s degree in the same fields plus at least 4 years of AI/ML experience.
• At least 6 years of programming experience in Python, Go, Scala, or Java.

Preferred Qualifications

• Seven years of experience deploying scalable and responsible AI solutions on cloud platforms (AWS, Google Cloud, Azure, or equivalent private cloud).
• Experience designing, developing, integrating, delivering, and supporting complex AI systems.
• Ability to lead and mentor an engineering team and influence cross functional stakeholders.
• Experience developing AI/ML algorithms or technologies such as LLM inference, similarity search and vector databases, guardrails, and memory using Python, C++, C#, Java, or Golang.
• Experience optimizing training and inference software to improve hardware utilization, latency, throughput, and cost.
• Strong interest in AI research and systems, with skill in applying novel techniques in production.
• Excellent communication and presentation skills, capable of articulating complex AI concepts to peers.