Lead AI Engineer (FM Hosting, LLM Inference)

This senior AI engineering role on Capital One's IFX team centers on foundation model training, large language model inference, and the end-to-end design, development, and deployment of AI powered products for associates and customers.

Job Details

• Location: New York, NY (onsite)
• Salary: USD 215,200 - 245,600 per year
• Minimum Experience: 2 years
• Education: Master's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields

Responsibilities

• Collaborate with a cross functional team of engineers, research scientists, technical program managers, and product managers to deliver AI powered products that reshape how colleagues work and how customers engage with Capital One.
• Design, develop, test, deploy, and support AI software components including foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Utilize a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and more.
• Develop and apply advanced LLM optimization techniques to enhance scalability, cost efficiency, latency, and throughput of大型 production AI systems.
• Contribute to the technical vision and the long term roadmap of foundational AI systems at Capital One.

Requirements

• A bachelor's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 4 years of experience developing AI and ML algorithms or technologies; or a master's degree in a related field plus at least 2 years of such experience
• Proficiency in programming with Python, Go, Scala, or Java (4+ years)
• Experience deploying scalable and responsible AI solutions on cloud platforms (AWS, Google Cloud, Azure, or equivalent private cloud) for 6+ years
• Experience designing, developing, delivering, and supporting AI services
• Experience developing AI and ML algorithms or technologies (for example LLM Inference, Similarity Search and VectorDBs, Guardrails, Memory) using Python, C++, C#, Java, or Golang
• Experience optimizing training and inference software to improve hardware utilization, latency, throughput, and cost
• Strong interest in and up-to-date knowledge of the latest AI research and AI systems, with practical application of novel techniques in production

Technologies

• Python
• Go
• Scala
• Java
• AWS Ultraclusters
• Huggingface
• VectorDBs
• Nemo Guardrails
• PyTorch

Benefits

• Health benefits
• Financial benefits
• Incentives include performance-based incentive compensation (cash bonuses and/or long-term incentives)