AI Engineer

The AI Engineer role at Bain’s Coro team focuses on building AI-infused software and data products, including LLM-driven features and agentic workflows that move from proof of concept through scaled deployments.

Location

New York, NY (onsite)

Compensation

USD 128,500 - 171,500 per yearly. Compensation includes base salary, an annual discretionary performance bonus, and a 401(k) plan with an annual employer contribution as described below, along with Bain’s best-in-class benefits package.

• In Massachusetts, New York, District of Columbia, Georgia, Illinois, Texas, Washington, and California, the good-faith, reasonable annualized full-time salary range is $128,500-$171,500.
• Placement within this range varies based on experience, education, licensure/certifications, training, and skill level.
• The role may also be eligible for other elements of discretionary compensation.

Role Summary

Within Bain’s Coro team, you will build LLM-enabled applications and workflows that support commercial teams, including retrieval and knowledge pipelines, MLOps and GenAIOps practices, and secure enterprise integration. Work can include POCs, MVPs, and production-scale deployments.

Key Responsibilities

• Design and develop GenAI applications (for example, copilots, workflow automation, decision support) using modern LLM stacks.
• Implement agentic workflows when they add clear value, including tool use, multi-step execution, and human-in-the-loop controls, with focus on reliability, safety, and clear failure modes.
• Design and build advanced search, retrieval, and knowledge pipelines across diverse data structures and stores, including hybrid search, vector stores, and graph databases or knowledge graphs; cover indexing strategies, metadata design, relevance tuning and reranking, freshness, caching, access controls, and source attribution.
• Build robust agent capabilities including context engineering, short-term and long-term memory and state management, orchestration, routing, and tool integration patterns.
• Integrate solutions into enterprise environments and workflows via APIs and data systems and collaboration tools, balancing quality, latency, cost, privacy, and adoption.
• Translate ambiguous client needs into clear technical requirements, tradeoffs, and delivery plans.
• Build ML solutions end-to-end, including data preparation, feature engineering, model selection, training, validation and testing, and performance analysis.
• Apply appropriate methods across classical ML and deep learning (including sequence, text, and image models when relevant).
• Create reproducible training and evaluation pipelines with versioning, experiment tracking, robust validation, and documentation.
• Demonstrate fluency with modern deep learning and LLM concepts, including transformer fundamentals and LLM pre-training versus post-training (such as instruction tuning and preference optimization approaches).
• Write clean, testable, maintainable code and deliver AI services through the full SDLC: build, test, deploy, monitor, and iterate.
• Implement MLOps and GenAIOps practices including CI/CD, reproducibility, environment parity, and model/prompt/agent versioning, with operational readiness.
• Build evaluation and observability for GenAI and agentic systems, including tracing and instrumentation, regression test suites, automated scoring when appropriate, and iteration loops for prompt and policy optimization.
• Design for secure enterprise deployment with access controls, auditability, and data handling for sensitive and PII data, plus responsible AI guardrails.
• Create reusable components and accelerators (templates, evaluation harnesses, connectors, orchestration patterns) that scale across client contexts.
• Communicate with technical and non-technical stakeholders, lead working sessions, present recommendations, and write crisp technical documentation.
• Work with Bain consultants to prioritize technical decisions that unlock business value.
• Support proposal shaping and scoping, including effort sizing, architecture options, risk assessment, and delivery roadmaps.

Required Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience).
• 3-5+ years of professional AI/ML engineering experience (or equivalent), with strong backend engineering fundamentals.
• Strong proficiency in Python, experience building APIs/services (REST/gRPC), and integrating with enterprise systems.
• Hands-on experience building LLM-powered applications, with delivery considerations including latency, cost, reliability, and security.
• Experience building advanced retrieval/search systems such as hybrid retrieval, vector search, and reranking, and comfort working across multiple data stores (vector, graph, relational/document/search).
• Experience implementing agentic patterns (context management, tool integration, orchestration, memory/state handling) using modern frameworks such as LangGraph, OpenAI Agents SDK, or Pydantic AI, or custom agent loops, with judgment on where agentic approaches apply.
• Experience creating reusable skills/tools/services for agent use, including MCP, and schema validation (such as Pydantic) to enforce reliable data contracts.
• Strong engineering practices including testing, code review, version control, CI/CD, and performance profiling.
• Experience deploying and operating services on AWS, GCP, and/or Azure, including environment management, reliability, observability, and scaling.
• Experience with Docker and Kubernetes (or equivalent orchestration) and operating services in production, including debugging, performance, and resilience.
• Proven ability to implement security, privacy, and governance requirements for AI systems, including authentication/authorization, access controls, PII/sensitive data handling, and enterprise risk controls.
• Experience training, validating, and testing ML models, with understanding of overfitting, generalization, and evaluation methodology.
• Practical experience with feature engineering and data preprocessing for real-world datasets.
• Familiarity with a broad set of ML algorithms (classical ML and deep learning) and selecting methods matching business and data constraints.
• Familiarity with deep learning frameworks such as PyTorch/TensorFlow and ML lifecycle tooling such as experiment tracking and model registry, plus feature store concepts.
• Ability to work in ambiguity and complexity, manage priorities, and deliver outcomes independently or with a team.
• Excellent interpersonal and communication skills to explain technical decisions, tradeoffs, and results to mixed audiences.
• Strong stakeholder management skills and comfort working directly with clients.

Technologies

• English
• Python
• REST, gRPC
• LLM stacks, LangGraph
• OpenAI Agents SDK, Pydantic AI, Pydantic
• MCP
• AWS, GCP, Azure
• Docker, Kubernetes
• Vector stores, graph databases, knowledge graphs
• Hybrid retrieval, vector search, reranking
• Transformer fundamentals
• Instruction tuning, preference optimization approaches
• PyTorch, TensorFlow
• CI/CD
• APIs, data pipelines

Benefits

• Medical, dental, and vision programs (100% individual employee premiums).
• Generous paid time off, including parental leave, sick leave, and paid holidays.
• Fully vested 401(k) company contribution.
• Paid Life and Long-Term Disability insurance.
• Annual fitness reimbursements.
• 401(k) plan with an annual employer contribution based on years of service: 4.5% 401(k) company contribution, increasing after 3 years of service, and 100% vested upon start date.
• Annual discretionary performance bonus.

Preferred Qualifications

• MBA or PhD in a technical field.
• Background in consulting, professional services, or B2B analytics environments.
• Experience working with major AI ecosystem partners on real client deployments.