EngineerJobs.io
← Back to all jobs

Job Description

The Lead AI Engineer will help design, build, and operationalize a shared enterprise platform that supports safe, scalable development and deployment of production AI applications and agents. The role is hybrid and anchored on Microsoft Azure and Microsoft Foundry, with a focus on durable engineering, security, and developer enablement.

Location and Employment

  • Location: Boston, MA (hybrid)
  • Salary: USD 90,000 - 180,000 per yearly
  • Minimum experience: 8 years

Key Responsibilities

  • Design and implement shared platform capabilities, including model access, agent runtimes, orchestration, gateways, MCP and tool integration, retrieval services, agent and tool registries, observability, evaluations, guardrails, and FinOps controls.
  • Build reusable platform capabilities that let application and business teams deliver secure, observable, and resilient production solutions through streamlined self-service.
  • Establish patterns, frameworks, and reference implementations for building single-agent and multi-agent applications using Microsoft Agent Framework or LangGraph.
  • Define standards for agent-to-agent interoperability, including prompts, context, state, memory, human approvals, kill switches, retries, fallback, versioning, service-level objectives, and controlled promotion to production.
  • Create a governed, model-agnostic access layer covering authentication, identity propagation, routing, quotas, rate limits, failover, usage attribution, policy enforcement, and provider abstraction.
  • Support model access across Azure, AWS, and approved on-premises services, including patterns such as Azure API Management in front of Foundry model deployments.
  • Build secure MCP servers and gateways, and define patterns for hosting, OAuth, delegated authorization, on-behalf-of identity, token exchange, tool discovery, registration, and allowlisting.
  • Partner with Identity, Cybersecurity, and Cloud Engineering teams to implement enterprise identity controls.
  • Develop workload and per-agent identity with lifecycle management and Conditional Access using Microsoft Entra ID and Entra Agent ID.
  • Implement role-based access control, least privilege, app-registration automation, secrets elimination, cross-cloud federation, and entitlement-aware data access.
  • Design and operate agent evaluation frameworks and regression pipelines to measure task success, tool-use correctness, groundedness, safety, and use-case-specific outcomes, and use evaluations as automated quality gates for production promotion.
  • Instrument agent runs and LLM request flows, including retrieval, tool calls, latency, errors, quality, policy events, token usage, and cost using OpenTelemetry-based standards where possible.
  • Implement guardrails and safeguards such as content-safety controls, prompt-injection defenses, data-loss protections, red-team testing, audit trails, and agent and model inventories.
  • Align platform capabilities to applicable regulatory and model-risk requirements, including NIST AI Risk Management Framework, SR 11-7, and the EU AI Act where relevant.
  • Treat the platform as a product using Infrastructure as Code, CI/CD, automated promotion workflows from test to stage to production, policy as code, SDKs, templates, reference implementations, documentation, and self-service onboarding.
  • Own significant platform capabilities from architecture through production operation, while making pragmatic build-versus-buy decisions and evaluating rapidly changing AI technologies without pursuing every new framework.
  • Establish engineering standards, reference architectures, code-review practices, and production-readiness criteria.
  • Lead technically complex initiatives across AI, Cloud, Security, Data, Architecture, and product engineering teams, and mentor AI and platform engineers to reduce dependency on individual specialists or external resources.
  • Communicate trade-offs, risks, sequencing, and investment needs clearly to engineers and senior leaders.

Required Qualifications

  • 8 or more years of professional software, cloud, or platform engineering experience.
  • 3 or more years of experience building production machine learning or advanced data platforms.
  • 1 to 2 or more years of experience delivering production LLM or agent-based systems.
  • Demonstrated technical leadership for significant production systems.
  • Strong hands-on software engineering skills in Python and at least one additional language such as TypeScript, Java, C#, or Go.
  • Deep experience with several of: LLM APIs, agentic architecture, tool or function calling, retrieval-augmented generation, embeddings, vector retrieval, context engineering, orchestration, model routing, observability, and MCP.
  • Hands-on experience designing and operating agent evaluations, including test datasets, automated and LLM-as-judge scoring, tool-use and trajectory assessment, safety and groundedness checks, and regression gating within CI/CD.
  • Strong cloud platform engineering experience, including Microsoft Azure technologies such as Microsoft Foundry, Foundry Agent Service, model deployments, Azure AI services, API Management, Entra ID, private networking, Key Vault, Application Insights, Azure Monitor, Infrastructure as Code, and CI/CD.
  • Experience building platform capabilities used by other engineering teams, including APIs, SDKs, self-service infrastructure, developer tooling, runtime controls, telemetry, and production support.
  • Strong understanding of distributed systems, API design, reliability, security, identity, networking, and operational risk.

Technology Stack

  • Microsoft Azure, Microsoft Foundry, Foundry Agent Service, Microsoft Agent Framework, LangGraph
  • Microsoft Entra ID, Entra Agent ID
  • Azure API Management, MCP, OAuth
  • OpenTelemetry
  • NIST AI Risk Management Framework, SR 11-7, EU AI Act
  • Python; TypeScript; Java; C#; Go
  • CI/CD, Infrastructure as Code, policy as code
  • Azure AI services, Key Vault, Application Insights, Azure Monitor, Azure AI Search
  • Microsoft Purview, Microsoft Fabric
  • Kubernetes, AKS, Bicep, Terraform
  • GitHub Actions, Azure DevOps
  • AWS

Benefits

  • retirement plan
  • health and wellbeing
  • dental coverage
  • vision coverage
  • pharmacy coverage
  • health savings account
  • flexible spending accounts and commuter program
  • employee assistance program
  • life and disability insurance
  • adoption assistance
  • back-up childcare
  • tuition/CFA reimbursement
  • paid time off (leave of absence, paid holidays, volunteer, sick and vacation time)
  • discretionary Corporate Bonus and/or Incentives (if eligible)
  • flexible work environment (work from offices 4 days a week with flexibility to work remotely 1 day a week)

Preferred Qualifications

  • Experience with agent orchestration frameworks such as Microsoft Agent Framework or LangGraph.
  • Familiarity with agent-to-agent interoperability and related standards.
  • Experience developing MCP servers and implementing OAuth, on-behalf-of identity, and token-exchange patterns.
  • Experience implementing Azure API Management or another AI gateway.
  • Experience with OpenTelemetry, Kubernetes or AKS, Bicep or Terraform, and GitHub Actions or Azure DevOps.
  • Familiarity with Microsoft Purview, Microsoft Fabric, Azure AI Search, and enterprise data-security controls.
  • Experience with advanced evaluation practices, including adversarial testing, red-team testing, online evaluation, model comparison, agent traceability, quota management, and AI FinOps.
  • Experience with AWS and hybrid or multi-cloud architecture.
  • Experience in financial services or another highly regulated environment.

Critical Capabilities

  • Platform builder: Creates shared, opinionated capabilities that application teams can consume through stable interfaces and automation.
  • Production engineer: Designs for security, resilience, scale, traceability, operability, and lifecycle management from the beginning.
  • AI technologist: Applies deep understanding of agent behavior, context, tools, retrieval, evaluation, and model trade-offs beyond cloud infrastructure.
  • Enterprise integrator: Coordinates identity, entitlements, cross-cloud connectivity, internal APIs, and regulated-data controls.
  • Technical leader: Sets direction through working software, clear standards, strong reviews, mentorship, and credible decision-making.

Similar Jobs