EngineerJobs.io
← Back to all jobs

Job Description

Design and operationalize agentic AI for Firm Risk Management in a governed enterprise environment.

Responsibilities

  • Design and build modular, maintainable Python applications, APIs, and services integrating LLMs, enterprise data, and business tools.
  • Develop LangGraph agent workflows with explicit state, conditional routing, tool execution, checkpointing, and human-in-the-loop approval or escalation when needed.
  • Engineer agent context by selecting and assembling relevant instructions, retrieved information, tool outputs, and conversation history within token and access-control constraints.
  • Design, version, and test prompts with structured outputs, grounding instructions, and safe handling of incomplete or conflicting information.
  • Implement short-term and long-term agent memory strategies including persistence, retrieval, summarization, retention, and isolation between users and sessions.
  • Build evaluation frameworks using representative datasets and repeatable regression tests with automated and human evaluation.
  • Assess answer quality across grounding, tool-call correctness, task completion, and failure behavior.
  • Implement LLM and agent observability using Galileo or comparable platforms (execution traces, quality signals, latency, token usage, cost, and failure analysis).
  • Apply guardrails for prompt injection, unsafe tool use, sensitive-data exposure, and unsupported outputs with access controls and auditable execution paths.
  • Collaborate with risk specialists, product owners, data engineers, and platform teams to translate requirements into tested applications and support deployment and troubleshooting.
  • Participate in technical and code reviews, documentation, and engineering guidance while staying active in implementation.

Requirements

  • Strong hands-on Python engineering: application design, asynchronous programming, APIs, automated testing, error handling, and maintainable code.
  • Demonstrated experience implementing AI agents with LangGraph, including stateful orchestration and integration with external tools or services beyond chatbots, tutorials, or conceptual designs.
  • Practical understanding of LLMs, embeddings, retrieval-augmented generation (RAG), vector search, and retrieval-context tradeoffs.
  • Hands-on context engineering, prompt engineering, and agent memory management.
  • Experience designing and implementing evaluation frameworks for LLM/agent applications, including test-set construction and interpreting evaluation results.
  • Experience with Galileo or a comparable LLM/agent evaluation and observability platform, using traces to debug failures and improve applications.
  • Experience delivering and supporting deployed applications, including security, reliability, version control, CI/CD, and production troubleshooting.
  • Ability to explain technical decisions, distinguish personal contributions from team outcomes, and collaborate with business and engineering stakeholders.

Technologies

  • Python
  • LangGraph
  • LLMs
  • Embeddings
  • Retrieval-Augmented Generation (RAG)
  • Vector Search
  • Galileo
  • Snowflake
  • Databricks
  • Model Context Protocol (MCP)
  • GitHub Copilot
  • Claude Code
  • Amp
  • CI/CD
  • Containers

Highly Desirable

  • Knowledge of Model Risk Management, model development/validation lifecycles, model documentation, governance, or model monitoring.
  • Experience in broader risk management, especially credit risk, market risk, or financial-services risk workflows.
  • Experience with Snowflake, Databricks, lakehouse architectures, enterprise data pipelines, or governed knowledge integration.
  • Familiarity with Model Context Protocol (MCP), cloud deployment, containers, and enterprise identity/access controls.
  • Prior experience using AI coding assistants such as GitHub Copilot, Claude Code, or Amp with disciplined review, testing, and secure handling of code and data.

Location: New York, NY (onsite)

Compensation: USD 90 - 90 per hourly

Similar Jobs