Llm Observability

Llm Observability Engineer Jobs

2 jobs found

Today

Staff Applied AI Engineer

DOSS San Francisco, CA

San Francisco, CA

Today

$230k - $260k/yr Full time Agentic Ai Agentic Workflows Ai Agent

Apply Now

Job Description

The Staff Applied AI Engineer role at DOSS is responsible for end-to-end applied AI that powers Dossbot, AI-assisted schema and workflow generation, and agentic automations grounded in live operational data. This position focuses on production architecture, evaluation and safety, and building enterprise workflows that remain tenant-isolated and auditable.

Responsibilities

Own the applied AI architecture end-to-end, including model selection, orchestration, RAG over the unified operational data model, production evals and guardrails, plus cost and latency budgets.
Develop Dossbot as the primary operations interface, enabling natural-language search, analysis, and automation grounded in live supply-chain, finance, and ops data.
Build AI-assisted tooling that uses foundation models to propose, validate, and apply customer schemas and workflow definitions directly on live production tenants.
Design agentic workflows that operate within enterprise guardrails, including permission-aware execution, auditability, tenant isolation, and safe runs against a customer system of record.
Stand up evaluation and observability infrastructure to measure quality, detect regressions, and proactively identify anomalous or risky emergent behavior.
Partner with product, design, and the Forward Deployed team to deliver high-leverage AI use cases.
Mentor and shape engineering culture as the center of gravity for applied AI, raising standards for how the team builds, evaluates, and ships LLM-powered systems.

Requirements

Shipped LLM-powered features or agents to production and operated them at scale, with experience beyond prototypes or demos.
Strong hands-on experience with foundation model APIs, RAG pipelines, tool use and function calling, prompt and context engineering, and fine-tuning when it is warranted.
Experience building evaluation harnesses and guardrails for non-deterministic systems working with business-critical data.
Solid backend fundamentals, including system design, Postgres, and cloud services for reliable multi-tenant web applications.
Ability to go from 0 to 1 in ambiguous settings, with sound judgment on when AI is the right approach and when it is not.
Collaborative working style, bringing positivity, openness, and curiosity to the team.
Growth mindset focused on continuous learning and improvement for self, team, and company.
Craftsmanship and attention to quality in delivered systems.
Customer-first orientation focused on the customer experience journey and building a product intended to be genuinely delightful.
Pragmatic decision-making based on the situation, not theory.
Low-ego collaboration, with effectiveness in environments where team members support one another.

Technologies

RAG
Postgres
Foundation model APIs
Tool use and function calling
Fine-tuning
LLM

Compensation

USD 230,000 - 260,000 per year.

Location

San Francisco, CA (onsite).

View full job page →

1mo ago

Machine Learning Engineer I

Indeed Remote 🌍 Remote

Remote

1mo ago Closed

$118k - $245k/yr Full time Agent Orchestration Agentic Systems AI

This position is no longer accepting applications (closed Sep 16, 2026).

Job Description

Build and deploy machine learning and AI capabilities for Indeed’s Sourcing team, with a focus on production reliability and LLMOps.

Responsibilities

Develop a range of machine learning models, moving from prototype stages to production
Apply best practices for model evaluation, monitoring, and MLOps
Work with cross-functional teams on model inference and product applications for Sourcing
Contribute to code review and process improvements for software engineers and Machine Learning Engineers, and document your work
Use technical and product thinking to drive ownership of team priorities and Indeed’s mission to help people get jobs

Requirements

Bachelor’s degree in Computer Science, Mathematics, Statistics, or a related field plus at least 2 years of related experience; or an advanced degree with no experience
Experience with agent orchestration frameworks and LLM observability tools
Strong domain knowledge for training, fine-tuning, and evaluating language models
Knowledge and practical experience with ML/LLM/Deep Learning models (for example: Torch, Tensorflow, Huggingface, XGBoost)
Python and SQL programming experience

Tech Stack

Python
SQL
Torch
Tensorflow
Huggingface
XGBoost
LLM observability tools
Agent orchestration frameworks

Salary Range

Tier 1 (United States of America): $118,000 - $176,000 USD per year
Tier 2 (United States of America): $130,000 - $196,000 USD per year
Tier 3 (United States of America): $143,000 - $215,000 USD per year
Tier 4 (United States of America): N/A
Tier 5 (United States of America): $163,000 - $245,000 USD per year

Salary Range Transparency: The range reflects the minimum and maximum compensation for the role. Offers are typically made between the range minimum and the midpoint. Actual compensation depends on job-related skills, experience, and expertise evaluated during the interview process. The range(s) is one component of Indeed’s total compensation package. Other rewards may include quarterly bonuses, Restricted Stock Units (RSUs), a Paid Time Off policy, and region-specific benefits. Compensation can vary by location, which is grouped into geographic pay tiers. Candidates can view geographic pay tiers at https://www.indeed.com/careers/paytiers, and recruiters can confirm how location is considered for a specific role.

Benefits

Medical, dental, vision, disability, and life insurance
Enrollment in the company’s 401(k) plan
Equity-based incentive program
Open paid time off
11 paid holidays a year
Up to 26 weeks of paid parental leave
Paid time off policy

Location

Remote

Apply

Deadline to apply: 8/17/2026
Job postings may be extended at the hiring team’s discretion based on application volume
Reference ID: 47241

AI Notice

Indeed uses artificial intelligence (AI) tools to assist in screening, assessment, and selection of applicants by analyzing information provided in resumes and applications
AI use does not replace human decision-making
Unless otherwise notified, Indeed does not use AI constituting an AEDT or an ADMT as defined in applicable laws

View full job page →

Llm Observability Engineer Jobs

Staff Applied AI Engineer

Machine Learning Engineer I

Get Llm Observability Engineer Jobs alerts