Senior Staff AI Engineer

Zscaler is hiring a Senior Staff AI Engineer to lead two IT AI/ML Engineering groups focused on a shared AI platform and production AI solutions.

Responsibilities

• Lead the AI platform group: own architecture, roadmap, and delivery for shared services including LLM gateways and model routing, AI cost optimization, observability and evaluation, an internal catalog of agents and skills, and long-term memory and context for AI applications.
• Lead the AI solutions and support group: own delivery and quality for AI-powered help and search, rollout and administration of enterprise AI platforms, and L2 support covering triage, escalation, runbooks, and training workshops.
• Anchor a distributed team: collaborate with engineers and contractors across the US and India, coordinate across time zones, run planning and reviews, and maintain bidirectional context and decision flow.
• Plan and prioritize: translate business requests into a priority order, balance team capacity, flag risks early, and report status to engineering leadership and stakeholders.
• Stay hands-on across the stack: design and review APIs, services, and code (front-end and back-end), plus CI/CD pipelines; unblock engineers and set standards for code quality, testing, and documentation.
• Set the quality bar for AI solutions: define release gates, regression checks, and cost and latency targets using evaluation and observability tooling; partner on model selection and tuning with the team that owns models and agents.
• Run operations well: reduce repeat support tickets via automation and documentation, lead incident response and postmortems, and track platform cost, usage, and reliability.
• Partner across the company: coordinate with business owners in sales, support, finance, and engineering, and work with Security, Privacy, and Legal on data handling, access controls, and approvals.
• Grow the team: mentor engineers, assist with hiring, manage contractor performance, and share best practices through documentation and workshops.

Requirements

• 8+ years of software engineering experience, including 3+ years building production applications with LLMs or other ML services.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field (or equivalent experience).
• 3+ years as a technical lead (or equivalent), leading engineers on shared deliverables, ideally across time zones and with contractors.
• Strong full-stack skills: Python and TypeScript/Node.js (or Java/Go), a modern front-end framework such as React, and solid API design.
• Practical experience with LLM applications: prompt and tool design, retrieval-augmented generation, agents and function calling or MCP, and LLM gateways or routing layers.
• Experience with evaluation and observability for AI systems: test sets, regression suites, LLM-as-judge, tracing, and cost and latency tracking.
• Cloud and DevOps skills: AWS (or GCP/Azure), Docker, Kubernetes, and CI/CD with monitoring plus canary and rollback practice.
• Experience running or supporting a production service, including on-call, incident response, runbooks, and L2 escalation.
• Clear written and verbal communication, with a record of managing priorities and expectations with leaders and business partners.

Technologies

• Python, TypeScript, Node.js, Java, Go
• React
• LLM gateways, MCP, retrieval-augmented generation
• AWS, GCP, Azure
• Docker, Kubernetes, CI/CD
• LLM-as-judge, tracing

Benefits

• 401(k)
• Dental insurance
• Flexible schedule
• Retirement plan

Preferred Qualifications

• Experience administering enterprise AI platforms and vendor accounts (Claude, OpenAI, Amazon Bedrock, Vertex), including usage controls and cost governance.
• Experience with vector and graph databases, plus agent memory and context management.
• Experience building search or help experiences, Slack bots, and web widgets integrated with AI services.
• Familiarity with model inference, fine-tuning, and open-weight models (helpful but not required).
• Experience running enablement: workshops, documentation, and internal training for AI tools.
• Experience in regulated environments such as FedRAMP.

Team, Location and Compensation

• Team: IT AI/ML Engineering. Lead two groups and work closely with the group that builds agents and models.
• Day-to-day tools: Python, TypeScript, AWS, Kubernetes, LLM gateways, and evaluation tooling.
• Location: Hybrid, three days per week in San Jose, CA or Bellevue, WA; remote within the US considered for exceptional candidates.
• Work location: Hybrid remote in San Jose, CA 95139.
• Pay: $107,000.00 - $130,000.00 per year.