Senior AI Engineer

Pulley is hiring a Senior AI Engineer to help build AI-powered permitting intelligence, including evaluation and observability for LLM features. This role will own AI-powered product surfaces end-to-end, converting unstructured permitting documents into structured outputs and agent-driven work.

Role Overview

You will be responsible for end-to-end delivery of LLM-powered experiences within permitting workflows. The work spans prompt and pipeline design, defining correctness for real workflows, production deployment, and iterative improvement based on measurable quality and regressions.

Responsibilities

• Own AI-powered features end-to-end, from user-facing interactions and defining what “correct” means for permitting workflows through prompt and pipeline design, evals, deployment, and iteration in production
• Convert unstructured permitting documents, city regulations, and jurisdiction workflows into reliable structured outputs, including extraction, classification, retrieval, and agentic workflows over documents not designed for machine-readability
• Build evaluation and observability foundations to ship LLM-powered features with confidence by defining ground truth, measuring quality and regressions, and determining whether model changes are true improvements
• Use AI agents as part of daily delivery, directing and reviewing agent-driven work at high velocity while owning the quality bar
• Make technical and product decisions that directly impact customers and their projects
• Raise the engineering bar for building with LLMs through design review, mentorship, and the standards set in personal work

Required Qualifications

• 4+ years of software engineering experience, with a substantial portion building production LLM or ML systems
• Experience owning an LLM-powered product surface end-to-end, from requirements through production, including data quality, eval design, cost and latency, and failure handling
• Hands-on production experience with large language models, including prompting, retrieval-augmented generation, structured extraction, tool use, and agentic workflows, plus judgment on when each approach is appropriate
• Experience designing evals and building reliability for LLM-powered production features
• Experience building with AI coding agents where agents performed substantial implementation under your direction
• Ability to architect durable systems while making pragmatic tradeoffs
• Based in the San Francisco Bay Area and willing to work in person 4 days per week

Technology Stack

• LLM, ML
• Retrieval-augmented generation
• TypeScript, React
• Google Cloud
• OCR
• Vision-language models

Compensation and Benefits

• Salary: USD 230,000 - 280,000 per year
• Equity: Offers Equity
• Compensation: $230K – $280K • Offers Equity

Location and Employment Details

• Location: San Francisco, CA (onsite)
• Employment type: Full time
• Department: Engineering

What You Bring

• Comfort with ambiguity, preferring to define the right problem over executing a fixed spec
• Product mindset with a focus on whether the solution solves customer needs, including communicating with users
• Rigorous approach to what “working” means, using evals rather than demos and building measurement before building features
• Strong views on quality and velocity, seeking tools, abstractions, and processes that support both
• Ownership mindset to fix broken or missing components rather than escalating

Nice to Have

• Experience with document understanding at scale, including OCR, layout-aware parsing, or vision-language models for scanned PDFs, drawings, or forms
• Experience fine-tuning models or building data pipelines to generate training and eval sets from real-world usage
• Experience in construction tech, govtech, proptech, or other domains with messy real-world documents and processes
• Modern full-stack experience with TypeScript, React, and Google Cloud, plus interest in application code that delivers AI features to users
• Startup experience contributing to team building, not only the product
• Experience mentoring engineers or leading technical direction across teams