Soulside AI is building AI that helps make behavioral health documentation more reliable. As an Applied AI Engineer, you will own the model layer that powers trustworthy clinical documentation, working closely with clinical experts and the CTO while you create post-training pipelines, evaluation systems, and production monitoring to improve quality at scale.
Role scope: build and iterate on the end-to-end modeling workflow, from post-training through rigorous evaluation to serving and ongoing monitoring. The position is based in San Francisco, CA (onsite) and reports to the CTO.
What you will do
- Develop post-training pipelines on open-source models for domain-specific clinical tasks, including SFT, preference optimization (DPO/RLHF), LoRA/adapters, and distillation.
- Fine-tune, deploy, and serve models across managed inference and fine-tuning platforms such as Fireworks AI, Baseten, and Together AI, making pragmatic build-vs-buy decisions for each workload.
- Create and maintain rigorous evaluation sets for high-stakes tasks like clinical reasoning and AI note generation, including metric design, gold-standard data curation, and automated plus human-in-the-loop eval harnesses.
- Turn evaluation results into a fast iteration loop that catches regressions before release and quantifies the impact of model and prompt changes.
- Optimize the full LLM pipeline, including prompting, retrieval, structured output validation, plus latency and cost.
- Collaborate with clinical experts to translate documentation and compliance requirements into model behavior and evaluation criteria.
- Monitor models in production for quality, drift, and failure modes, feeding learnings back into training data and evals.
What you bring
- 3+ years in applied ML/AI engineering, or a Master’s degree in a related field, with hands-on experience taking LLM-based systems into production.
- Practical experience with post-training / fine-tuning open-source models (for example Llama, Qwen, Mistral) using SFT, LoRA/PEFT, or preference-based methods.
- Experience serving or fine-tuning on managed platforms such as Fireworks AI, Baseten, or Together AI (or comparable infrastructure).
- Proven ability to build evaluation frameworks for LLM tasks with measurable quality targets.
- Strong Python and familiarity with modern ML tooling such as PyTorch and Hugging Face.
- Solid grounding in prompt engineering and structured-output validation.
- Ability to thrive in a fast-paced, remote startup environment and communicate clearly with both technical and clinical teammates.
- Visa sponsorship is available for the right candidate, including H-1B and O-1.
Benefits
- Salary range: $150,000–$200,000 per year, plus equity with significant upside potential as a founding team member.
- Comprehensive health, dental, and vision insurance.
- Flexible, remote-first culture.
- Direct access to founders with influence over technical direction.
- Professional development budget and conference attendance.
- The opportunity to build AI that measurably improves mental health care at scale.
Bonus points
- Experience with healthcare, clinical NLP, or other high-stakes or regulated domains.
- Familiarity with HIPAA and handling sensitive clinical data.
- RAG systems, retrieval quality tuning, or long-context document workflows.
- Experience with LLM observability, monitoring, and drift detection in production.
- Data pipeline and labeling workflow experience for curating high-quality training and eval sets.
- Open-source contributions in the ML/LLM ecosystem.
Technologies you will work with
Python, PyTorch, Hugging Face, Fireworks AI, Baseten, Together AI, Llama, Qwen, Mistral, SFT, LoRA, PEFT, DPO, RLHF, LoRA/adapters, distillation, retrieval, structured output validation.
How to apply
Send your resume and a short note to anurag@soulside.ai. Include details about a model or pipeline you took to production and how you determined it was actually working.
