Off Policy Evaluation Engineer Jobs

1 job found

Today

Machine Learning Engineer, Applied Research

Whatnot Seattle, WA 🇺🇸 Remote

Seattle, WA

Today

$210k - $300k/yr Full time Artificial Intelligence Data Analysis Data Science

Apply Now

Job Description

Lead applied machine learning research that improves live marketplace dynamics at Whatnot, from hypothesis-driven studies through production experimentation.

Responsibilities

Own applied research across marketplace dynamics, including simulation, auction and allocation mechanics, long-term objective modeling, exploration and information value, and marketplace experimentation methods
Drive ideas from hypothesis to production using literature review, prototyping, offline validation, shadow testing, and online experiments launched through partner teams in Discovery and the Seller org
Build system-level models of marketplace behavior, including learned simulators to estimate segment-level effects of ranking and policy changes, plus surrogate models for long-term marketplace outcomes
Model Whatnot’s real market mechanics, covering auction and bidding dynamics and discovery exposure allocation treated as a portfolio problem, including allocation to rising sellers
Improve evaluation for changes in a multi-sided live marketplace using off-policy evaluation, switchback and interference-robust experiment designs, and variance reduction techniques
Strengthen Whatnot’s external technical presence via publications, open-source contributions, and public benchmarks

Requirements

5+ years of industry experience building and deploying ML models to solve user problems at scale
Deep expertise in at least one area: recommendation systems, causal inference, off-policy evaluation, reinforcement learning and bandits, auction or mechanism design, or marketplace experimentation
Proven track record applying scientific methods to real-world problems on consumer-scale data
Advanced proficiency in Python, SQL, and common ML frameworks such as PyTorch and XGBoost
Strong grounding in applied statistics, experiment design, and theoretical machine learning
Strong communication and leadership skills, including influencing roadmaps and aligning cross-functional teams in a remote environment
Experience with two-sided marketplaces, ads and auction systems, or pricing
Experience building simulators or economic models of platform behavior

Technologies

Python
SQL
PyTorch
XGBoost

Compensation

Full-Time (Salary) US-based applicants: $210,000/year to $300,000/year + benefits + equity
Final salary depends on level, relevant experience, skills, and expertise; the listed range covers base salary only
Compensation Range: $210K - $300K

Benefits

Flexible Time Off Policy and company-wide holidays including a spring and winter break
Health Insurance options including Medical, Dental, Vision
Work From Home Support
Home office setup allowance
Monthly allowance for cell phone and internet
Care benefits
Monthly allowance for wellness
Annual allowance towards Childcare
Lifetime benefit for family planning, such as adoption or fertility expenses
Retirement: 401k offering for Traditional and Roth accounts in the US (employer match up to 4% of base salary) and Pension plans internationally
Monthly allowance to dogfood the app
Parental Leave: 16 weeks of paid parental leave + one month gradual return to work

Location: Seattle, WA (remote)

Summary: Lead applied ML research for live marketplace dynamics at Whatnot, moving from hypothesis to production and experimentation. Build models and methods for simulation, auction and allocation mechanics, multi-sided evaluation, and contribute to external technical presence.

Please Note: Whatnot will only contact you through official @whatnot.com email addresses. If you see an email impersonating a Whatnot recruiter, disregard and report it as spam.

EOE: Whatnot is an Equal Opportunity Employer. They value diversity and do not discriminate on protected characteristics under local law.

View full job page →

Get Off Policy Evaluation Engineer Jobs alerts

New Off Policy Evaluation Engineer Jobs, daily.