Model Experimentation Engineer Jobs

1 job found

3d ago

Wayy Research Remote 🌍 Remote

Remote

3d ago

$60 - $70/hr Full time Marketing Python Algorithms

Develop and own algorithms for budget allocation, attribution, and rigorous experimental measurement in a remote AI engineering role.

Build and maintain a multi-armed bandit engine using Thompson Sampling on Beta-Bernoulli reward models, with contextual extensions as needed; tune exploration versus exploitation, handle cold-start scenarios, and monitor non-stationary reward drift.
Implement attribution with Shapley value for multi-touch credit assignment and compare against first-touch, last-touch, and position-based baselines.
Design and operate experiment infrastructure for A/B and multivariate tests, including power analysis, sequential testing with proper alpha spending, and significance testing that remains valid under peeking.
Conduct cross-channel causal measurement using geo holdouts, incrementality tests, and difference-in-differences designs to isolate true interaction effects from correlation.

Strong Python skills with NumPy, SciPy, pandas, and scikit-learn.
Hands-on experience with multi-armed bandits including Thompson Sampling, UCB, and epsilon-greedy; understands regret bounds beyond API usage.
Expertise in hypothesis testing and experimental design: power analysis, multiple comparisons correction, sequential testing, and the peeking problem.
Production-scale A/B testing experience; you have shipped tests, not only analyzed them.
Proficiency in causal inference methods such as DiD, synthetic control, instrumental variables, or incrementality testing.
Advanced SQL skills with PostgreSQL.

You expect a control group before reporting lift numbers and can explain the difference between correlation and causation to a CMO without condescension.

New Model Experimentation Engineer Jobs, daily.