EngineerJobs.io
← Back to all jobs

Job Description

Develop and own algorithms for budget allocation, attribution, and rigorous experimental measurement in a remote AI engineering role.

Responsibilities

  • Build and maintain a multi-armed bandit engine using Thompson Sampling on Beta-Bernoulli reward models, with contextual extensions as needed; tune exploration versus exploitation, handle cold-start scenarios, and monitor non-stationary reward drift.
  • Implement attribution with Shapley value for multi-touch credit assignment and compare against first-touch, last-touch, and position-based baselines.
  • Design and operate experiment infrastructure for A/B and multivariate tests, including power analysis, sequential testing with proper alpha spending, and significance testing that remains valid under peeking.
  • Conduct cross-channel causal measurement using geo holdouts, incrementality tests, and difference-in-differences designs to isolate true interaction effects from correlation.

Requirements

  • Strong Python skills with NumPy, SciPy, pandas, and scikit-learn.
  • Hands-on experience with multi-armed bandits including Thompson Sampling, UCB, and epsilon-greedy; understands regret bounds beyond API usage.
  • Expertise in hypothesis testing and experimental design: power analysis, multiple comparisons correction, sequential testing, and the peeking problem.
  • Production-scale A/B testing experience; you have shipped tests, not only analyzed them.
  • Proficiency in causal inference methods such as DiD, synthetic control, instrumental variables, or incrementality testing.
  • Advanced SQL skills with PostgreSQL.

Technologies

  • Python
  • NumPy
  • SciPy
  • pandas
  • scikit-learn
  • PostgreSQL
  • PyMC
  • Stan
  • NumPyro

Benefits

  • Pay: $60.00 - $70.00 per hour
  • Remote

You’ll fit if

  • You expect a control group before reporting lift numbers and can explain the difference between correlation and causation to a CMO without condescension.

Strongly preferred

  • Marketing mix modeling or media measurement background
  • Bayesian methods with PyMC, Stan, or NumPyro
  • Shapley values or cooperative game theory applied to attribution
  • Production ML deployment experience, not just notebooks

Similar Jobs