Webdataset Engineer Jobs

1 job found

Today

Foundation Model Data Engineer

Sciforium San Francisco, CA

San Francisco, CA

Today

$155k - $210k/yr Full time Ai Foundation Models Ai/ml Data Engineering Big Data

Apply Now

Job Description

Sciforium is hiring a Foundation Model Data Engineer to own dataset strategy and engineering for foundation models, from pre-training corpora through post-training alignment data and multimodal expansion.

Responsibilities

Lead end-to-end creation of pre-training datasets for LLMs, including defining the mix of web data, code, books, and technical papers to improve downstream performance.
Design and implement pipelines for data cleaning, exact and fuzzy deduplication, and high-quality signal extraction from petabytes of raw, unstructured data.
Build post-training datasets, including Supervised Fine-Tuning (SFT) instructions, multi-turn dialogues, and preference modeling data for RLHF and DPO.
Drive acquisition and processing of vision and video data, covering multimodal alignment challenges, video compression, and temporal consistency.
Develop high-throughput ingestion and transformation tooling using Python, with multiprocessing and multithreading to prevent bottlenecks at massive scale.
Run deep statistical analyses on training corpora to detect biases, knowledge gaps, and quality regressions to keep the training “diet” mathematically balanced.
(Added Value) Build pipelines for high-reasoning synthetic data to fill gaps in natural datasets, using existing models for data labeling and refinement.

Requirements

5+ years of industry experience in Data Science or Machine Learning, with a track record building and managing datasets for foundation models.
Expert-level skills for high-performance data engineering, including multiprocessing, multithreading, and efficient memory management.
Proven experience working with petabyte-scale datasets used to train production-grade LLMs or Large Vision Models.
Hands-on experience assembling massive LLM training sets from scratch, including raw web crawls (such as Common Crawl) and specialized domain data.
Experience building datasets for RLHF, DPO, and multi-turn instruction following, including human-labeling workflows and quality gold-sets.
Mastery of data-at-scale frameworks such as Spark or Ray, plus high-performance data-loading formats such as WebDataset and Parquet.

Technologies

Python
multiprocessing
multithreading
Spark
Ray
WebDataset
Parquet
Common Crawl

Benefits

Medical, dental, and vision insurance
401k plan
Daily lunch, snacks, and beverages
Flexible time off
Competitive salary and equity

Nice-to-Haves

Experience building large-scale image or video datasets from scratch (for example, LAION-style pipelines).
Familiarity with large-scale crawling of multimodal data, including video processing, codecs, and compression challenges.
Experience designing complex labeling schemas for reasoning, coding, and mathematical benchmarks.
A Master’s or PhD in a quantitative field with a focus on data-centric AI or information retrieval.

Location: San Francisco, CA (onsite)

Salary: USD 155,000 - 210,000 per yearly

View full job page →

Get Webdataset Engineer Jobs alerts

New Webdataset Engineer Jobs, daily.