Foundation Model Data Engineer

Sciforium is hiring a Foundation Model Data Engineer to own dataset strategy and engineering for foundation models, from pre-training corpora through post-training alignment data and multimodal expansion.

Responsibilities

• Lead end-to-end creation of pre-training datasets for LLMs, including defining the mix of web data, code, books, and technical papers to improve downstream performance.
• Design and implement pipelines for data cleaning, exact and fuzzy deduplication, and high-quality signal extraction from petabytes of raw, unstructured data.
• Build post-training datasets, including Supervised Fine-Tuning (SFT) instructions, multi-turn dialogues, and preference modeling data for RLHF and DPO.
• Drive acquisition and processing of vision and video data, covering multimodal alignment challenges, video compression, and temporal consistency.
• Develop high-throughput ingestion and transformation tooling using Python, with multiprocessing and multithreading to prevent bottlenecks at massive scale.
• Run deep statistical analyses on training corpora to detect biases, knowledge gaps, and quality regressions to keep the training “diet” mathematically balanced.
• (Added Value) Build pipelines for high-reasoning synthetic data to fill gaps in natural datasets, using existing models for data labeling and refinement.

Requirements

• 5+ years of industry experience in Data Science or Machine Learning, with a track record building and managing datasets for foundation models.
• Expert-level skills for high-performance data engineering, including multiprocessing, multithreading, and efficient memory management.
• Proven experience working with petabyte-scale datasets used to train production-grade LLMs or Large Vision Models.
• Hands-on experience assembling massive LLM training sets from scratch, including raw web crawls (such as Common Crawl) and specialized domain data.
• Experience building datasets for RLHF, DPO, and multi-turn instruction following, including human-labeling workflows and quality gold-sets.
• Mastery of data-at-scale frameworks such as Spark or Ray, plus high-performance data-loading formats such as WebDataset and Parquet.

Technologies

• Python
• multiprocessing
• multithreading
• Spark
• Ray
• WebDataset
• Parquet
• Common Crawl

Benefits

• Medical, dental, and vision insurance
• 401k plan
• Daily lunch, snacks, and beverages
• Flexible time off
• Competitive salary and equity

Nice-to-Haves

• Experience building large-scale image or video datasets from scratch (for example, LAION-style pipelines).
• Familiarity with large-scale crawling of multimodal data, including video processing, codecs, and compression challenges.
• Experience designing complex labeling schemas for reasoning, coding, and mathematical benchmarks.
• A Master’s or PhD in a quantitative field with a focus on data-centric AI or information retrieval.

Location: San Francisco, CA (onsite)

Salary: USD 155,000 - 210,000 per yearly