Senior Machine Learning Engineer, Data Infrastructure
Job Description
Build and evolve a large-scale offline data platform that generates training datasets and orchestrates reliable ML data workflows.
Responsibilities
- Develop big data infrastructure supporting batch and stream processing using technologies such as Flink, Spark, Ray
- Design and operate large-scale data pipelines that produce training datasets for ML training and experimentation
- Integrate training-data pipelines with workflow orchestration systems such as Flyte, Airflow (or similar) for reliable multi-stage training workflows
- Increase reproducibility and observability via dataset validation, monitoring, and automated testing
- Optimize performance and resource utilization across distributed compute systems used for data processing
- Partner with ML engineers to support efficient large-scale experimentation and iterative model development
- Lead architectural improvements to keep offline data pipelines scalable, reliable, and cost-efficient
Requirements
- Experience working with distributed computing frameworks including Flink, Spark, Ray for distributed data processing
- Experience building infrastructure for training data generation, dataset preparation, or ML feature pipelines
- Experience optimizing big data pipelines and infrastructure for cost efficiency
- Strong Python programming skills with experience handling large-scale distributed workloads
- Experience with modern data infrastructure such as data lakes, warehouses, orchestration systems, and streaming platforms
- Strong systems thinking to evaluate performance, scalability, reliability, and cost tradeoffs in distributed systems
- Demonstrated ability to lead technical direction and influence architectural decisions across teams without formal authority
Technologies
- Flink
- Spark
- Ray
- Flyte
- Airflow
- Python
Location and Compensation
- Location: Mountain View, CA (onsite)
- Salary: USD 165,600 - 273,400 per year
Benefits
- Comprehensive health, life, and disability insurance
- Commute subsidy
- Employee stock ownership
- Competitive retirement/pension plans
- Generous vacation and personal days
- Support for new parents through leave and family-care programs
- Office food snacks
- Mental Health and Wellbeing programs and support
- Employee Resource Groups
- Global Employee Assistance Program
- Training and development programs
- Volunteering and donation matching program
- Equity awards
- Participation in company incentive plans (for example, annual discretionary bonuses or sales commissions)
Benefits eligibility, specific offerings, and coverage vary based on country and employment status.