AI Data Engineer – Platform & Analytics
Analytics
Apache Airflow
Artificial Intelligence
Automation
Big Data
Bigdata
Cloud
Cloud Platform
Data
Data Analysis
Data Analytics
Data Architecture
Data Build Tool
Data Engineer
Data Integration
Data Lake
Data Lakehouse
Data Management
Data Modeling
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Database
Databases
Databricks
ETL
Iceberg
Large Language Models
Machine Learning
Snowflake
Spark
SQL
Job Description
AMD seeks an AI Data Engineer to architect and implement a medallion data lakehouse and end-to-end data pipelines for manufacturing, IoT, and yield-analysis data, leveraging AI tooling to accelerate development across multiple engines.
Responsibilities
- Architecture & Design: Architect and maintain a scalable medallion data lake house structure (Bronze, Silver, Gold layers) utilizing Apache Iceberg table formats.
- Multi-Engine Data Consumption: Build and optimize curated, semantic Gold-layer data products designed for seamless, high-performance consumption across an open ecosystem of multiple compute and query engines including Snowflake, Databricks (Spark), Trino/Starburst, AWS Athena, and Presto.
- Pipeline Orchestration: Design, develop, and manage complex, resilient data workflows and DAGs using Apache Airflow.
- Pipeline Development: Build, deploy, and monitor robust end-to-end ETL/ELT pipelines to ingest diverse semiconductor data streams into the data lake.
- Data Modeling: Design and implement high-performance consumption data models, ensuring clean, transformed, and production-ready datasets.
- SQL Optimization: Write and tune complex, highly optimized SQL queries for data transformation, analysis, and performance benchmarking.
- AI-Driven Delivery: Utilize generative AI coding assistants and automation tools to accelerate pipeline development, documentation, and testing.
- Data Governance: Implement data quality checks, schema evolution rules, and governance practices inherent to Iceberg and Snowflake environments.
Requirements
- Experience: dedicated experience in software, data engineering and data management.
- Workflow Orchestration: Strong hands-on experience scheduling and monitoring production-grade pipelines with Apache Airflow.
- Lakehouse Expertise: Proven track record of designing medallion architectures and working extensively with the Apache Iceberg table format.
- Tech Stack Mastery: Advanced proficiency with Snowflake and deep hands-on experience building transformation models in DBT Core.
- Expert-level SQL & Python knowledge and complete mastery of modern ETL/ELT patterns and design principles.
- Ability to write high quality code with keen attention to detail
- Experience with modern concurrent programming and threading APIs
- Experience with software development processes and tools such as debuggers, source code control systems (GitHub) and profilers is a plus
- AI Tooling: Demonstrated experience using AI tools (e.g., GitHub Copilot, Snowflake Cortex, LLM APIs, Claude Code, etc.) to speed up code development and problem-solving.
- Problem Solving: Experience delivering multiple enterprise-grade, production-level, end-to-end data pipeline solutions from scratch.
Technologies
- Apache Iceberg
- Snowflake
- Databricks (Spark)
- Trino/Starburst
- AWS Athena
- Presto
- Apache Airflow
- DBT Core
- SQL
- Python
- GitHub Copilot
- Snowflake Cortex
- LLM APIs
- Claude Code
Education & Credentials
- BS Degree in Engineering or related field
Location
Santa Clara, CA (onsite)
Similar Jobs
