Senior Data Engineer (Foundations)

Senior Data Engineer role focused on building and evolving the shared ingestion platform for company-wide data delivery.

Responsibilities

• Design and implement reusable ingestion components using dlt and dbt-core for both structured and unstructured sources
• Build ingestion pipelines that support high-volume, append-heavy, and schema-drifting data patterns
• Own the Apache Airflow platform end-to-end: extend and maintain DAGs and shared operators
• Handle deployments and version upgrades for Airflow and provide hands-on support to consuming teams
• Deliver incremental loading strategies with data quality checks and production-ready lineage metadata across pipelines
• Identify and remove redundant ingestion patterns across teams, driving standardization onto Platform Foundation components
• Partner with Solution Architects to evolve platform architecture for new data sources and changing business requirements
• Support data product exposure by defining and implementing governed interfaces for reliable internal access
• Contribute to infrastructure managed with Terraform; participate in multi-cloud deployment patterns (AWS and Azure)
• Use and evaluate AI-assisted development tools (e.g., GitHub Copilot, Claude Code) to accelerate platform Foundation delivery
• Promote AI tooling adoption within the squad, including best practices and guardrails for AI-generated code review
• Explore AI-powered options for internal platform documentation and self-service enablement (e.g., RAG pipelines, LLM-assisted data cataloguing)
• Maintain and improve CI/CD pipelines using TeamCity and GitHub Actions for platform Foundation components
• Define and enforce observability standards, including DAG/Task-level alerting and SLA tracking
• Participate in on-call rotation for critical ingestion pipelines and drive post-incident improvements
• Create and maintain Platform Foundation documentation, runbooks, and enablement materials for consuming squads
• Translate ambiguous or evolving business requirements into concrete technical designs, including challenging scope when needed
• Mentor mid-level engineers and participate in hiring and technical assessments

Requirements

• Bachelor’s degree in Business, Information Systems, Data/Analytics, Computer Science, or a related field
• Minimum 5 years in data engineering, including at least 2 years in a senior or platform-level position
• Proven experience building production ingestion and transformation pipelines at scale
• Experience contributing to shared platform or internal developer tooling used by multiple teams
• Python: idiomatic, testable, production-grade code (not just scripting)
• dbt-core: advanced modeling (custom materializations), testing, documentation, packages
• Apache Airflow: DAG design patterns, custom operators, dynamic task mapping, SLA management
• Comfort with one or more major cloud data platforms (e.g., Snowflake, BigQuery, Databricks, Microsoft Fabric)
• SQL: complex analytical queries, window functions, query profiling
• Git and CI/CD: trunk-based development, automated testing gates, pipeline-as-code
• Daily user of AI coding assistants (Copilot, Claude Code, or equivalent)
• Understands the limits of AI-generated code and performs rigorous review rather than blind trust
• Interest in LLM-powered data tooling (e.g., RAG pipelines, Cortex, semantic layers) is a plus

Tech Stack

• dlt
• dbt-core
• Apache Airflow
• Terraform
• AWS, Azure
• Python
• Git
• TeamCity, GitHub Actions
• Snowflake, BigQuery, Databricks, Microsoft Fabric
• SQL
• GitHub Copilot, Claude Code
• RAG pipelines, LLM-assisted data cataloguing
• Cortex, semantic layers

AI & Modern Tooling

• Daily use of AI coding assistants (Copilot, Claude Code or equivalent)
• Knows AI-generated code limitations and applies rigorous review
• Interest in LLM-powered data tooling is a plus (RAG pipelines, Cortex, semantic layers)

Location

• Auburn Hills, MI (onsite)