Sr Databricks Data Engineer
Apache Airflow
Automation
Big Data
Bigdata
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Lake
Data Lakehouse
Data Pipeline
Data Platform
Data Security
Data Warehouse
Database
Databases
Databricks
Databricks Lakeflow
Databricks Workflows
Delta Lake
Delta Live Tables
ETL
Spark
SQL
Structured Streaming
Job Description
Join Deloitte's AI & Data practice on site in Pittsburgh to help modernize data platforms, enable analytics and AI, and drive measurable business outcomes. This role offers a competitive compensation range of USD 116,200 to 229,100 per year, plus a discretionary annual incentive and a benefits package aligned with the Core Talent Model. You will work in a collaborative, data-driven environment that prizes technical excellence and leadership, with opportunities to shape cloud based data engineering solutions on Databricks.
Benefits
- Discretionary annual incentive program
- Benefits package aligned with Core Talent Model
- Competitive salary range: USD 116,200 – 229,100 per year
Responsibilities
- Champion best practices by documenting and promoting superior data architecture, integration, and modeling approaches
- Own data pipelines and architectures to support large scale enterprise data needs
- Lead efforts to improve data quality, operational efficiency, and process scalability
- Evaluate, pilot, and integrate new big data and analytics technologies; lead and develop teams of data engineers and architects
- Design and implement governance, security, and compliance strategies for modern cloud data ecosystems
- Communicate technical concepts and business value to executives, business leads, and technology teams
- Oversee CI/CD practices with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell to streamline deployments and operations
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field
- 5+ years of hands-on data engineering experience focusing on Databricks on AWS, Azure, or GCP
- Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
- Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata driven ingestion and data quality frameworks using PySpark
- 1+ year leading complex, cross-functional data projects and technical teams, including experience with Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data engineering pipelines, code, and compute resources
- Ability to travel up to 50% on average based on the work and client needs
- Limited immigration sponsorship may be available
Preferred qualifications
- Master's degree in Computer Science, Engineering, or a related field
- Experience across one or more cloud ecosystems (AWS, Azure, GCP) and associated big data services
- Experience tuning and optimizing performance in Databricks and Apache Spark environments
- Experience with Databricks Lakeflow
- Experience with artificial intelligence and machine learning solutions
Technologies
- Databricks
- AWS, Microsoft Azure, Google Cloud Platform (GCP)
- Apache Spark, Delta Lake, Unity Catalog
- Delta Live Tables, Autoloader, Structured Streaming
- Databricks Workflows, Apache Airflow
- Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
- PySpark, Databricks Lakeflow
Similar Jobs