Senior Databricks Data Engineer role within Deloitte's AI & Data practice, focused on designing, building, and optimizing cloud-based data platforms to enable analytics, AI use cases, and data-driven decision making across large-scale enterprise transformations.

Location

Tempe, AZ on site

Compensation

USD 116,200 - 229,100 per year

Responsibilities

Establish and promote best-in-class approaches for data architecture, integration, and modelling.
Own the design, development, and maintenance of robust data pipelines and architectures that support enterprise-scale data needs.
Lead efforts to improve data quality, operational efficiency, and scalability of data processes.
Act as a team and technology lead by evaluating, piloting, and integrating new big data and analytics technologies, while mentoring data engineers and architects.
Advise on data governance, security, and compliance strategies tailored to modern cloud data ecosystems.
Communicate technical concepts and business value to executives, business leads, and technology teams.
Oversee DevOps and automation practices, implementing CI/CD with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell to streamline deployments and operations.

Requirements

Self-directed and collaborative work style.
Strong written and verbal communication skills.
Detail-oriented mindset with a focus on high-quality work products.
Ability to build and sustain professional relationships.
Proven ability to lead projects or workstreams.
Capable of managing and prioritizing multiple tasks in a fast-paced environment.
Professional demeanor with strong interpersonal skills.
Ability to meet deadlines and manage time effectively.
Bachelor's degree in Computer Science, Engineering, or a related field.
5+ years of hands-on data engineering experience focusing on Databricks across AWS, Azure, or GCP.
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms.
Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark.
1+ year leading complex cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources.
Ability to travel 50% on average, depending on client engagements.
Limited immigration sponsorship may be available.

Technologies

Databricks
Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
Delta Lake, Apache Spark, PySpark
Delta Live Tables, Autoloader, Structured Streaming
Databricks Workflows, Apache Airflow, Unity Catalog
Databricks Lakeflow
AWS, Azure, GCP

The Team

Deloitte's Core AI & Data practice helps organizations modernize data platforms, strengthen enterprise data foundations, and scale analytics and AI capabilities across the business. The team designs, engineers, and deploys cloud-based data solutions to improve decision-making, drive innovation, and support large-scale transformation, collaborating across business and technology functions to solve data modernization challenges.

Qualifications Required

Bachelor's degree in Computer Science, Engineering, or a related field.
5+ years of hands-on data engineering experience focusing on Databricks across AWS, Azure, or GCP.
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms.
Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark.
1+ year leading complex cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources.
Ability to travel 50% on average.
Limited immigration sponsorship may be available.

Preferred

Master's degree in Computer Science, Engineering, or a related field.
Experience across one or more cloud ecosystems (AWS, Azure, GCP) and associated big data services.
Proven performance tuning and optimization for Databricks and Apache Spark environments.
Experience with Databricks Lakeflow.
Experience with artificial intelligence and machine learning solutions.

Sr Databricks Data Engineer

Job Description