Sr Databricks Data Engineer

Senior Databricks Data Engineer role within Deloitte's AI & Data practice, focused on designing, building, and optimizing cloud-based data platforms to enable analytics, AI use cases, and data-driven decision making across large-scale enterprise transformations.

Location

Tempe, AZ on site

Compensation

USD 116,200 - 229,100 per year

Responsibilities

• Establish and promote best-in-class approaches for data architecture, integration, and modelling.
• Own the design, development, and maintenance of robust data pipelines and architectures that support enterprise-scale data needs.
• Lead efforts to improve data quality, operational efficiency, and scalability of data processes.
• Act as a team and technology lead by evaluating, piloting, and integrating new big data and analytics technologies, while mentoring data engineers and architects.
• Advise on data governance, security, and compliance strategies tailored to modern cloud data ecosystems.
• Communicate technical concepts and business value to executives, business leads, and technology teams.
• Oversee DevOps and automation practices, implementing CI/CD with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell to streamline deployments and operations.

Requirements

• Self-directed and collaborative work style.
• Strong written and verbal communication skills.
• Detail-oriented mindset with a focus on high-quality work products.
• Ability to build and sustain professional relationships.
• Proven ability to lead projects or workstreams.
• Capable of managing and prioritizing multiple tasks in a fast-paced environment.
• Professional demeanor with strong interpersonal skills.
• Ability to meet deadlines and manage time effectively.
• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of hands-on data engineering experience focusing on Databricks across AWS, Azure, or GCP.
• Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms.
• Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark.
• 1+ year leading complex cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources.
• Ability to travel 50% on average, depending on client engagements.
• Limited immigration sponsorship may be available.

Technologies

• Databricks
• Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
• Delta Lake, Apache Spark, PySpark
• Delta Live Tables, Autoloader, Structured Streaming
• Databricks Workflows, Apache Airflow, Unity Catalog
• Databricks Lakeflow
• AWS, Azure, GCP

The Team

Deloitte's Core AI & Data practice helps organizations modernize data platforms, strengthen enterprise data foundations, and scale analytics and AI capabilities across the business. The team designs, engineers, and deploys cloud-based data solutions to improve decision-making, drive innovation, and support large-scale transformation, collaborating across business and technology functions to solve data modernization challenges.

Qualifications Required

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of hands-on data engineering experience focusing on Databricks across AWS, Azure, or GCP.
• Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms.
• Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark.
• 1+ year leading complex cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources.
• Ability to travel 50% on average.
• Limited immigration sponsorship may be available.

Preferred

• Master's degree in Computer Science, Engineering, or a related field.
• Experience across one or more cloud ecosystems (AWS, Azure, GCP) and associated big data services.
• Proven performance tuning and optimization for Databricks and Apache Spark environments.
• Experience with Databricks Lakeflow.
• Experience with artificial intelligence and machine learning solutions.