Sr Databricks Data Engineer

The Sr Databricks Data Engineer role at Deloitte focuses on designing, building, and optimizing cloud-native data platforms using Databricks to enable analytics and AI across AWS, Azure, and GCP.

Location

Charlotte, NC onsite

Compensation

USD 137,500 to 193,600 per year

Responsibilities

• Set and promote leading data architecture, integration, and modeling practices across the organization.
• Own the design, development, and stewardship of scalable data pipelines and architectures that support enterprise data needs.
• Assess, pilot, and integrate new big data and analytics technologies; mentor and develop teams of data engineers and architects to ensure effective project delivery.
• Advise on and implement governance, security, and compliance strategies for modern cloud data ecosystems.
• Translate complex technical concepts and business value for executives, business leads, and technology teams.
• Oversee CI/CD implementations with Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell to streamline deployments and operations.
• Provide clear guidance and mentorship to colleagues and project teams.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field
• Master's degree in Computer Science, Engineering, or a related field
• 5+ years of hands-on data engineering experience with a focus on Databricks on AWS, Azure, or Google Cloud Platform
• Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
• Background in data warehousing, third normal form (3NF), dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
• 1+ year leading complex, cross-functional data projects and technical teams, with hands-on work in Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data engineering pipelines, code, and compute resources
• Ability to travel approximately 50% based on client engagements
• Limited immigration sponsorship may be available
• Experience across one or more cloud ecosystems (AWS, Azure, GCP) and related big data services
• Proven ability to tune and optimize performance in Databricks and Apache Spark environments
• Experience with Databricks Lakeflow
• Experience with artificial intelligence and machine learning solutions

Key Technologies

• Databricks, AWS, Microsoft Azure, Google Cloud Platform (GCP)
• Lakehouse architecture, Apache Spark, Delta Lake
• PySpark, Delta Live Tables, Autoloader, Structured Streaming
• Databricks Workflows, Apache Airflow, Unity Catalog
• CI/CD pipelines, Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
• Databricks Lakeflow

The Team

Deloitte's Core AI & Data practice helps organizations modernize data platforms, strengthen enterprise data foundations, and scale analytics and AI capabilities across the business. The team collaborates with clients to architect, engineer, and deploy cloud-based data solutions that improve decision-making and enable transformation, working across both business and technology functions to address complex data modernization challenges.