Sr Databricks Data Engineer

Deloitte's AI & Data practice is seeking a Sr Databricks Data Engineer to design, build, and optimize cloud-based data solutions on Databricks, enabling analytics and AI initiatives while modernizing enterprise data platforms. This on-site role is based in Nashville, Tennessee, with a salary range of USD 137,500 to 193,600 per year and opportunities to lead data engineering programs across client projects.

Responsibilities

• Establish and promote best-in-class approaches for data architecture, integration, and modeling.
• Own the design, development, and maintenance of scalable data pipelines and architectures that support enterprise data needs.
• Drive improvements in data quality, operational efficiency, and process scalability.
• Lead and mentor data engineering and architecture teams; assess, pilot, and integrate new big data and analytics technologies to keep the organization at the forefront.
• Advise on data governance, security, and compliance strategies for modern cloud data ecosystems.
• Translate technical concepts and business value for executives, business leads, and technology partners.
• Oversee DevOps practices and automation, implementing CI/CD with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell.
• Provide clear guidance to colleagues and project teams.

Requirements

• Ability to work both independently and collaboratively within a team.
• Strong written and verbal communication skills.
• Meticulous attention to detail and high-quality deliverables.
• Ability to build and sustain professional relationships.
• Capability to lead projects or workstreams.
• Ability to manage multiple tasks in a fast-paced, dynamic environment.
• Strong interpersonal skills and professional demeanor.
• Ability to meet deadlines.
• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of hands-on data engineering experience with a focus on Databricks across AWS, Azure, or GCP.
• Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases and storage solutions, and distributed compute platforms.
• Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark.
• 1+ year leading complex, cross-functional data projects and technical teams, including exposure to Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources.
• Ability to travel 50% on average, depending on client engagement.
• Limited immigration sponsorship may be available.
• Master's degree in Computer Science, Engineering, or a related field.
• Experience with one or more cloud ecosystems (AWS, Azure, GCP) and associated big data services.
• Experience tuning performance in Databricks and Apache Spark environments.
• Experience with Databricks Lakeflow.
• Experience with artificial intelligence and machine learning solutions.

Technologies

• Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
• Databricks across AWS, Microsoft Azure, and Google Cloud Platform
• Lakehouse architecture, Apache Spark, Delta Lake
• Cloud-native databases, storage solutions, distributed compute platforms
• PySpark, Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows
• Apache Airflow, Unity Catalog, CI/CD pipelines
• Databricks Lakeflow

Benefits

• Discretionary annual incentive program