Sr Databricks Data Engineer
Job Description
Join Deloitte's Core AI & Data practice to help organizations modernize data platforms, strengthen enterprise data foundations, and scale analytics and AI capabilities across the business. This onsite role in McLean, VA offers the opportunity to design, build, and optimize cloud-based data solutions on Databricks while collaborating with business and technology leaders to enable analytics across the enterprise. The position comes with a competitive salary in the range of USD 137,500 to 193,600 per year, plus a discretionary annual incentive program and a culture focused on growth, collaboration, and delivering measurable impact.
What Deloitte offers
Work with clients to architect, engineer, and deploy cloud data solutions that drive better decision making and support large-scale transformation. The team emphasizes cross-functional collaboration and continuous learning, with a focus on governance, platform engineering, and delivering insights that move the business forward.
Responsibilities
- Champion Best Practices: establish and promote leading approaches for data architecture, integration, and modeling.
- Pipeline Ownership: lead the design, development, and maintenance of robust data pipelines and architectures that support enterprise data needs at scale.
- Drive Excellence: drive efforts to improve data quality, operational efficiency, and process scalability.
- Team and Technology Lead: evaluate, pilot, and integrate new big data and analytics technologies; lead and mentor teams of data engineers and architects to ensure successful delivery.
- Data Governance: advise on and implement governance, security, and compliance strategies for modern cloud data ecosystems.
- Communication: translate technical concepts and business value for executives, business leads, and technology teams.
- DevOps and Automation: oversee CI/CD implementations using tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell for streamlined deployments and operations.
- Guidance: provide clear direction to colleagues and project teams.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field
- 5+ years of hands-on data engineering experience with a focus on Databricks on AWS, Azure, or GCP
- Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases and storage, and distributed compute platforms
- Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
- 1+ year leading complex, cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources
- Ability to travel 50% on average based on client and project needs
- Limited immigration sponsorship may be available
Technologies
- Databricks
- Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
- Delta Lake, Apache Spark, PySpark
- Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows
- Unity Catalog, Apache Airflow, Databricks Lakeflow
- AWS, Azure, GCP
Benefits
Discretionary annual incentive program, subject to program rules and individual and organizational performance outcomes.
Team
Deloitte's Core AI & Data practice helps organizations modernize data platforms, strengthen data foundations, and scale analytics and AI across the business. Practitioners work with clients to architect, engineer, and deploy cloud-based data solutions that improve decision making, enable innovation, and support large-scale transformation. The team collaborates across business and technology functions to tackle challenges in data modernization, governance, platform engineering, and insight delivery.
Preferred
- Master's degree in Computer Science, Engineering, or a related field
- Experience across one or more of AWS, Azure, and GCP and their big data services
- Proven ability to tune and optimize performance in Databricks and Apache Spark environments
- Experience with Databricks Lakeflow
- Exposure to artificial intelligence and machine learning solutions