Sr Databricks Data Engineer
Manager
Big Data
Bigdata
Cloud Platform
Cloud Platforms
Data Analytics
Data Architecture
Data Engineer
Data Governance
Data Integration
Data Lake
Data Lakehouse
Data Pipeline
Data Platform
Data Processing
Data Security
Databricks
Databricks Workflows
Delta Lake
Delta Live Tables
Engineering
ETL
Spark
Technical Lead
Job Description
Senior Databricks Data Engineer role at Deloitte in New York, onsite.
Responsibilities
- Establish and promote best in class data architecture, integration, and modelling practices, with thorough documentation and governance.
- Own the design, development, and ongoing maintenance of scalable data pipelines and architectures that meet enterprise data needs.
- Lead initiatives to enhance data quality, boost operational efficiency, and improve process scalability.
- Provide team and technology leadership by evaluating, piloting, and integrating new big data and analytics tools; mentor and grow data engineers and architects to ensure effective project delivery.
- Advise on data governance, security, and compliance strategies tailored to cloud data ecosystems.
- Translate technical concepts and business value for executives, business leads, and technology teams.
- Drive DevOps and automation through CI/CD practices using Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell to streamline deployments and operations.
Requirements
- Ability to work independently and collaboratively as part of a team
- Strong written and verbal communication skills
- Meticulous attention to detail and quality of work product
- Ability to build and sustain professional relationships
- Ability to lead projects or workstreams
- Ability to manage and prioritize multiple tasks in a fast paced and dynamic environment
- Strong interpersonal skills and professional demeanor
- Ability to meet deadlines
Technologies
- Databricks
- Delta Lake
- Apache Spark
- PySpark
- Delta Live Tables
- Autoloader
- Structured Streaming
- Databricks Workflows
- Unity Catalog
- Apache Airflow
- CI/CD pipelines
- Azure DevOps
- AWS Code Pipeline
- Jenkins
- TFS
- PowerShell
- AWS
- Azure
- GCP
- Databricks Lakeflow
Benefits
- Discretionary annual incentive program
The Team
Deloitte's Core AI & Data practice helps organizations modernize data platforms, strengthen enterprise data foundations, and scale analytics and artificial intelligence capabilities across the business. The team collaborates with clients to architect, engineer, and deploy cloud-based data solutions that improve decision-making, enable innovation, and support large-scale transformation. Practitioners work across business and technology functions to tackle complex challenges in data modernization.
Preferred qualifications
- Master's degree in Computer Science, Engineering, or a related field
- Experience with one or more of AWS, Azure, and GCP cloud ecosystems and associated big data services
- Experience tuning and optimizing performance in Databricks and Apache Spark environments
- Experience with Databricks Lakeflow
- Experience with artificial intelligence and machine learning solutions
Compensation
- Salary range: USD 116,200 - 229,100 per year
Similar Jobs