Databricks Data Engineer
Azure Data Engineer
Azure Databricks
Big Data
Bigdata
Bronze Silver Gold
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Lake
Data Lakehouse
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Warehouse
Database
Databases
Databricks
Databricks Dlt
Databricks Lakehouse
Databricks Lakehouse Platform
Databricks Pyspark
Databricks Spark
Databricks Workflows
Delta Lake
ETL
Informatica
Spark
SQL
Job Description
The Azure Databricks Engineer will design and deliver scalable Lakehouse solutions on Azure using Databricks and Delta Lake. This onsite role in Minneapolis focuses on performance optimization, data migration, governance and security, and building reliable data pipelines.
Responsibilities
- Develop, review, and support module-level code, including creating replication patterns and performing root cause analysis to identify recurring issues.
- Collaborate with internal and external stakeholders to maintain stable, well-documented deliverables aligned with business objectives.
- Support requirements gathering for assigned modules and document changes clearly, capturing linkages and dependencies across business processes.
- Contribute to application solution designs for a module under the Project Manager’s guidance, ensuring alignment with business requirements and technical standards.
- Review developed code, resolve technical queries for the assigned module, and ensure code and replication patterns align with project standards.
- Work with project managers throughout the software development life cycle and support deployment readiness through validation and dependency management.
- Troubleshoot recurring technical issues using root-cause analysis and modify code to resolve errors, adapt to changing environments, enhance performance, and upgrade interfaces.
- Collaborate with technical teams to validate configurations and facilitate integration of new applications during deployment.
- Prepare user training documents and related frameworks to support adoption of new applications and reduce transition risk.
- Design Databricks Lakehouse architecture using Delta Lake and Bronze/Silver/Gold medallion layers.
- Build data pipelines using Databricks Workflows, Delta Live Tables, Auto Loader, Azure Data Factory, and Databricks SQL.
- Develop enterprise data solutions using Azure Data Lake Storage Gen2, Synapse, PySpark, Spark SQL, and SQL.
- Optimize Spark workloads through partitioning, cluster sizing, data skew handling, broadcast joins, and Photon acceleration.
- Lead migration from SAS, Hadoop, Teradata, Oracle, SQL Server, Synapse, and legacy ETL platforms.
- Define data modeling and architecture for batch/streaming, event-driven processing, and establish governance, security, lineage, and access controls.
Required Experience and Skills
- Strong hands-on experience in cloud data architecture, Databricks platform design, performance optimization, migration, data modeling, streaming, and security.
- Mandatory Skills: DataBricks - Data Engineering.
- Experience: 5-8 years (minimum 5 years).
Technologies
- Azure, Databricks, Delta Lake
- PySpark, Spark SQL, SQL
- ADLS Gen2, Azure Data Factory, Synapse
- Databricks Workflows, Delta Live Tables, Auto Loader, Databricks SQL
- Bronze/Silver/Gold Medallion, Photon acceleration
- SAS, Hadoop, Teradata, Oracle, SQL Server
- Legacy ETL platforms
Benefits
- Full range of medical and dental benefits options
- Disability insurance
- Paid time off (inclusive of sick leave)
- Other paid and unpaid leave options
Role Purpose
The purpose of this role is to facilitate application solutions by developing and reviewing module-level code, creating replication patterns, performing root cause analysis to identify recurring issues, and collaborating with internal and external stakeholders to deliver stable, well-documented outcomes that meet business objectives.
Compensation and Location
- Location: Minneapolis, Minnesota (onsite)
- Salary: USD 60,000 - 135,000 per year