Data Engineer – Databricks & ADF (ETL)
Big Data
Data
Data Analysis
Data Analytics
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Database
Databases
ETL
Informatica
Information Technology (IT)
Programming
Programming Language
Programming Languages
Software Development
SQL
Job Description
This onsite Data Engineer position in Dallas, TX supports the design, development, and ongoing optimization of Databricks and Azure Data Factory (ADF) ETL/ELT data pipelines. The role focuses on building reliable data workflows, processing large datasets, and ensuring data quality across warehouses and data lakes.
Responsibilities
- Develop and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
- Use Databricks to process and transform data.
- Write and optimize SQL for data extraction and transformation tasks.
- Apply PySpark or Python to support large-scale data processing.
- Integrate data from multiple sources into data warehouses or data lakes.
- Monitor, troubleshoot, and optimize pipeline performance and reliability.
- Collaborate with cross-functional teams to support data quality and reliability.
Requirements
- Experience with Databricks and Azure Data Factory (ADF).
- Strong understanding of ETL/ELT concepts.
- Good working knowledge of SQL and data transformation practices.
- Basic experience with Python or PySpark.
- Understanding of Azure cloud services and data lake concepts.
Technology Stack
- Databricks
- Azure Data Factory (ADF)
- ETL, ELT
- SQL
- PySpark, Python
- Azure cloud services
- Data lake
Location and Compensation
Location: Dallas, TX (onsite)
Compensation: USD 78 per hour (USD 78 - 78 per hourly)