Lead Azure Data Engineer
Job Description
Robotics Technologies is hiring a Lead Azure Data Engineer for an onsite engagement in Houston, TX. This 10-month project includes the possibility of extension and focuses on delivering modern data engineering capabilities using Azure Data Factory and Azure Databricks, with CI/CD support across Azure environments. Domain exposure includes Aerospace/ Airlines.
This is a hands-on leadership opportunity for an engineer who can build, optimize, and operationalize data pipelines and storage systems, while partnering with stakeholders through testing and delivery activities.
Responsibilities
- Develop Azure Data Factory pipelines, including data flows, triggers, integration runtimes, and parameterized workflows.
- Build and optimize Azure Databricks notebooks, jobs, workflows, and Delta Lake solutions.
- Create modular data engineering code using Python or Scala, applying unit testing practices.
- Write and tune advanced SQL queries using CTEs and window functions for performance on large datasets.
- Manage data storage and optimization with ADLS Gen2, Parquet, partitioning, Z-Ordering, OPTIMIZE, and VACUUM.
- Support Git-based CI/CD to promote changes across Azure Data Factory and Databricks environments.
- Collaborate with stakeholders and participate in testing and delivery activities.
Requirements
- 8-10 years of data engineering experience.
- At least 3 years of Azure experience.
- Experience with Azure Data Factory.
- Experience with Azure Databricks.
- Strong advanced SQL skills.
- Proficiency in Python or Scala.
- Experience working with large datasets and optimizing data pipelines.
Location: Houston, TX (onsite)
Duration: 10 Months + Extension
Pay: $63/hour (C2C)
Employment Type: W-2, 1099, C2C
Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other
Technologies: Azure Data Factory, data flows, triggers, integration runtimes, parameterized workflows, Azure Databricks, Delta Lake, Python, Scala, SQL, CTEs, window functions, ADLS Gen2, Parquet, partitioning, Z-Ordering, OPTIMIZE, VACUUM, Git-based CI/CD, CI/CD, unit testing
Job ID: J55503