EngineerJobs.io
← Back to all jobs

Job Description

Robotics Technologies is seeking a Databricks Data Engineer to support the Retail Spine initiative in Phoenix, AZ. This onsite role focuses on building and optimizing scalable ETL/ELT data solutions in a fast-paced Azure environment, with hands-on work across Databricks, PySpark, Delta Lake, and Kafka streaming.

Role Overview

In this position, you will design, build, and improve end-to-end data engineering pipelines that support incremental processing and real-time or near real-time needs. The work includes new pipeline development as well as enhancements to existing workflows, including understanding and iterating on current codebases to deliver improvements efficiently.

Key Responsibilities

  • Design, build, and optimize end-to-end ETL/ELT pipelines using Databricks and Apache Spark (PySpark).
  • Develop and maintain scalable data solutions on Microsoft Azure leveraging Databricks, Delta Lake, and Kafka streaming.
  • Build robust pipelines with Change Data Capture (CDC) to enable incremental data processing and real-time or near real-time data use cases.
  • Support both new pipeline creation and enhancement of existing pipelines, including quickly understanding current codebases to deliver targeted improvements.
  • Integrate and process data from diverse sources, including Flat files, XML, JSON, SQL Server and JDBC systems, Kafka topics, APIs, MQ, and Unity Catalog volumes and other enterprise platforms.
  • Implement data quality frameworks, including writing technical data quality rules and quarantining error records.
  • Apply canonical data modeling techniques, including SCD Type 1 and SCD Type 2 implementations.
  • Ensure high levels of data reliability, performance, scalability, and monitoring across all workflows.
  • Automate workflows using Databricks Workflows and CI/CD pipelines with GitHub, emphasizing reuse, efficiency, and standardization.
  • Collaborate with developers, architects, and product owners, engaging with stakeholders, source teams, and downstream consumers as needed.
  • Participate in Agile/Scrum ceremonies and deliver high-quality solutions within defined timelines.

Required Qualifications

  • Extensive hands-on experience with Databricks, with the ability to contribute from day one.
  • Proven experience building end-to-end data engineering pipelines with CDC capabilities.
  • PySpark and SQL proficiency (required); Python experience is a plus.
  • Experience working in Microsoft Azure cloud environments.

Technologies

  • Databricks, Apache Spark, PySpark, Microsoft Azure
  • Delta Lake, Kafka, Change Data Capture (CDC)
  • SQL Server, JDBC, Unity Catalog
  • Databricks Workflows, CI/CD, GitHub
  • SCD Type 1, SCD Type 2, Agile/Scrum
  • XML, JSON, Flat files, APIs, MQ, SQL

Location, Duration, and Employment Details

  • Location: Phoenix, AZ (onsite)
  • Duration: 9 Months + Extension (9 Months+ Extension)
  • Employment Type: W-2, 1099, C2C
  • Domain Exposure: Consumer Staples
  • Client: To Be Discussed Later

Compensation and Work Authorization

  • Salary: USD 71 - 71 per hourly
  • Bill Rate: $71/hour (C2C)
  • Work Authorization: US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other; USC, GC, H-1B, OPT-EAD, GC-EAD, H4-EAD

Job ID

J55801

Similar Jobs