Databricks Data Engineer
Azure
Big Data
Bigdata
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Engineer
Cloud Native
Cloud Operations
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Integration Tools
Data Lakehouse
Data Management
Data Pipeline
Data Platform
Data Processing
Data Security
Data Warehouse
Database
Databases
Databricks
ETL
Google Cloud
Informatica
Pyspark
Spark
SQL
Job Description
Robotics Technologies is hiring a Databricks Data Engineer for an onsite role in Atlanta, GA with a contract term of 24 months plus extension. The position focuses on building and operating enterprise-scale data processing solutions using Databricks and the Spark ecosystem.
Responsibilities
- Design and develop scalable data engineering solutions using Databricks.
- Build batch and streaming data pipelines using Apache Spark.
- Create data transformations using PySpark and Spark SQL.
- Implement lakehouse architecture using Delta Lake.
- Manage data access, governance, and permissions through Unity Catalog.
- Build and maintain automated pipelines and schedules using Databricks Workflows.
- Implement incremental processing patterns and CDC pipelines.
- Optimize Spark workloads to improve processing performance and job efficiency.
- Integrate Databricks with cloud storage and enterprise databases.
- Develop data validation routines and quality monitoring processes.
- Collaborate with data scientists and analytics teams on data needs and delivery.
- Troubleshoot production issues and support reliable pipeline operations.
Required Qualifications
- Strong hands-on experience with Databricks.
- Advanced programming skills in Python and PySpark.
- Strong knowledge of Apache Spark and Spark SQL.
- Experience with Delta Lake and lakehouse architecture.
- Knowledge of Unity Catalog and data governance practices.
- Experience with Delta Live Tables / Lakeflow.
- Experience building ETL and ELT pipelines.
- Knowledge of AWS, Azure, or GCP cloud platforms.
- Experience with SQL, data modeling, and data warehousing.
- Familiarity with Git, CI/CD, and workflow orchestration.
Preferred Qualifications
- Experience with Databricks Lakeflow and Lakeflow Connect.
- Knowledge of Apache Kafka and real-time data processing.
- Experience with Terraform and cloud infrastructure automation.
- Knowledge of MLflow and machine learning workflows.
- Experience with enterprise data migration projects.
Technologies
- Databricks
- Apache Spark
- PySpark
- Delta Lake
- Delta Live Tables
- Lakeflow
- Unity Catalog
- Spark SQL
- Databricks Workflows
- AWS
- Azure
- GCP
- Git
- CI/CD
Location and Compensation
Location: Atlanta, GA (onsite)
Compensation: USD 76 - USD 76 per hour
Contract Duration: 24 months with extension