Senior Data Engineer - Databricks
Analytics
Apache Airflow
Azure Databricks
Big Data
Bigdata
Business Intelligence
CI/CD
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Integration Tools
Data Lake
Data Lakehouse
Data Management
Data Pipeline
Data Platform
Data Processing
Data Security
Data Visualization
Data Warehouse
Database
Databases
Databricks
Databricks Lakehouse
Databricks Lakehouse Platform
Databricks Marketplace
Databricks Pyspark
Delta Lake
Delta Sharing
DevOps
Devops Tools
Digital Marketing
ETL
Gcp Data Engineer
Informatica
Information Technology (IT)
Programming
Programming Language
Programming Languages
Reporting and Analytics
Software Development
Software Engineering
Spark
SQL
Visual Design
Job Description
Capgemini is hiring a Senior Data Engineer to design and optimize scalable data solutions on the Databricks Lakehouse Platform in Atlanta, GA (onsite).
Responsibilities
- Define and deploy an end-to-end data architecture that includes Databricks medallion architecture
- Lead large teams of engineers and developers to deploy Databricks solutions
- Manage senior client stakeholders in delivery and architecture discussions
Requirements
- 6+ years of experience in data engineering
- 4+ years designing and delivering data engineering solutions with Databricks
- Expert hands-on development in PySpark/Python
- Expert hands-on skills in SQL
- Hands-on cloud platform experience in two of: Azure, AWS, Google Cloud
- Prior expertise deploying Delta Lake solutions into production
- Experience extracting value from large, complex datasets from multiple sources and databases
- Strong understanding of database engineering and design principles
- CI/CD familiarity desired
- Background working with orchestration tools such as Airflow
- Strong knowledge of Unity Catalog
- Experience with ETL tools such as Infomatica, Talend, Matillion desired
- Delta Sharing and Marketplace expertise
- Experience working with Databricks Apps
- Working knowledge of Databricks Genie, AI/BI, and Databricks Data Visualization capabilities
- Conceptual, logical, and physical data modeling experience preferred
Technologies
- Databricks Lakehouse Platform
- Apache Spark, PySpark
- Python
- Delta Lake
- SQL
- Azure, AWS, GCP
- Unity Catalog
- Airflow
- Infomatica, Talend, Matillion
- Delta Sharing, Databricks Marketplace
- Databricks Apps
- Databricks Genie, AI/BI, Databricks Data Visualization
- CI/CD
Job Details
- Location: Atlanta, GA (onsite)
- Salary range: USD 89,400 - 166,398 per year
- Minimum experience: 6 years
Benefits
- Paid time off based on employee grade (A-F) per policy: Vacation 12-25 days depending on grade, Company-paid holidays, Personal Days, Sick Leave
- Medical, dental, and vision coverage (or provincial healthcare coordination in Canada)
- Retirement savings plans (e.g., 401(k) in the U.S., RRSP in Canada)
- Life and disability insurance
- Employee assistance programs
- Other benefits as provided by local policy and eligibility
Additional Role Notes
- Design, build, and optimize scalable data solutions on the Databricks Lakehouse Platform
- Develop end-to-end ETL/ELT pipelines using Apache Spark (PySpark), Delta Lake, and cloud platforms such as Azure, AWS, or GCP
- Support data warehousing, performance tuning, data modeling, and integration of structured and unstructured data from multiple sources