Data Engineer

Proteam Solutions is seeking a Senior Data Engineer with deep Databricks expertise to design, develop, and optimize enterprise-scale data pipelines. The role focuses on building scalable Lakehouse solutions and ensuring reliable, high-quality data for analytics, reporting, and downstream applications. This on-site position is based in Dublin, OH.

Responsibilities

• Architect and implement scalable ETL/ELT pipelines using Databricks
• Create PySpark and Spark SQL transformations
• Construct Bronze, Silver, and Gold data layers following the Medallion Architecture
• Deploy Delta Live Tables, Workflows, and Unity Catalog integrations
• Tune Spark jobs for scalability and performance
• Design dimensional data models for analytical workloads
• Establish data quality validations and monitoring mechanisms
• Collaborate with analytics, engineering, and architecture teams
• Develop reusable frameworks and automation
• Support CI/CD deployment pipelines and Git-based development processes

Requirements

• 7–10+ years of data engineering experience
• Extensive hands-on Databricks experience
• Strong PySpark programming
• Advanced Spark SQL skills
• Experience with Delta Lake
• Experience implementing Medallion Architecture
• Hands-on Delta Live Tables experience
• Unity Catalog experience
• Strong SQL optimization and performance tuning abilities
• Experience with AWS, Azure, or GCP
• CI/CD and Git experience
• Solid understanding of distributed data systems
• Experience designing enterprise data models

Technologies

• Databricks
• PySpark
• Spark SQL
• Delta Lake
• Medallion Architecture
• Delta Live Tables
• Unity Catalog
• AWS
• Azure
• GCP
• Git
• CI/CD

Preferred Qualifications

• Databricks certification
• Experience with streaming data
• Experience building reusable data frameworks
• Experience with enterprise governance
• Exposure to AI-assisted development tools, including Databricks AI Genie