Data Engineering Lead

Builders is a mid-sized mutual rated A by AM Best with a reputation for reliability and fairness. In this onsite role in Atlanta, GA, you will serve as the technical owner of a mission-critical Azure Databricks lakehouse platform supporting property and casualty insurance data, with a team-first, collaborative culture recognized by Top Workplaces and “Culture Excellence” honors.

Platform impact and engineering ownership

The Data Engineering Lead drives the design, development, and modernization of the Azure Databricks lakehouse that supports policy and claims operations and downstream analytics. You will align engineering execution with data governance through Unity Catalog, establish standards for secure and reliable delivery, and lead a team of Data Engineers and junior contractors.

What you will do

• Design the Databricks lakehouse architecture using Delta Lake with bronze/silver/gold medallion layers for property and casualty policy, rating, claims, billing, reinsurance, and customer data.
• Plan and lead phased migration from Guidewire (PolicyCenter, ClaimCenter, BillingCenter) and legacy systems into Azure Databricks, including historical loads, incremental/CDC loads, reconciliation, and support for decommissioning.
• Model core property and casualty insurance entities including accounts, policies, insured risks and exposures, coverages, endorsements, premiums, claims, reserves, payments, and recoveries.
• Provide data for Actuarial and Finance use cases such as written/earned/unearned premium, loss triangles, IBNR inputs, loss ratios, and statutory reporting including Schedule P and NAIC.
• Integrate third-party data sources including geospatial and weather data.
• Define Unity Catalog architecture (metastore, catalogs, schemas, and environments for development, test, and production), naming conventions, permissions, row/column-level security, and lineage.
• Manage pipeline orchestration and operations in Databricks: schedule and dependencies using Databricks Workflows (Lakeflow Jobs), set up retries, alerting and SLA monitoring, and establish failure handling and runbooks.
• Own performance and cost optimization across Spark workloads, including tuning, partitioning and liquid clustering, Photon, and cluster and SQL warehouse sizing.
• Work with IT and Platform Engineering on storage, networking, security, identity, and IaC, including CI/CD.
• Set engineering standards for pipelines, testing, code review, and documentation; lead, mentor, and review the work of Data Engineers and junior contractors.
• Partner with the Data Governance Lead to implement access controls, PII masking, lineage, and data quality rules.
• Perform other duties as assigned.

Qualifications

• Associate degree or an equivalent combination of education and experience.
• 10+ years of data engineering experience, including 6+ years hands-on with Databricks (PySpark, Spark SQL, Delta Lake), plus experience leading or mentoring engineering teams.
• Certification as a Databricks Data Engineer Professional. Azure data or solutions architect certification preferred.
• Knowledge of Unity Catalog: governance model, permissions, row/column-level security, lineage, and system tables.
• Experience leading large migrations from core insurance and legacy systems into a lakehouse or data warehouse, including property and casualty insurance data and understanding of the policy and claims lifecycle and core P&C KPIs.
• Data modeling knowledge including dimensional modeling, data vault, SCD patterns, and insurance data models.
• Technical depth in Spark internals and performance tuning, streaming (structured streaming, auto loader), CDC patterns, MERGE, and Lakeflow/Delta Live Tables.
• Expert-level SQL and strong Python skills, plus software engineering practices such as testing, modular design, and code review.
• Knowledge of data infrastructure: ADLS Gen2, Azure Data Factory, Event Hubs, Key Vault, Entra ID, VNets, and Private Link.
• Experience scheduling and monitoring production pipelines with Databricks Workflows and/or Azure Data Factory, including dependencies, parameterization, alerting, and monitoring with Databricks system tables.
• Experience with Git-based version control (GitHub or Bitbucket), branching strategies, pull request reviews, and Databricks Git folders.
• Infrastructure-as-code and CI/CD for data: Terraform, Databricks Asset Bundles, and tools such as Azure DevOps, GitHub Actions, or Bitbucket Pipelines.
• Guidewire experience (Guidewire DataHub/InfoCenter, Cloud Data Access) is preferred, including supporting actuarial reserving and pricing for P&C lines of business. API and integration tools such as MuleSoft preferred.
• Knowledge of insurance data standards such as ACORD is preferred.

Compensation and benefits

• Competitive Salary
• Bonus Structure
• Profit Sharing
• Medical, Dental, Vision Insurance
• Employer Paid Short Term Disability
• Employer Paid Long Term Disability
• Employer Paid Life Insurance
• Voluntary Life Insurance
• 401K with Company Match
• PTO

Equal opportunity

Builders Insurance Group is an equal opportunity employer. They welcome applicants from all walks of life and do not discriminate based on any protected status.