Data Engineer

Design, build, and operate data pipelines that power Builders’ Azure Databricks lakehouse for property and casualty insurance.

Responsibilities

• Develop pipelines for phased migration into the Azure Databricks lakehouse from Guidewire (PolicyCenter, ClaimCenter, BillingCenter) and legacy systems, including historical loads, incremental/CDC (Change Data Capture) loads, and reconciliation
• Create bronze and silver layer transformations in PySpark and Spark SQL following the medallion architecture
• Support data modeling for property and casualty domains: policies, insured risks and exposures, coverages, endorsements, premiums, claims, reserves, payments, and recoveries
• Integrate third-party datasets, including geospatial and weather data
• Schedule and run production pipelines using Databricks Workflows (Lakeflow Jobs) or Azure Data Factory, including job dependencies, parameters, retries, and alerting
• Investigate and resolve pipeline failures in production
• Apply Unity Catalog controls set by the Data Engineering Lead and Data Governance lead, including permissions, PII masking, tags, and lineage
• Implement data quality checks, error handling, and monitoring
• Tune pipeline performance and cost using partitioning, liquid clustering, and right-sized job compute
• Use Git version control (GitHub or Bitbucket) with CI/CD via Databricks Asset Bundles to version, review, and deploy code across development, test, and production
• Write documentation and runbooks; participate in code reviews; take long-term ownership of contractor-built pipelines
• Perform other duties as assigned

Requirements

• Associate degree or an equivalent combination of education and experience
• Certification in Databricks engineering
• 3+ years in data engineering, including 1+ year hands-on with Databricks (PySpark, Spark SQL, Delta Lake)
• Property and casualty insurance experience, ideally Guidewire data; experience migrating data from legacy source systems
• Knowledge of Python/PySpark and SQL (Structured Query Language)
• Knowledge of Unity Catalog: permissions, row/column-level security, and lineage
• Experience with incremental loading patterns such as CDC, MERGE, SCD (Slowly Changing Dimension), and Auto Loader
• Knowledge of data modeling, including dimensional modeling
• Experience scheduling and monitoring pipelines with Databricks Workflows and/or Azure Data Factory
• Experience with Git-based version control (GitHub or Bitbucket), including branching, pull request reviews, and Databricks Git folders
• Experience with Azure data services such as ADLS (Azure Data Lake Storage) Gen2, Azure Data Factory, and Key Vault
• CI/CD fundamentals: Databricks Asset Bundles, Azure DevOps, GitHub Actions, or Bitbucket Pipelines
• Experience with streaming (Structured Streaming, Event Hubs)
• Experience with API (Application Programming Interface) and integration tools such as MuleSoft is an added advantage

Technologies

• Azure Databricks, Delta Lake, Unity Catalog
• Lakeflow, Databricks SQL, PySpark, Spark SQL
• Databricks Workflows, Azure Data Factory
• Git, GitHub, Bitbucket, Databricks Asset Bundles
• Python, SQL (Structured Query Language)
• ADLS (Azure Data Lake Storage) Gen2, Key Vault
• Structured Streaming, Event Hubs, Auto Loader
• MERGE, SCD (Slowly Changing Dimension), CDC (Change Data Capture)
• MuleSoft, API (Application Programming Interface)
• Azure DevOps, GitHub Actions, Bitbucket Pipelines

Benefits

• Competitive Salary
• Bonus Structure
• Profit Sharing
• Medical, Dental, Vision Insurance
• Employer Paid Short Term Disability
• Employer Paid Long Term Disability
• Employer Paid Life Insurance
• Voluntary Life Insurance
• 401K with Company Match
• PTO

Position Summary

• Design, develop, and maintain data pipelines for Builders’ Azure Databricks lakehouse supporting underwriting, claims, actuarial, and finance
• Build reliable, high-quality, well-tested, and well-documented data solutions using Delta Lake, Unity Catalog, Lakeflow pipelines, and Databricks SQL
• Deliver phased migration pipelines from Guidewire and legacy policy, claims, and billing systems into Azure Databricks
• Align solutions with Unity Catalog governance and engineering standards led by the Data Engineering Lead
• Strengthen long-term sustainability by progressively assuming responsibility for contractor-developed pipelines

About Builders

• Builders is a mid-sized mutual with an A rating by AM Best
• Recognized for reliability and fairness in fulfilling promises to customers
• Teams foster a collaborative, customer-centric environment; “Culture Excellence” honors from Top Workplaces
• Supports policyholders through partnerships with knowledgeable independent agencies

Equal Opportunity: Builders is an equal opportunity employer and does not discriminate based on any protected status.

Location: Atlanta, GA (onsite) • Experience: 3+ years • Education: Associate degree