Cloud Data Migration Engineer Jobs

1 job found

Today

Data Engineer

Builders Atlanta, GA

Atlanta, GA

Today

Full time Azure Data Engineer Azure Data Lake Big Data

Apply Now

Job Description

Design, build, and operate data pipelines that power Builders’ Azure Databricks lakehouse for property and casualty insurance.

Responsibilities

Develop pipelines for phased migration into the Azure Databricks lakehouse from Guidewire (PolicyCenter, ClaimCenter, BillingCenter) and legacy systems, including historical loads, incremental/CDC (Change Data Capture) loads, and reconciliation
Create bronze and silver layer transformations in PySpark and Spark SQL following the medallion architecture
Support data modeling for property and casualty domains: policies, insured risks and exposures, coverages, endorsements, premiums, claims, reserves, payments, and recoveries
Integrate third-party datasets, including geospatial and weather data
Schedule and run production pipelines using Databricks Workflows (Lakeflow Jobs) or Azure Data Factory, including job dependencies, parameters, retries, and alerting
Investigate and resolve pipeline failures in production
Apply Unity Catalog controls set by the Data Engineering Lead and Data Governance lead, including permissions, PII masking, tags, and lineage
Implement data quality checks, error handling, and monitoring
Tune pipeline performance and cost using partitioning, liquid clustering, and right-sized job compute
Use Git version control (GitHub or Bitbucket) with CI/CD via Databricks Asset Bundles to version, review, and deploy code across development, test, and production
Write documentation and runbooks; participate in code reviews; take long-term ownership of contractor-built pipelines
Perform other duties as assigned

Requirements

Associate degree or an equivalent combination of education and experience
Certification in Databricks engineering
3+ years in data engineering, including 1+ year hands-on with Databricks (PySpark, Spark SQL, Delta Lake)
Property and casualty insurance experience, ideally Guidewire data; experience migrating data from legacy source systems
Knowledge of Python/PySpark and SQL (Structured Query Language)
Knowledge of Unity Catalog: permissions, row/column-level security, and lineage
Experience with incremental loading patterns such as CDC, MERGE, SCD (Slowly Changing Dimension), and Auto Loader
Knowledge of data modeling, including dimensional modeling
Experience scheduling and monitoring pipelines with Databricks Workflows and/or Azure Data Factory
Experience with Git-based version control (GitHub or Bitbucket), including branching, pull request reviews, and Databricks Git folders
Experience with Azure data services such as ADLS (Azure Data Lake Storage) Gen2, Azure Data Factory, and Key Vault
CI/CD fundamentals: Databricks Asset Bundles, Azure DevOps, GitHub Actions, or Bitbucket Pipelines
Experience with streaming (Structured Streaming, Event Hubs)
Experience with API (Application Programming Interface) and integration tools such as MuleSoft is an added advantage

Technologies

Azure Databricks, Delta Lake, Unity Catalog
Lakeflow, Databricks SQL, PySpark, Spark SQL
Databricks Workflows, Azure Data Factory
Git, GitHub, Bitbucket, Databricks Asset Bundles
Python, SQL (Structured Query Language)
ADLS (Azure Data Lake Storage) Gen2, Key Vault
Structured Streaming, Event Hubs, Auto Loader
MERGE, SCD (Slowly Changing Dimension), CDC (Change Data Capture)
MuleSoft, API (Application Programming Interface)
Azure DevOps, GitHub Actions, Bitbucket Pipelines

Benefits

Competitive Salary
Bonus Structure
Profit Sharing
Medical, Dental, Vision Insurance
Employer Paid Short Term Disability
Employer Paid Long Term Disability
Employer Paid Life Insurance
Voluntary Life Insurance
401K with Company Match
PTO

Position Summary

Design, develop, and maintain data pipelines for Builders’ Azure Databricks lakehouse supporting underwriting, claims, actuarial, and finance
Build reliable, high-quality, well-tested, and well-documented data solutions using Delta Lake, Unity Catalog, Lakeflow pipelines, and Databricks SQL
Deliver phased migration pipelines from Guidewire and legacy policy, claims, and billing systems into Azure Databricks
Align solutions with Unity Catalog governance and engineering standards led by the Data Engineering Lead
Strengthen long-term sustainability by progressively assuming responsibility for contractor-developed pipelines

About Builders

Builders is a mid-sized mutual with an A rating by AM Best
Recognized for reliability and fairness in fulfilling promises to customers
Teams foster a collaborative, customer-centric environment; “Culture Excellence” honors from Top Workplaces
Supports policyholders through partnerships with knowledgeable independent agencies

Equal Opportunity: Builders is an equal opportunity employer and does not discriminate based on any protected status.

Location: Atlanta, GA (onsite) • Experience: 3+ years • Education: Associate degree

View full job page →

Get Cloud Data Migration Engineer Jobs alerts

New Cloud Data Migration Engineer Jobs, daily.