Senior Data Engineer

Optum is hiring a Senior Data Engineer to design and run governed data platforms and pipelines for healthcare data solutions.

Responsibilities

• Lead the design and delivery of scalable, governed data engineering solutions, including technical direction, implementation planning, risk management, and production readiness for data products used by business, analytics, and AI/ML stakeholders
• Develop and manage ETL/ELT pipelines to ingest, transform, and integrate data from heterogeneous sources (relational databases, APIs, file shares, and end-user-computing assets)
• Create Python and PySpark solutions for distributed processing, automation, integration, and reusable transformation logic
• Improve data platform performance and manage cloud consumption using SQL tuning, Spark job optimization, storage format selection, partitioning, cluster configuration, and workload scheduling
• Build and optimize workflows on Azure Cloud services and Databricks, including medallion (bronze/silver/gold) layering and orchestrated pipelines
• Migrate business logic from legacy and end-user-computing systems into documented, testable transformations over governed tables
• Define and implement data quality practices, including validation, reconciliation, and parity testing to ensure migrated pipelines are accurate, reliable, auditable, and consistent
• Establish data lineage, ownership, and cataloguing, including documented rollup and allocation rules when sources operate at different grains
• Implement security and compliance controls for PHI/PII and confidential business data in accordance with HIPAA or similar regulations, including de-identification and masking for controlled sharing
• Build and improve CI/CD, version control, and automated testing for data pipelines
• Implement production monitoring, alerting, logging, and operational runbooks for reliability, timely issue detection, incident response, and continuous improvement
• Support AI/ML and advanced analytics by provisioning trusted data, scalable infrastructure, and governed consumption patterns for data scientists, analysts, and partners
• Design and publish reusable data products, data services, APIs, and consumption layers so downstream applications and analytics/AI-ML use cases consume trusted data consistently
• Partner with product owners, business stakeholders, architects, data scientists, analysts, and engineering teams to translate requirements into scalable solutions and communicate technical tradeoffs
• Design, develop, and deploy AI-enabled data solutions for complex business challenges and responsible AI use, leveraging AI-powered development tools to improve productivity and delivery quality
• Mentor data engineers and analysts through design reviews, code reviews, and best-practice guidance

Requirements

• 6+ years overall experience in data engineering, data warehousing, distributed data processing, or enterprise data platform delivery
• 6+ years of experience with SQL, data modeling, and performance tuning for large-scale systems
• 3+ years with ETL/ELT frameworks, pipeline orchestration, and heterogeneous source-system integration
• 3+ years with Python for data processing, automation, and integration
• 3+ years with Databricks, Spark/PySpark, and Delta Lake
• 3+ years with public cloud platforms such as Azure or GCP
• 1+ year integrating data from heterogeneous source systems
• 1+ year in data quality, validation, reconciliation, observability, and production support for large-scale data pipelines
• 1+ year with Git, CI/CD platforms, automated testing frameworks, release management, and deployment practices for production data pipelines
• 1+ year with AI-powered development tools used responsibly to improve engineering productivity and delivery quality

Technologies

• Python, PySpark, SQL, Databricks, Spark, Delta Lake
• Azure, GCP
• ETL, ELT
• Git, CI/CD
• HIPAA
• Azure Data Factory, Airflow, dbt
• Unity Catalog, Microsoft Purview
• APIs

Benefits

• Comprehensive benefits package
• Incentive and recognition programs
• Equity stock purchase
• 401k contribution

Work Model

• Remote within the U.S.
• For hires in the Minneapolis or Washington, D.C. area: required to work in the office a minimum of four days per week

Preferred Qualifications

• Experience with batch and near-real-time patterns, including event-driven ingestion, streaming pipelines, or comparable technologies
• Experience with Azure Data Factory or comparable orchestration tooling such as Airflow, dbt, or similar
• Experience migrating legacy or end-user-computing (spreadsheet- or macro-based) business logic into governed data pipelines
• Experience with data governance, lineage, and catalog tooling (Unity Catalog, Microsoft Purview, or similar)
• Familiarity with AI/ML frameworks and supporting infrastructure for data science workflows
• Experience working with PHI/PII and restricted client data in compliance with HIPAA or similar regulations, including de-identification and masking
• Healthcare payer, actuarial, pricing, or financial-modelling domain experience

Pay

• $91,700 - $163,700 per year, based on full-time employment and several factors

Application Details

• Posted for a minimum of 2 business days or until a sufficient candidate pool has been collected
• Posting may come down early due to volume of applicants

Additional Information

• UnitedHealth Group is a drug-free workplace; candidates are required to pass a drug test before beginning employment

Location: Eden Prairie, MN (remote)