Principal Data Engineer

Optum is seeking a Principal Data Engineer to design, implement, and operate data-intensive systems while steering data architecture across product and engineering teams from Eden Prairie, MN with remote work options. The role blends hands-on development with mentorship, focusing on data quality, governance, and reliable deployments within scalable data platforms. Compensation ranges from USD 112,700 to 193,200 per year.

Responsibilities

• Collaborate with product owners, system analysts, and engineering squads to deliver data solutions aligned with an Agile roadmap.
• Work with data architects and platform engineers to translate requirements into scalable, robust data architecture.
• Act as technical lead and primary contact for Scrum teams, guiding cross-team solutions and mentoring junior engineers.
• Design, implement, test, and operate data-intensive systems, including streaming and batch pipelines, using Databricks, Kafka, Snowflake, and cloud-native storage.
• Create and maintain ETL/ELT pipelines, data transformations, and orchestration workflows with Python, SQL, and distributed processing frameworks.
• Establish and enhance CI/CD, DevOps, and DataOps practices to enable reliable, automated deployments of data pipelines and analytics workloads.
• Develop and maintain configuration management, infrastructure automation, deployment strategies, and monitoring/observability tools for data platforms.
• Apply software and data engineering best practices, emphasizing reliability engineering, fault-tolerant architecture, performance tuning, and automated testing.
• Contribute to incident management, perform root cause analysis, and implement remediation using ServiceNow.
• Define and implement data quality, governance, security, and resilience strategies to ensure trusted and compliant data products.
• Support data solutions across the full software development life cycle, from design and development through production deployment and ongoing support.
• Participate in design reviews, architecture discussions, code reviews, defect triage, and efforts to optimize performance.
• Adhere to established data modeling, coding, and platform standards while continually improving how data solutions are built and delivered.

Requirements

• Bachelor's degree in engineering or an equivalent background.
• More than five years designing and implementing cloud-based data solutions on Azure, Google Cloud, or AWS.
• Over five years with modern data platforms and technologies such as Azure Databricks, Apache Kafka or comparable streaming platforms, Snowflake, plus GitHub and GitHub Actions.
• At least two years hands-on experience with Python and/or Scala for data engineering workloads.
• Strong SQL skills for data querying and transformation, with familiarity in version control, CI/CD pipelines, and automated deployment practices for data platforms.
• Proven ability to collaborate effectively within Agile, distributed, and onshore/offshore teams.

Technologies

• Azure Databricks
• Apache Kafka
• Snowflake
• GitHub
• GitHub Actions
• Python
• SQL
• ServiceNow
• RESTful APIs
• Large Language Models (LLMs)
• Retrieval Augmented Generation (RAG)
• Azure
• Google
• AWS

Benefits

• Comprehensive benefits package
• Incentive and recognition programs
• Equity stock purchase plan
• 401(k) contribution

Application Deadline

This job posting remains active for a minimum of two business days or until a sufficient candidate pool is reached. The posting may be removed earlier if application volume is high.