Principal Data Engineer

Abbott Laboratories is hiring a Principal Data Engineer to lead cross-domain enterprise data engineering initiatives in an onsite role in Madison, WI. This is a senior individual contributor position focused on technical leadership, enterprise standards, and hands-on delivery across prototypes, production, and lifecycle support. The role also provides architecture guidance that improves consistency, reduces enterprise cost and risk, and strengthens reliability and governance.

Salary: USD 129,300 - 258,700 per year.

Responsibilities

• Own technical outcomes for assigned cross-domain initiatives, moving from initial ambiguity through production and ongoing lifecycle support.
• Translate enterprise needs into integrated architecture and executable technical plans, breaking complex work into deliverable increments across domains.
• Coordinate execution across domain Staff Engineers and partner teams, identifying dependencies and architectural risks and escalating decisions needing business or delivery authority.
• Stay hands-on through prototypes, reference implementations, critical-path development, technical validation, and production problem solving.
• Define, steward, and evolve enterprise data engineering standards, patterns, and reference architectures, and drive convergence when implementations are inconsistent or duplicative.
• Lead architecture for enterprise capabilities, including semantic and metrics layers, canonical data models, and batch, API, event-driven, and streaming patterns for governed data products supporting analytics, machine learning, and AI.
• Establish enterprise data-contract standards (schemas, service expectations, compatibility, breaking-change policy, and producer-consumer responsibilities) and collaborate with Platform Engineering to convert recurring needs into shared capabilities.
• Lead enterprise design and code reviews for high-complexity or cross-domain work and mentor Staff and Senior Engineers across domains without formal people management responsibility.
• Communicate architecture and technical tradeoffs to engineering, business, and executive stakeholders, and produce reusable guidance for consistent engineering across Enterprise Data.
• Evaluate architecture tradeoffs across reliability, scalability, performance, security, privacy, operability, adoption, and technical cost, including cost-efficient use of compute, storage, streaming, and orchestration.
• Act as enterprise technical escalation for incidents involving multiple domains or shared architecture patterns; lead root-cause analysis and preventive changes for recurring issues.
• Design and review architectures that handle protected health information to meet applicable security, privacy, lineage, audit, Quality Management System, HIPAA, CLIA, and regulatory requirements.
• Provide technical direction and due diligence for vendor and external-partner solutions, advancing responsible engineering practices including approved AI-assisted development capabilities.
• Ability to work nights and/or weekends, as needed.

Requirements

• Bachelor's Degree in Data Science, Computer Science, Information Systems, Mathematics, or Engineering.
• Expert-level experience in software development design and development, with relevant domain specific skills.
• Spark on Databricks (or comparable platforms); experience with Python, Scala, SQL, and Snowflake.
• ETL and ELT data pipelines, including batch and event-driven patterns.
• Experience designing and implementing data modeling solutions using relational, dimensional, and/or NoSQL databases.
• Database architecture testing methodology, including executing test plans, debugging, and testing scripts and tools.
• Open data file and table formats including Parquet, Avro, and Delta Lake, along with cloud infrastructure and delivery services such as AWS, S3, SQS, and GitLab CI/CD.
• REST API development; familiarity with BI concepts and Tableau performance considerations.
• Agile development tools including JIRA and Confluence.
• Ability to lead through influence across multiple teams and communicate complex technical decisions to senior engineering, business, and executive stakeholders.
• Ability to perform essential job duties with or without accommodation.

Technologies

Databricks, Unity Catalog, Spark, Python, Scala, SQL, Snowflake, ETL, ELT, Parquet, Avro, Delta Lake, AWS, S3, SQS, GitLab CI/CD, REST API, Tableau, JIRA, Confluence, Kafka, change data capture, NoSQL, HIPAA, CLIA, FHIR, GitLab, Azure, Google Cloud Platform.

Preferred Qualifications

• Databricks, Apache Spark, Delta Lake, and Unity Catalog at enterprise scale.
• Kafka, change data capture, and production event-streaming architectures.
• Semantic or metrics layer design, canonical data models, and governed data products.
• Cloud data architecture in AWS, Azure, or Google Cloud Platform.
• Life sciences, diagnostics, or clinical laboratory environments involving protected health information, including HIPAA, CLIA, FDA, or Quality Management System requirements.
• Technical assessment and architecture direction for vendor and external-partner platforms.