Lead Clinical Data Engineer

Yale University is seeking a Lead Clinical Data Engineer to help advance biomedical research data capabilities for the Yale School of Medicine and the Yale New Haven Health System. In this remote role based in New Haven, CT, you will lead technical efforts around the Aligned Data Warehouse (ADW), expanding functionality, integrating new sources, and delivering reliable ETL and data pipeline operations that support downstream analytics and research data sharing initiatives.

Role focus

• Lead the design and development of the Aligned Data Warehouse (ADW) for biomedical research.
• Expand ADW functionality by integrating new data sources, building and maintaining ETL/data pipelines, and supporting downstream analytics.
• Contribute to initiatives supporting research data sharing.

Responsibilities

• Design, build, test, maintain, and control data pipelines and ETL jobs to integrate data into the ADW and other research databases.
• Contribute to DevOps and documentation, developing standards and procedures for database and pipeline design, operational management, and ongoing maintenance.
• Design, build, test, maintain, and control additional data management processes, including flat file handling in formats such as CSV, TSV, pipe-delimited, XML, and JSON.
• Facilitate design sessions, code walkthroughs, and peer reviews; produce technical documentation, SOPs, and KBAs.
• Performance tune database objects, stored procedures, ETL jobs, and related scripts to optimize end-user queries and ETL processing.
• Monitor scheduled ETL jobs and other processes to ensure expected functioning and uptime of data pipelines.
• Lead and/or coordinate troubleshooting and remediation for ETL job failures in a timely manner.
• Deliver world-class customer service across interactions with customers, stakeholders, and other teams.
• Maintain patient privacy and ensure the integrity and security of healthcare data in all databases and systems, following applicable laws, regulations, and institutional policies related to IRB, patient privacy, and IT cybersecurity.
• Independently investigate and stay current on new and emerging technologies relevant to Scientific Computing & Data to improve efficiencies, quality, or operating cost.
• Lead projects and data stewardship strategies, working closely with the DBA team and vendors to optimize pipelines.
• Assist with the design, build, test, and performance tuning of reports and dashboards using SSRS, Microsoft Power BI, or similar business intelligence tools.
• Perform related duties as assigned or requested.

Minimum requirements

• Minimum 4 years of experience in the field or a related area.
• Bachelor’s degree in a related area.
• Extensive experience developing, automating, and optimizing data integration workflows with a focus on Microsoft Fabric Data Pipelines for large-scale ingestion, transformation, and integration workloads.
• Proven ability to design, develop, and optimize scalable data models, semantic layers, and storage architectures supporting enterprise reporting, self-service analytics, operational intelligence, and advanced analytics.
• Experience building and maintaining Fabric Notebooks using PySpark, Spark SQL, and related technologies for transformation, enrichment, validation, and automation.
• Demonstrated experience designing and implementing Microsoft Fabric Lakehouses, Warehouses, and enterprise data organization strategies supporting governance, performance, security, and scalability.
• Ability to collaborate with business stakeholders, analysts, architects, and technical teams to translate complex requirements into scalable data engineering solutions.

Technologies

• Microsoft SQL Server, T-SQL, SSIS, SQL Agent
• Microsoft Fabric, Metadata-driven dynamic SQL, Linux, Windows
• OMOP Common Data Model and OHDSI OMOP (Observational Medical Outcomes Partnership)
• SSRS, Microsoft Power BI, PySpark, Spark SQL
• Fabric Notebooks, Microsoft Fabric Data Pipelines, Microsoft Fabric Lakehouses, Microsoft Fabric Warehouses
• Large Language Models, GPU-based software
• Flat file formats: CSV, TSV, pipe-delimited, XML, JSON
• Healthcare data systems: Epic’s data warehouse, Epic’s Clarity & Caboodle, PCORnet, OnCore, IRES - IRB

Preferred skills and abilities

• In-depth knowledge of associated technology areas impacting the role, including healthcare technology experience.
• Experience integrating EHR, clinical, research, financial, or operational healthcare data in cloud-based analytics platforms.
• Certification and experience with Epic’s Clarity & Caboodle databases.
• Experience with OMOP Common Data Model, PCORnet, and familiarity with research administrative systems including OnCore or IRES - IRB.

Preferred education and experience

• Master’s degree.
• Experience in data engineering, analytics engineering, or data integration, including at least five (5) years designing, developing, and maintaining enterprise-scale healthcare data platforms, ETL processes, and cloud-based data solutions.

Compensation, schedule, and work model

• Salary: USD 90,000 - 165,750 per year (salary range: $90,000.00 - $165,750.00).
• Time type: Full time.
• Duration type: Staff.
• Work model: Remote.
• Location: New Haven, CT (remote).
• Job category: Professional.
• Compensation grade: Administration & Operations (Manager; Program Leader (P6)).
• Bargaining unit: NON.
• Job posting date: 10/13/2026.

Background and health requirements

• Employment is subject to pre-employment background screening for this position, which may include motor vehicle, DOT certification, drug testing, and credit checks based on job requirements.
• Offers are contingent upon successful completion of the background check.
• Certain positions have health requirements based on job responsibilities and may include vaccinations, tests, or examinations as required by law, regulation, or university policy.

Benefits

A general description of the benefits for this Managerial & Professional position is available at: Yale Benefits.

Posting disclaimer: Salary offers are determined by qualifications, experience, skills, and education relative to position requirements, along with the role’s grade profile and internal and external market conditions. This job description provides a representative summary of essential functions and is not a declaration of specific duties and responsibilities. Employees will be assigned specific job-related duties through their hiring department.

Note: Yale University is a tobacco-free campus.