Join Essentia Health as a Data Engineer II and help build reliable data platforms that support batch and near real-time analytics for business-critical tools. This role focuses on scalable pipeline design, data quality and governance, and secure data workflows, with opportunities to collaborate across data science and analytics teams. The position is located in Duluth, MN with a remote option and a day rotation schedule from 8:00 AM to 4:30 PM.
Compensation: $89,523.20 - $134,284.80 per year. FTE: 1. Building location: Peerless Building. Department: 1007670 ANALYTICS AND ARCHITECTURE - EH SS. Holidays: No. Call obligation: No.
What you’ll do
- Design and optimize scalable, robust data pipelines and workflows that support large datasets and meet functional and non-functional business requirements.
- Implement advanced data processing and scheduling approaches using frameworks such as Apache Spark, ADF, and Airflow.
- Write and optimize advanced SQL queries for reporting and application use cases.
- Support data quality, governance, and security using automated testing and monitoring solutions.
- Partner with data scientists, analysts, and other stakeholders to understand data requirements and translate them into reliable pipelines.
- Develop and maintain infrastructure that supports real-time and batch processing workflows.
- Provide technical guidance and mentorship to junior data engineers.
- Deliver actionable data products and insights to power critical organizational tools.
What you bring
- Bachelor’s degree in computer science, data analytics, statistics, or a related field.
- 2 to 5 years of experience in data engineering or a related role.
- Proficiency in SQL and experience with database management systems.
- Strong programming skills in Python, Java, or Scala.
- Experience building pipelines with orchestration tools such as Apache Airflow or AWS Glue for ingestion and delivery.
- Experience with modern data management technologies such as data lake, lakehouse, and data catalogs.
- Experience supporting both batch and streaming data to enable near real-time analytics and reporting.
Tools you may work with
Apache Spark, ADF, Airflow, SQL, Python, Java, Scala, AWS Glue, dbt, Microsoft Fabric, and modern data platforms including data lakes, lakehouses, and data catalogs. The role may also involve work related to AI and machine learning data use cases.
Preferred qualifications
- Familiarity with applying AI and machine learning techniques to data use cases.
- Experience creating robust data transformation workflows, including metadata management, dependency tracking, and workload orchestration.
- Experience with dbt.
- Experience with Microsoft Fabric.
- Excellent problem-solving skills and attention to detail.
- Strong communication and collaboration skills.
Benefits
- Medical, dental, vision, life, and disability insurance
- 401(k) plan with employer contributions
- Training, tuition reimbursement, and educational programs
- Flexible scheduling
- Generous time off
- Wellness resources focused on physical, mental, and emotional health
