Data Engineer

AmeriHealth Caritas is seeking an Azure Data Engineer to design, build, and optimize large-scale cloud data solutions on Microsoft Azure. The role supports ingestion across varied sources, including real-time streaming, dimensional modeling, and delivery within Agile sprints.

Responsibilities

• Design and implement scalable solutions using Azure Data Factory, Azure Databricks, Azure Event Hubs, Kafka, HVR, Snowflake, and Azure Data Lake Storage
• Build robust ETL/ELT pipelines for batch and streaming workloads using PySpark and SQL, ensuring fault tolerance, schema evolution support, and idempotency
• Design and operate event-driven ingestion and processing with Event Hubs and Kafka, and integrate CDC using HVR
• Tune Databricks jobs and clusters to meet SLAs and cost targets, including cluster sizing, autoscaling, caching, partitioning, shuffle optimization, and Delta Lake optimizations
• Use Azure DevOps to manage backlogs, support Git workflows, create YAML pipelines, enable automated testing, and deploy repeatably across environments
• Conduct code reviews, enforce coding standards and best practices, and mentor junior developers
• Partner with product owners, data scientists, analysts, and architects to translate business needs into prioritized Agile user stories and deliverables
• Automate manual processes, optimize data delivery, and redesign infrastructure to improve scalability
• Identify and resolve data management issues to improve data quality
• Apply best practices for data management, including master data, reference data, metadata, data quality, and lineage

Requirements

• Bachelor’s degree in computer science, Information Systems, or Software Engineering
• Hands-on experience: Azure Data Factory (4+ years), Azure Databricks (4+ years), HVR (2+ years), Kafka (4+ years), Event Hubs (2+ years), Snowflake (4+ years), Azure Data Lake Storage, PySpark (8+ years), and SQL (8+ years)
• Expertise in dimensional modeling, SCD implementation, and data warehouse best practices
• Proven delivery experience for data projects in Agile teams (Scrum or Kanban)
• Practical experience with Azure DevOps for CI/CD, branching strategies, and pipeline automation
• Demonstrated ability to profile and optimize Databricks jobs and large-scale data processing workloads
• Track record of conducting code reviews and providing technical guidance
• Clear communication skills with the ability to present technical tradeoffs to technical and non-technical stakeholders
• Experience with Collibra data governance features
• Familiarity with MLOps patterns and integrating data platforms with ML pipelines
• Azure certifications such as DP 700, AZ 400, or Databricks Data Engineer certification
• Experience with cost optimization and observability tools for cloud data platforms
• Practical experience applying Gen AI (LLMs, embeddings, retrieval augmentation) to data engineering workflows

Technologies

• Azure Data Factory, Azure Databricks, Azure Event Hubs, Kafka, HVR, Snowflake, Azure Data Lake Storage
• PySpark, SQL, Delta Lake, Azure DevOps, Git, YAML
• Event Hubs, CDC solutions, SCD, Dimensional modeling
• Scrum, Kanban, Collibra, MLOps, ML pipelines
• Gen AI, LLMs, embeddings, retrieval augmentation
• DP 700, AZ 400, Databricks Data Engineer certification

Benefits

• Flexible work solutions including remote options and hybrid work schedules
• Competitive pay
• Paid time off including holidays and volunteer events
• Health insurance coverage for you and your dependents on Day 1
• 401(k)
• Tuition reimbursement
• More

Work Arrangement

Hybrid position based in the Ellis Preserve office in Newtown Square, PA. Core office days are Tuesday, Wednesday, and Thursday.

Remote/Hybrid Requirements

• For roles that are 100% remote or hybrid: minimum bandwidth of 50 Mbps download and 5 Mbps upload is required.
• Fully remote associates residing in states where service is required by contract, law, or regulation may be allowed to submit for reimbursement.

Minimum Experience

4+ years