Senior Data Engineer – Spark / Scala

Astyra Corporation is seeking a Senior Data Engineer to help modernize legacy ETL workloads and deliver Spark-based data pipelines. This contract role is focused on ensuring data accuracy and consistency while validating results between legacy and target platforms, alongside performance optimization for production workloads.

Based in Richmond, VA with hybrid remote work, the position supports end-to-end pipeline design, transformation development, and migration activities across enterprise data modernization initiatives.

Key Responsibilities

• Design, build, and maintain scalable data pipelines using Apache Spark and Scala
• Support migration of legacy ETL workloads into a modern Spark-based data framework
• Develop and enhance data transformation and processing solutions
• Perform data comparison, reconciliation, and validation between legacy and target platforms
• Identify and troubleshoot data discrepancies throughout the migration process
• Develop data quality checks and controls to ensure completeness and accuracy
• Optimize Spark jobs and pipelines for performance, scalability, and reliability
• Collaborate with data engineers, architects, and other technical teams during modernization efforts
• Support testing, deployment, troubleshooting, and ongoing maintenance of production data pipelines
• Contribute to broader feature engineering and data modernization initiatives

Required Qualifications

• 7+ years of experience in data engineering or a related technical discipline
• Strong hands-on experience designing and developing enterprise data pipelines
• Advanced experience with Apache Spark
• Strong development experience with Scala
• Experience with ETL development, modernization, and migration initiatives
• Strong understanding of data transformation, data quality, and reconciliation
• Experience comparing and validating data across legacy and modernized systems
• Experience developing and supporting production data-processing workloads
• Familiarity with modern CI/CD and software development practices
• Strong troubleshooting and problem-solving skills

Technologies

• Apache Spark, Spark
• Scala
• Python
• ETL
• CI/CD

Preferred Qualifications

• Experience migrating legacy ETL platforms to Spark-based architectures
• Experience supporting large-scale technology or data modernization initiatives
• Experience within financial services or other highly regulated environments
• Python experience
• Exposure to Generative AI or AI-assisted development and modernization tools
• Previous experience within a large enterprise engineering environment

Compensation and Schedule

• Pay: USD $66.00 - $71.00 per hour
• Expected hours: 40.0 per week
• Job type: Contract

Location

• Richmond, VA 23238 (Required)
• Work location: Hybrid remote in Richmond, VA 23238

Benefits

• 401(k)
• Dental insurance
• Health insurance
• Vision insurance