Senior Data Engineer

Vytwo is hiring a Senior Data Engineer to help design and maintain scalable data pipelines and keep Databricks and Spark workloads running efficiently for analytics-ready data. This role is based in Minneapolis, MN (onsite), with flexible work from home options available.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Apache Spark with PySpark and/or Scala
• Build and optimize Databricks data workflows, including Delta Lake, notebooks, and scheduled jobs
• Ingest, transform, and curate large-scale structured and semi-structured datasets
• Perform performance tuning and cost optimization for Spark workloads and Databricks clusters
• Implement data quality checks, monitoring, and error handling
• Collaborate with analytics and business stakeholders to deliver well-modeled, analytics-ready data
• Support batch processing and, where applicable, streaming data pipelines
• Follow best practices for testing, documentation, security, and version control

Requirements

• 5+ years of experience in Data Engineering or a related role
• Strong hands-on experience with Apache Spark (with PySpark or Scala)
• Proven experience working in Databricks environments
• Strong SQL skills and experience with relational and analytical databases
• Experience building and maintaining ETL/ELT pipelines at scale
• Familiarity with modern data lake architectures (including Delta Lake preferred)
• Experience with Git-based version control
• Must be a U.S. Citizen
• Must be currently located in Minnesota (MN), or willing to relocate

Technology Focus

Apache Spark, PySpark, Scala, Databricks, Delta Lake, Git, SQL