Data Engineer

Shift4 is seeking a hands-on Data Engineer to build and continuously improve SQL- and Python-based data jobs that run in containers. In this role, you will transform raw payments data in Snowflake into dependable datasets and reports for internal teams and merchant-facing consumers, with a strong focus on correctness, reconciliation, and cost-aware execution.

This onsite position is based in Center Valley, PA.

What you’ll do

• Design, write, and maintain Python jobs that execute SQL over Snowflake to deliver transaction, settlement, fee, and reconciliation datasets
• Develop performant, readable SQL including window functions, CTEs, incremental logic, and VARIANT or JSON handling across large transaction tables
• Package and run workloads as Docker containers using the existing scheduler, ensuring each job is idempotent, rerunnable, parameterized, and observable with logging and alerts for failure or bad output
• Implement data-quality checks and reconciliation controls so totals align across sources and reporting periods
• Manage Snowflake cost through warehouse sizing, clustering, query profiling, and minimizing wasteful scans
• Partner with finance, operations, and product stakeholders to translate payments questions into correct, maintainable queries
• Apply security controls for a payments environment, including secrets handling, least-privilege access, PII masking, and audit trails
• Document the job library and contribute to code review, testing, and CI practices
• Actively explore and apply AI tools to improve efficiency across query authoring, testing, and documentation

What you’ll bring

• 1 to 3 years of experience in data engineering, analytics engineering, or backend roles building SQL-heavy data workloads in production
• Strong SQL skills: window functions, CTEs, aggregation and joins over large tables, query tuning, and the ability to read execution plans
• Hands-on Snowflake experience including warehouses, credit and cost awareness, stages and COPY INTO, streams and tasks, time travel, and VARIANT semi-structured data; experience with Snowpark or the Snowflake Python connector
• Practical Python for data work, including pandas or Polars, parameterized SQL, configuration and secrets management, structured logging, error handling, and unit tests
• Docker fundamentals: building images, running scheduled batch jobs in containers, and debugging container failures
• Comfort with Git and basic CI/CD habits, including code review and writing testable, rerunnable jobs
• A careful approach to numerical correctness, including investigating when totals do not tie out

Technologies

Python, SQL, Snowflake, Docker, pandas, Polars, Git, Snowpark, Snowflake Python connector, VARIANT, JSON, CTEs, window functions, and GitHub-style code review and CI/CD (generic).

Nice to have

• Payments domain knowledge: authorization, capture, settlement, refunds, chargebacks, interchange and scheme fees, plus merchant and acquirer data models and reconciliation
• Experience with orchestration and transformation tooling such as Airflow, Prefect, Dagster, or dbt
• AWS exposure from the developer side: S3, IAM, Secrets Manager, and ECS/Fargate scheduled tasks
• Data-quality frameworks and monitoring such as Great Expectations, dbt tests, Soda, and pipeline monitoring or alerting
• Experience in a PCI or otherwise regulated environment
• Exposure to BI tools consuming outputs such as Sigma, Tableau, Power BI, or Looker