Senior, Data Engineer

Lead the design and delivery of scalable data pipelines and data products on AWS and Greenplum in a hybrid Chicago role.

Responsibilities

• Develop and execute software tests to identify software problems and their causes.
• Author and optimize complex SQL scripts and build Python scripts for large-scale transformations on Greenplum.
• Create test plans, run unit and integration tests, and deliver reproducible, well-documented results.
• Design and maintain AI-enabled PySpark data pipelines on AWS primarily using EMR, Glue, Athena, DynamoDB, and S3 to process large datasets at scale.
• Harden and tune jobs for performance, cost, and compliance; manage data partitioning, cataloging, and lineage.
• Automate deployments and infrastructure with CloudFormation and parameterized configurations; schedule batch workloads for timely and reliable pipelines.
• Build internal dashboards and data products (including Amplify apps) and instrument monitoring and alerting to catch issues early.
• Handle client tickets by converting ambiguous requests into clear technical tasks.

Requirements

• Master’s degree in Computer Science, Information Technology, Engineering, or related field.
• Employer will accept two years of experience in the job offered, or two years in any occupation where the required experience was gained.
• Build PySpark ETL on AWS EMR and Glue reading and writing to S3, including partitioning, predicate pushdown, and broadcast joins to deliver analytics-ready fact and dimension tables on schedule.
• Author and tune complex SQL on Greenplum/Postgres using window functions, ROW_NUMBER dedupe, distribution keys, AO/AOCO compression, and VACUUM/ANALYZE for efficient processing of large datasets.
• Manage AWS Glue Data Catalog and query curated layers with Athena; design S3 layouts (prefix/partition strategy, lifecycle policies) to reduce query time and storage cost.
• Implement CI/CD using CodeCommit, CodeBuild, and CodePipeline; parameterize environments and use CloudFormation for infrastructure as code to support repeatable, auditable workflows and fast rollbacks.
• Engineer observability with CloudWatch logs/metrics/alarms and job health checks; create SLA dashboards and alerting to detect failures early and reduce MTTR.
• Enforce data quality with SQL/Python unit tests and reconciliation (row counts, sums, duplicates), referential integrity checks, and documented UAT evidence for change control.
• Apply security and data-governance controls: normalize and encrypt sensitive identifiers, use KMS-managed keys, and enforce IAM least privilege with audit logging.
• Develop GenAI-assisted delivery integrating LLMs on AWS Bedrock with PySpark/SQL to auto-suggest transformations, generate test cases, and summarize validation results.
• Build lightweight internal data apps with AWS Amplify and DynamoDB/Athena to manage lookup keys and brand lists and enable stakeholder self-service with controlled access.

Technologies

• SQL, Python, Greenplum, PySpark, AWS EMR, AWS Glue, AWS Athena
• Amazon DynamoDB, Amazon S3
• CloudFormation, AWS CodeCommit, AWS CodeBuild, AWS CodePipeline
• Amazon CloudWatch, AWS KMS, AWS IAM
• AWS Bedrock, AWS Amplify, Postgres

Benefits

• Flexible time off (FTO), 14 paid holidays, paid sick time
• Parental/new child leave, childcare & elder care assistance, adoption assistance
• Comprehensive health coverage
• 401(k)
• Tuition assistance
• Commuter benefits
• Professional development
• Employee recognition
• Charitable donation matching
• Health coaching and counseling

Job Location & Work Schedule

• 35 West Wacker Drive, Chicago, IL 60601
• Required to work from the Company Office in Chicago certain days per week
• Telecommuting allowed for the balance of the week

Compensation Range

• USD $145,870 - $182,000 per year

Additional Information

• Epsilon is a global leader in data and identity, with expertise in digital media, clean rooms, customer data platforms, loyalty and marketing services.
• Epsilon operates in 30+ countries across EMEA, the Americas and APAC, with more than 8,000 employees.
• Culture values: Act with integrity; Work together to win together; Innovate with purpose; Respect all voices; Empower with accountability.
• Equal opportunity employer; accommodations available for applicants who need support during the application process. Contact: [email protected].
• For San Francisco Bay and Los Angeles areas: consideration of qualified applicants with criminal histories consistent with applicable fair chance ordinances.