AWS Glue Data Engineer
Backend Developer
Amazon Web Services
AWS
Aws Cloudfront
Aws Data Catalog
Aws Glue
Aws Govcloud
Aws S3 Data Lake
Big Data
Bigdata
Cloud
Cloud Computing
Cloud Data Warehouse
Cloud Data Warehouse
Cloud Platform
Cloud Platforms
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Integration
Data Lake
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Data Warehousing
Database
ETL
Informatica
IT Services
Pyspark
Snowflake
Spark
SQL
Job Description
This role focuses on engineering scalable ETL data pipelines using AWS Glue and PySpark/Spark, with tight integration to Snowflake and governance controls through AWS Lake Formation. You will also support a GovCloud proof of concept and convert existing SQL logic into maintainable PySpark while validating data quality and performance.
Responsibilities
- Design, build, test, and optimize scalable ETL pipelines using AWS Glue and PySpark/Spark
- Configure and maintain AWS Glue Crawlers, Data Catalog objects, jobs, workflows, and connections
- Integrate AWS Glue workloads with Snowflake using SQL and supported connectors
- Implement data access, governance, and security controls with AWS Lake Formation
- Collaborate with the cloud engineering team to deliver a proof of concept in a GovCloud environment
- Convert existing SQL logic into maintainable PySpark code and validate data quality and performance
- Document architecture, configuration, deployment steps, test results, risks, and operational handover
- Participate in the on-site Phoenix buildout week and support troubleshooting through proof of concept completion
- Provide strong written and verbal communication and coordinate directly with technical stakeholders in a fast-paced proof of concept environment
- Maintain ownership of deliverables, risks, dependencies, and provide timely status reporting
Requirements
- Hands-on AWS Glue expertise, including PySpark/Spark ETL development, Crawlers, and Data Catalog
- Snowflake experience, including SQL, connectors, and basic administration
- Data governance experience using AWS Lake Formation
- Eligibility to work on ITAR/export-controlled GovCloud engagements; US citizenship is required
- Ability to work on-site in Phoenix, Arizona during the buildout week
Preferred Qualifications
- Experience with Snowflake Catalog Federation and/or Iceberg REST Catalog
- Experience with Kiro or another agentic IDE for SQL-to-PySpark code conversion
- Experience with Hub-and-Spoke VPC architecture and inspection firewalls, preferably Check Point
Technology Focus
- AWS Glue, PySpark, Spark
- AWS Glue Crawlers, AWS Glue Data Catalog objects, AWS Glue jobs, AWS Glue workflows, AWS Glue connections
- Snowflake, SQL
- AWS Lake Formation
- GovCloud
Location and Compensation
- Location: Phoenix, AZ (onsite)
- Salary: USD 120,000 per year
Similar Jobs
