Sr Data Engineer
Senior
Apache Airflow
Azure
Big Data
Bigdata
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Data Warehouse
Cloud Data Warehouse
Cloud Engineer
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Build Tool
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Management
Data Modeling
Data Operations
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Transforms
Data Warehouse
Data Warehousing
Database
Databases
ETL
Google Cloud Bigquery
Informatica
Information Technology (IT)
Programming Languages
Snowflake
Software Development
Spark
SQL
Workflow Orchestration
Job Description
Build and scale enterprise data pipelines as a Senior Data Engineer in a hybrid Chicago role.
Responsibilities
- Lead the design, architecture, and implementation of complex data pipelines and storage solutions
- Mentor teammates and peers with technical guidance and expertise
- Assess and recommend new data technologies and tooling to improve efficiency
- Partner with data scientists to optimize and scale machine learning workflows
- Develop advanced data transformations and aggregation processes
- Establish and drive data engineering best practices and team standards
- Participate in cross-functional project planning and execution
- Manage and optimize cloud resources and costs for data processing
- Improve data quality, data lineage, and metadata management
- Collaborate with business stakeholders to understand and prioritize data needs for strategic initiatives
Requirements
- 6+ years of relevant experience in data engineering or a related field
- Advanced proficiency in Python and SQL; expert proficiency in alternative general programming languages considered
- Extensive experience with data orchestration tools such as Prefect or Airflow
- Strong experience with dbt and data warehouse technologies including Snowflake and BigQuery, or data processing frameworks such as Apache Spark
- Extensive experience with version control systems and CI/CD tools
- Hands-on experience with one major cloud platform required; Azure preferred
- Leadership skills for mentoring and guiding junior team members
- Problem-solving skills for complex data engineering challenges
- Effective communication across cross-functional teams
- Project management abilities for overseeing multiple data projects
Technologies
- Python, SQL
- Prefect, Airflow
- dbt
- Snowflake, BigQuery
- Apache Spark
- Azure
- Data orchestration tools, version control systems, CI/CD tools
Compensation
- $155,000 - $189,000 per year
- Discretionary 13% annual bonus
Work Model and Location
- Chicago, IL (hybrid)
- Onsite Monday through Thursday
- Remote work option on Fridays
Physical and Visual Activities
- Ability to hear and speak well enough to converse verbally over the phone and in-person
- Ability to interpret, evaluate, and communicate detailed written or verbal instructions accurately
- Ability to see well enough to use a computer and read written communications
- Ability to lift, push, pull, or carry up to 15 pounds for moving documents, supplies, and materials
- May be required to climb stairs depending on the property structure
- Reasonable accommodation will be provided as required by law for otherwise qualified employees with a known disability