EngineerJobs.io
← Back to all jobs
SN Cloud Solutions

Data Engineer

SN Cloud Solutions
Chicago, IL $0k+/yr Full time Posted 3d ago
Python Apache Airflow Azure Data Factory Azure Synapse Analytics Big Data Bigdata Bigquery Cloud Cloud Platform Data Architecture Data Engineer Data Integration Data Lake Data Lakehouse Data Pipeline Data Pipelines Data Platform Data Processing Data Warehouse Data Warehousing Database Databases Databricks Delta Lake ETL Kafka Microsoft Azure Snowflake Spark SQL Stream Processing
Apply Now

Job Description

Data Engineer position in Chicago, IL (hybrid) focused on designing, building, and maintaining scalable data pipelines and analytics platforms.

Responsibilities

  • Design, develop, and optimize ETL and ELT pipelines for large-scale data processing.
  • Build scalable data solutions using Python, PySpark, and Apache Spark.
  • Develop and maintain data ingestion frameworks for structured and unstructured data.
  • Craft and optimize SQL queries, stored procedures, and related database objects.
  • Consolidate data from multiple sources, including APIs, databases, and cloud storage.
  • Establish data quality checks, validation, and monitoring mechanisms.
  • Collaborate with cloud platforms such as AWS, Azure, or GCP.
  • Construct and maintain data lakes and data warehouses.
  • Work with data scientists, analysts, and business stakeholders to inform analytics needs.
  • Automate workflows with orchestration tools such as Apache Airflow.
  • Troubleshoot production issues and optimize pipeline performance.
  • Follow Agile/Scrum practices and CI/CD workflows.

Requirements

  • Azure Data Factory experience with 7+ years (Required).
  • SQL proficiency with 3+ years (Required).
  • Data warehousing concepts with 4+ years (Required).
  • Strong Python and PySpark expertise, with hands-on Apache Spark experience.
  • Experience developing ETL/ELT processes.
  • Cloud platform experience across AWS, Azure, or GCP.
  • Solid knowledge of data modeling and database design.
  • Experience with Apache Airflow or similar orchestration tools.
  • Experience with Git, Jenkins, or Azure DevOps.
  • Experience with both relational and NoSQL databases.
  • Excellent problem-solving abilities and clear communication skills.

Technologies

  • Python
  • PySpark
  • Apache Spark
  • SQL
  • AWS
  • Azure
  • GCP
  • Apache Airflow
  • Databricks
  • Kafka
  • Delta Lake
  • Snowflake
  • Redshift
  • BigQuery
  • Synapse
  • Azure Data Factory
  • Git
  • Jenkins
  • Azure DevOps

Benefits

  • Flexible schedule

Job Details

  • Job Type: Full-time
  • Pay: From $50.00 per year
  • Location: Hybrid remote in Chicago, IL 60618
  • Experience requirements: Azure Data Factory 7+ years; SQL 3+ years; Data warehouse 4+ years

Similar Jobs