EngineerJobs.io
← Back to all jobs
System One

Senior AWS Python Developer / PySpark Data Engineer

System One
Lafayette, TN · Columbia, SC Full time Posted 11d ago
Backend Developer Python Analytics Analytics Engineer Aws Data Engineer Aws Data Engineering Aws Data Lake Aws Data Lakehouse Aws Data Services Aws S3 Data Lake Aws Solutions Architect Big Data Bigdata Cloud Data Engineering Data Analysis Data Analytics Data Architecture Data Engineer Data Engineering Data Integration Data Integration Tools Data Pipeline Data Platform Data Processing ETL Pyspark Software Development Spark SQL
Apply Now

Job Description

System One is hiring a Senior AWS Python Developer / PySpark Data Engineer for a hybrid role across Lafayette, LA, Knoxville, TN, Columbia, SC, and Birmingham, AL (direct hire). This is a hands-on position focused on AWS big data and data processing, with opportunities to build scalable Python and optimize production workloads end to end.

You will work at the intersection of data engineering and application integration, developing production-grade data processing solutions, improving performance on large datasets, and supporting automation through CI/CD. Strong fit areas include experience in cybersecurity or security data environments.

Responsibilities

  • Develop scalable Python applications and data processing solutions for production workloads
  • Integrate APIs to connect data sources and application services
  • Write, optimize, and maintain complex SQL queries for data transformations
  • Process large datasets using Apache Spark and AWS services
  • Troubleshoot application and data processing performance issues
  • Contribute to CI/CD automation activities
  • Collaborate with technical and business teams across a matrixed organization

Requirements

  • 8+ years of hands-on Python development, including data-intensive and big data applications
  • Strong Python fundamentals with experience building scalable, production-grade applications
  • 5+ years working with Apache Spark / PySpark, including troubleshooting and optimization of Spark jobs
  • 5+ years hands-on development with AWS, including EMR, Lambda, EventBridge, Step Functions, Redshift, S3, and Glue
  • Advanced SQL skills, including complex queries, transformations, joins, and recursive CTEs
  • Experience developing and consuming REST APIs and integrating with external services and data sources
  • Strong background working with large datasets including manipulation, transformation, analysis, and processing
  • Experience with big data technologies such as Hadoop / EMR, Spark, and Hive
  • 4+ years of experience working with AI-related technologies or applications
  • Hands-on CI/CD and infrastructure automation experience using GitLab and Terraform
  • Strong debugging, performance tuning, analytical, and problem-solving skills
  • Ability to collaborate effectively across development teams and a matrixed organization
  • Bachelor’s degree in Computer Science, Information Systems, or a related field

Technologies: Python, PySpark, Apache Spark, SQL, Amazon EMR, AWS Lambda, Amazon EventBridge, AWS Step Functions, Amazon Redshift, Amazon S3, AWS Glue, Hadoop, Hive, REST APIs, GitLab, Terraform, CI/CD, AWS services

Type: Direct Hire

Work Model: Hybrid

Similar Jobs