Lead Data Engineer

Hybrid to Charlotte, NC preferred. Bartech Staffing seeks a Lead Data Engineer to own the design, architecture, and implementation of enterprise data engineering solutions across the AWS ecosystem. In this role, you will provide technical leadership, guide engineering teams through best practices, and help modernize data platforms with an emphasis on reliability, security, and long-term maintainability.

Responsibilities

• Lead the design, architecture, and implementation of enterprise data engineering solutions across the AWS ecosystem
• Partner with Lead Developers, Data Scientists, Architects, Product Owners, and business stakeholders to define technical strategy and scalable solutions
• Provide technical leadership and mentorship to Data Engineers and development teams, promoting engineering excellence and best practices
• Drive key architectural decisions with Data Architects and Solution Architects to support scalability, security, reliability, and maintainability
• Design and oversee data warehouse and data lake solutions that balance usability, performance, and long-term sustainability
• Set engineering standards for data modeling, ETL frameworks, pipeline reliability, monitoring, and operational excellence
• Lead end-to-end delivery to ensure alignment with business requirements, enterprise architecture standards, and regulatory requirements
• Oversee production support and operational management of AWS-based data platforms, including root-cause analysis and performance optimization initiatives
• Champion data quality, governance, observability, and data stewardship practices across platforms and teams
• Identify modernization opportunities to improve operational efficiency through automation and cloud-native technologies

Requirements

• 8+ years of experience in Data Engineering, including 5+ years working extensively within AWS ecosystems
• Expert-level AWS experience with S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, and Step Functions
• Experience creating AI applications with AWS Bedrock
• Experience designing and implementing enterprise-scale data lake and data warehouse solutions using Lake Formation, Amazon Redshift, and Amazon Athena
• Extensive experience with Kafka-based streaming architectures, preferably Confluent Kafka
• Advanced SQL and data modeling skills, including dimensional modeling and data vault, for large-scale data warehousing
• Deep experience building, developing, and optimizing scalable and resilient data pipelines within AWS environments
• Strong knowledge of distributed processing frameworks, particularly PySpark and EMR
• Expertise in database management principles, performance tuning, and data architecture best practices
• Advanced Python development skills with hands-on PySpark experience
• Expertise in Infrastructure as Code using Terraform
• Experience implementing CI/CD frameworks using GitHub and GitHub Actions
• Deep understanding of AWS IAM roles, policies, governance, and security best practices
• Experience with workflow orchestration tools such as AWS Step Functions, Apache Airflow, or equivalent platforms
• Experience leading cloud migration, modernization, and enterprise data platform initiatives
• Strong understanding of data governance, metadata management, data quality frameworks, and observability principles
• Ability to lead hands-on development while providing technical direction, code reviews, and engineering oversight across multiple initiatives
• Experience establishing development environments, infrastructure standards, security controls, and migration strategies across multiple AWS accounts and environments
• Ability to architect, develop, and govern enterprise-scale data pipelines, ETL processes, data ingestion frameworks, and orchestration workflows
• Proven ability to identify data gaps, define remediation plans, and implement scalable automation that improves analytical capabilities
• Ability to build highly reliable data pipelines with a strong focus on data quality, observability, resiliency, and operational supportability
• Experience building and optimizing large-scale data warehousing solutions that prioritize business user experience and system performance
• Ability to influence architectural direction and communicate complex technical concepts to technical and non-technical stakeholders
• Proven track record mentoring engineers, fostering technical growth, and building high-performing data engineering teams
• Experience leading cross-functional initiatives across data engineering, analytics, architecture, platform engineering, and business stakeholders
• Strong problem-solving and leadership skills in fast-paced enterprise environments with competing priorities
• Experience building AI-ready data pipelines and ML workflows, including feature engineering and MLOps
• Proficiency in Python, SQL, Spark, and Generative AI technologies for scalable data and analytics solutions
• Hands-on experience with AWS, Azure, or Databricks and cloud-based AI/data platforms
• Knowledge of LLMs, RAG, and vector databases to support AI-powered applications and intelligent search
• Strong understanding of data governance, quality, security, and responsible AI practices

Technologies

• AWS, S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions
• AWS Bedrock, Lake Formation, Amazon Redshift, Kafka, Confluent Kafka
• SQL, dimensional modeling, data vault, PySpark, Python
• Terraform, GitHub, GitHub Actions
• AWS IAM, Apache Airflow, ETL, CI/CD
• Generative AI, Azure, Databricks, LLMs, RAG, vector databases, MLOps, Spark

Role Summary / Purpose

• Lead Data Engineer – Hybrid to Charlotte preferred

Nice-to-have / Preferred Skills

• AWS Certified Data Engineer, AWS Solutions Architect, or equivalent cloud certifications
• Experience implementing enterprise data governance and metadata management platforms
• Experience with real-time analytics, event-driven architectures, and streaming data platforms
• Knowledge of modern data architecture patterns including Data Mesh, Lakehouse, and domain-oriented design
• Experience leading large-scale cloud transformation or enterprise data modernization programs