Lead Data Engineer

Vertage is seeking a Lead Data Engineer to design, architect, and deliver enterprise data engineering solutions across the AWS ecosystem. This hybrid role in the United States focuses on scalable data pipelines, streaming architectures, and secure, well-governed data lake and data warehouse platforms, with ongoing operational excellence.

Role focus

You will provide technical leadership for end-to-end delivery, covering architectural decisions, production support, data governance, and modernization efforts. The role emphasizes reliability, observability, and data quality across AWS-based platforms, alongside the development of AI-ready data pipelines.

Key responsibilities

• Lead the design, architecture, and implementation of enterprise data engineering solutions across AWS
• Collaborate with Lead Developers, Data Scientists, Architects, Product Owners, and business stakeholders to define technical strategy and scalable solutions
• Provide technical leadership and mentorship to Data Engineers and development teams, promoting engineering excellence and best practices
• Drive architectural decisions with Data Architects and Solution Architects to support scalability, security, reliability, and maintainability
• Design and oversee data warehouse and data lake solutions that balance usability, performance, and long-term sustainability
• Establish engineering standards for data modeling, ETL frameworks, pipeline reliability, monitoring, and operational excellence
• Lead end-to-end solution delivery aligned to business requirements, enterprise architecture standards, and regulatory requirements
• Oversee production support and operational management of AWS-based data platforms, performing root-cause analysis and performance optimization
• Champion data quality, governance, observability, and data stewardship across platforms and teams
• Identify modernization opportunities to improve operational efficiency using automation and cloud-native technologies

Experience and requirements

• 8+ years of Data Engineering experience, including 5+ years working extensively within AWS ecosystems
• Expert-level experience with AWS services including S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, and Step Functions
• Experience creating AI applications with AWS Bedrock
• Strong experience designing and implementing enterprise-scale data lake and data warehouse solutions using Lake Formation, Amazon Redshift, and Amazon Athena
• Extensive experience with Kafka-based streaming architectures, preferably Confluent Kafka
• Advanced SQL and data modeling expertise, including dimensional modeling and data vault for large-scale warehousing
• Deep experience building and optimizing scalable, resilient data pipelines within AWS, including distributed processing (notably PySpark and EMR)
• Advanced Python development with extensive hands-on experience using PySpark
• Expertise in Infrastructure as Code using Terraform
• Experience designing and implementing CI/CD frameworks using GitHub and GitHub Actions
• Deep knowledge of AWS IAM roles, policies, governance, and security best practices
• Experience with workflow orchestration tools such as AWS Step Functions or Apache Airflow (or equivalents)
• Experience leading cloud migration, modernization, and enterprise data platform initiatives
• Strong understanding of data governance, metadata management, data quality frameworks, and observability principles
• Ability to lead hands-on development with technical direction, code reviews, and engineering oversight across multiple initiatives
• Experience establishing development environments, infrastructure standards, security controls, and migration strategies across multiple AWS accounts and environments
• Proven ability to identify data gaps, define remediation plans, and implement scalable automation that improves organizational analytics
• Experience building and optimizing large-scale data warehousing solutions with strong business usability and system performance
• Ability to influence architectural direction and communicate complex technical concepts to technical and non-technical stakeholders
• Proven track record mentoring engineers and building high-performing data engineering teams
• Experience leading cross-functional initiatives across data engineering, analytics, architecture, platform engineering, and business stakeholders
• Strong problem-solving and leadership skills in fast-paced enterprise environments
• Experience building AI-ready data pipelines and ML workflows, including feature engineering and MLOps
• Proficiency in Python, SQL, Spark, and Generative AI technologies
• Hands-on experience with AWS, Azure, or Databricks and cloud-based AI/data platforms
• Knowledge of LLMs, RAG, and vector databases to support AI-powered applications and intelligent search
• Strong understanding of data governance, quality, security, and responsible AI practices

Other details

• Location: United States (hybrid); hybrid to Charlotte is preferred
• Hourly rate: USD 80–85 per hour
• Start and end dates: 12 Oct 2026 – 24 Oct 2027
• Respond by: 10 Oct 2026
• Preferred qualifications: AWS Certified Data Engineer, AWS Solutions Architect, or equivalent cloud certifications; experience with enterprise data governance and metadata management platforms; experience with real-time analytics, event-driven architectures, and streaming data platforms; knowledge of modern data architecture patterns including Data Mesh, Lakehouse, and domain-oriented design; experience leading large-scale cloud transformation or enterprise data modernization programs
• Indicative technologies: AWS, Amazon S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions, AWS Bedrock, Lake Formation, Amazon Redshift, Confluent Kafka, Kafka, SQL, PySpark, Terraform, GitHub, GitHub Actions, AWS IAM, Apache Airflow, Python, and related data governance, observability, and AI/data platform capabilities