Senior GCP Data Engineer

Persistent is seeking a Senior GCP Data Engineer (Contract) based in Charlotte, NC. This role is focused on building and optimizing scalable GCP-based data pipelines and data virtualization and federated query solutions, with mandatory Starburst and/or Dremio experience.

Key Responsibilities

• Design, develop, and deploy AI agents using Python, LangGraph, Microsoft Agent Framework, and Azure OpenAI.
• Build and optimize Survey Programming Agents, including Questionnaire Parsing, Routing Logic, JSON Generation, Dimensions Setup, STG Generation, Translation Workflows, DMS Script Creation, Link Testing, Survey Quality Control, and Change Impact Analysis.
• Design, develop, and deploy scalable ETL/ELT pipelines using Python and GCP-native services.
• Build and maintain enterprise-grade data integration solutions across multiple data sources and platforms.
• Integrate MongoDB data into Google Cloud Storage and BigQuery environments.
• Design and optimize data ingestion, transformation, and analytical processing workflows.
• Develop and support data virtualization and federated query solutions using Starburst and/or Dremio.
• Optimize BigQuery workloads using partitioning, clustering, query tuning, and performance engineering.
• Monitor and troubleshoot production data pipelines and resolve data quality issues.
• Perform debugging, root cause analysis, and issue resolution for data processing failures.
• Optimize Dataflow jobs and MongoDB indexing strategies to improve performance and reduce operational costs.
• Implement monitoring, alerting, and operational support processes for data platforms.
• Collaborate with Data Scientists, Software Engineers, Architects, and business stakeholders to deliver reliable data solutions.
• Develop and maintain technical documentation, operational runbooks, and deployment procedures.
• Implement CI/CD processes for data engineering solutions and cloud deployments.
• Ensure adherence to data governance, security, compliance, and operational best practices.

Required Qualifications

• 7-12 years of experience in Data Engineering, Cloud Data Platforms, or Big Data environments.
• Strong hands-on experience with Google Cloud Platform (GCP).
• Expertise in BigQuery, Cloud Dataflow, Cloud Composer (Apache Airflow), and Cloud Storage.
• Mandatory hands-on experience with Starburst and/or Dremio.
• Strong experience implementing ETL/ELT pipelines and enterprise data integration solutions.
• Advanced Python programming and scripting expertise.
• Strong SQL development skills and query optimization experience.
• Experience working with MongoDB and NoSQL database technologies.
• Solid understanding of data warehousing, dimensional modeling, and analytical data platforms.
• Hands-on experience monitoring and supporting production-grade data environments.
• Experience with Cloud Logging and Cloud Monitoring, including incident troubleshooting.
• Knowledge of CI/CD implementation and DevOps practices for data engineering deployments.
• Strong analytical, troubleshooting, and problem-solving skills.
• Experience working in Agile and cross-functional teams.
• Excellent communication and stakeholder management abilities.

Technology Stack

• Google Cloud Platform (GCP)
• BigQuery
• Dataflow
• Cloud Composer (Apache Airflow)
• Cloud Storage
• Starburst
• Dremio
• Python
• LangGraph
• Microsoft Agent Framework
• Azure OpenAI
• MongoDB
• SQL
• Cloud Logging
• Cloud Monitoring

Location and Employment Type

Charlotte, NC (onsite)

Contract

Experience Level: 7-12 years

Benefits

• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check-ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment

• Hybrid work support and flexible hours to fit diverse lifestyles
• Accessibility-friendly office setup, including ergonomic setups and assistive technologies to support employees with physical disabilities
• If you are a person with disabilities and have specific requirements, please inform the team during the application process or at any time during employment