Data Integration Platform Engineer Jobs

1 job found

Today

Senior GCP Data Engineer

Persistent Charlotte, NC

Charlotte, NC

Today

Contract Senior Azure Azure Openai

Apply Now

Job Description

Persistent is seeking a Senior GCP Data Engineer (Contract) based in Charlotte, NC. This role is focused on building and optimizing scalable GCP-based data pipelines and data virtualization and federated query solutions, with mandatory Starburst and/or Dremio experience.

Key Responsibilities

Design, develop, and deploy AI agents using Python, LangGraph, Microsoft Agent Framework, and Azure OpenAI.
Build and optimize Survey Programming Agents, including Questionnaire Parsing, Routing Logic, JSON Generation, Dimensions Setup, STG Generation, Translation Workflows, DMS Script Creation, Link Testing, Survey Quality Control, and Change Impact Analysis.
Design, develop, and deploy scalable ETL/ELT pipelines using Python and GCP-native services.
Build and maintain enterprise-grade data integration solutions across multiple data sources and platforms.
Integrate MongoDB data into Google Cloud Storage and BigQuery environments.
Design and optimize data ingestion, transformation, and analytical processing workflows.
Develop and support data virtualization and federated query solutions using Starburst and/or Dremio.
Optimize BigQuery workloads using partitioning, clustering, query tuning, and performance engineering.
Monitor and troubleshoot production data pipelines and resolve data quality issues.
Perform debugging, root cause analysis, and issue resolution for data processing failures.
Optimize Dataflow jobs and MongoDB indexing strategies to improve performance and reduce operational costs.
Implement monitoring, alerting, and operational support processes for data platforms.
Collaborate with Data Scientists, Software Engineers, Architects, and business stakeholders to deliver reliable data solutions.
Develop and maintain technical documentation, operational runbooks, and deployment procedures.
Implement CI/CD processes for data engineering solutions and cloud deployments.
Ensure adherence to data governance, security, compliance, and operational best practices.

Required Qualifications

7-12 years of experience in Data Engineering, Cloud Data Platforms, or Big Data environments.
Strong hands-on experience with Google Cloud Platform (GCP).
Expertise in BigQuery, Cloud Dataflow, Cloud Composer (Apache Airflow), and Cloud Storage.
Mandatory hands-on experience with Starburst and/or Dremio.
Strong experience implementing ETL/ELT pipelines and enterprise data integration solutions.
Advanced Python programming and scripting expertise.
Strong SQL development skills and query optimization experience.
Experience working with MongoDB and NoSQL database technologies.
Solid understanding of data warehousing, dimensional modeling, and analytical data platforms.
Hands-on experience monitoring and supporting production-grade data environments.
Experience with Cloud Logging and Cloud Monitoring, including incident troubleshooting.
Knowledge of CI/CD implementation and DevOps practices for data engineering deployments.
Strong analytical, troubleshooting, and problem-solving skills.
Experience working in Agile and cross-functional teams.
Excellent communication and stakeholder management abilities.

Technology Stack

Google Cloud Platform (GCP)
BigQuery
Dataflow
Cloud Composer (Apache Airflow)
Cloud Storage
Starburst
Dremio
Python
LangGraph
Microsoft Agent Framework
Azure OpenAI
MongoDB
SQL
Cloud Logging
Cloud Monitoring

Location and Employment Type

Charlotte, NC (onsite)

Contract

Experience Level: 7-12 years

Benefits

Competitive salary and benefits package
Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
Opportunity to work with cutting-edge technologies
Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
Annual health check-ups
Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment

Hybrid work support and flexible hours to fit diverse lifestyles
Accessibility-friendly office setup, including ergonomic setups and assistive technologies to support employees with physical disabilities
If you are a person with disabilities and have specific requirements, please inform the team during the application process or at any time during employment

View full job page →

Get Data Integration Platform Engineer Jobs alerts

New Data Integration Platform Engineer Jobs, daily.