Google Pub/sub Engineer Jobs

1 job found

Today

GCP BigQuery Data Engineer

Virtues Irving, TX

Irving, TX

Today

$100k - $110k/yr Full time Analytics Big Data Bigdata

Apply Now

Job Description

Build cloud-native, scalable analytics and near-real-time ingestion pipelines across GCP BigQuery and event-driven streaming systems.

Responsibilities

Design, develop, and optimize enterprise data warehouse solutions using Google BigQuery.
Design, develop, and maintain real-time data ingestion pipelines using Apache NiFi to capture, transform, and route streaming data into Apache Kafka topics.
Build event-driven streaming architectures across Apache NiFi, Apache Kafka, and Google BigQuery for high-throughput, fault-tolerant, low-latency processing.
Configure and optimize Apache NiFi processors for extraction, transformation, routing, filtering, schema validation, error handling, retries, and reliable delivery to Kafka.
Develop streaming ingestion solutions enabling Google BigQuery to consume Kafka event streams and transform data into analytical tables and enterprise reporting datasets.
Design data models and implement ETL/ELT processes to move data from raw to curated and published layers.
Create and manage large-scale BigQuery datasets, including temporary/permanent and internal/external tables.
Optimize BigQuery workloads through query tuning, partitioning, clustering, and cost optimization techniques.
Monitor, troubleshoot, and optimize streaming workloads by tuning NiFi flows, Kafka topics/partitions, and BigQuery streaming ingestion to support high availability, data integrity, minimal latency, and cost-efficient processing.
Build scalable data lake frameworks and ingestion pipelines using GCP technologies.
Develop reporting and visualization solutions using Looker, Looker Studio (Data Studio), Connected Sheets, and other BigQuery reporting tools.
Collaborate with data architects, modelers, developers, DevOps engineers, project managers, and business stakeholders to deliver scalable enterprise analytics and continuous data platform improvements.

Requirements

7+ years of experience in Data Engineering, Data Warehousing, or Big Data platforms.
Must-have skills: Apache NiFi and Apache Kafka.
Strong hands-on experience with Google Cloud Platform (GCP), including BigQuery, Cloud Dataflow, Pub/Sub, and Google Cloud Storage (GCS).
Hands-on experience designing and supporting real-time streaming pipelines using Apache NiFi, Apache Kafka, and Google BigQuery.
Experience using BigQuery Console/Query Editor for data management, performance tuning, and SQL development.
Strong experience designing scalable data models, ETL/ELT pipelines, data lake architectures, and enterprise data warehouse solutions.
Experience with batch and streaming ingestion using GCP services.
Thorough understanding of BigQuery cost structure (storage, ingestion, and query costs) and experience implementing query optimization, partitioning, clustering, and other cost optimization techniques.
Experience managing large-scale datasets, including temporary/permanent and internal/external BigQuery tables.
Experience developing reporting and visualization solutions using Looker, Looker Studio (Data Studio), and Connected Sheets.
Excellent verbal and written communication skills, able to collaborate effectively with technical teams, business stakeholders, senior management, and executive leadership.
Experience working in Agile environments and collaborating with cross-functional teams to deliver enterprise data engineering solutions.

Technologies

Google BigQuery
Apache NiFi
Apache Kafka
Google Cloud Dataflow
Google Pub/Sub
SQL
ETL/ELT
Google Cloud Storage (GCS)
Looker
Looker Studio (Data Studio)
Connected Sheets
DevOps
CI/CD
Agile/Scrum
Python
Spark
Data Modeling
Data Warehousing
Data Lakes
Batch & Streaming Pipelines

Preferred Qualifications

Google Cloud certifications.
Experience delivering enterprise-scale analytics and cloud data solutions.

Pay

$100,000.00 - $110,000.00 per year

Work Location

Onsite in Irving, TX

View full job page →

Get Google Pub/sub Engineer Jobs alerts

New Google Pub/sub Engineer Jobs, daily.