Cloud Streaming

Cloud Streaming Engineer Jobs

2 jobs found

Today

Streaming Data Engineer

Bright Vision Technologies Remote 🇺🇸 Remote

Remote

Today

$78k - $105k/yr Full time Big Data Cloud Streaming Data

Apply Now

Job Description

Bright Vision Technologies is hiring a Streaming Data Engineer for a 100% remote role based in the U.S. In this position, you will architect, deploy, and operate large-scale Apache Kafka and Confluent Platform environments that support mission-critical, event-driven workloads.

You will own the Kafka platform end-to-end, from capacity planning and configuration through security, automation, observability, and developer enablement, partnering with engineering teams to turn ambiguous requirements into well-engineered streaming solutions.

Role Overview

Architect, deploy, and operate large-scale Apache Kafka and Confluent platform environments for mission-critical event-driven workloads.
Own the Kafka platform end-to-end, including cluster sizing, configuration, security, automation, observability, and developer enablement.
Partner with application teams to deliver a reliable, performant, and developer-friendly streaming platform.
Work closely with cross-functional partners including product, design, engineering, operations, and business stakeholders to translate ambiguous requirements into solutions.
Raise the bar through code review, design review, and mentorship of more junior engineers.

Requirements

Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
5+ years of experience operating Apache Kafka or Confluent Platform in production.
Deep, hands-on knowledge of Kafka internals: partitions, replication, ISRs, and consumer groups.
Strong experience with Kafka security: SASL, mTLS, ACLs, and RBAC.
Hands-on experience with Kafka Connect, Schema Registry, and either Kafka Streams or ksqlDB.
Experience implementing HA/DR strategies for Kafka.
Strong scripting skills in Python, Bash, or Go.
Hands-on experience with infrastructure-as-code using Terraform and Ansible.
Working knowledge of observability tooling for Kafka.
Excellent troubleshooting, communication, and documentation skills.

Technologies

Apache Kafka, Confluent Platform
Kafka Connect, Schema Registry
Kafka Streams, ksqlDB
SASL, mTLS, ACLs, RBAC
Python, Bash, Go
Terraform, Ansible
Kafka (observability tooling)

Preferred Qualifications

Confluent Certified Administrator or Developer credentials.
Experience operating Kafka on Kubernetes using Strimzi or Confluent Operator.
Exposure to managed Kafka services such as AWS MSK or Azure Event Hubs Kafka API.
Familiarity with stream processing frameworks including Flink or Spark Streaming.
Experience with data governance and lineage for streaming data.

Compensation and Location

Location: Remote (remote)
Salary: USD 78,000 - 105,000 per year

Visa Sponsorship

U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates are encouraged to apply.
We are unable to sponsor new H-1B visa petitions for this position.

How to Apply

For immediate consideration, send your resume to hilda@bvteck.com.

View full job page →

21d ago

Lead Data Engineer – Physical AI Platform

Caterpillar Irving, TX

Irving, TX

21d ago

$128k - $209k/yr Full time AWS Cloud Cloud Streaming

Apply Now

Job Description

Caterpillar is hiring a Lead Data Engineer to design, build, and maintain scalable data pipelines, microservices, and cloud-based data platforms that support reliable, high-quality data for business and engineering teams. The role emphasizes cloud-native ingestion and streaming, data integration frameworks, operational monitoring, and data quality validation within an agile delivery environment.

Key Responsibilities

Collaborate with Principal Software Engineers and Data Architects to define solution architecture.
Lead solution design and optimization of scalable data pipelines and microservices in Python, supporting both real-time and batch data processing across enterprise platforms.
Develop cloud-native data ingestion and streaming solutions using AWS services including Kinesis, S3, DynamoDB, EventBridge, and related technologies.
Own the design, implementation, and operational excellence of data integration frameworks and source data pipelines supporting CI Autonomy initiatives.
Work with business, product, and engineering stakeholders to convert complex requirements into scalable data architectures, workflows, mappings, and system designs.
Establish and enforce automated testing, data quality controls, and validation frameworks to support integrity, reliability, and compliance across distributed data ecosystems.
Lead operational monitoring, performance tuning, and root-cause analysis for production data platforms using observability tools such as CloudWatch to maintain high availability and service reliability.

Required Qualifications

Bachelor’s degree in Computer Science, Computer Engineering, or a related field.
8+ years of experience in data engineering or related disciplines with increasing responsibility.
Extensive experience working with modern, large-scale, complex data platforms.
Strong foundation developing and deploying Python solutions in production environments.
Experience leading teams to build high-throughput, scalable data pipelines.
Hands-on experience with AWS data services including Kinesis, S3, DynamoDB, EventBridge, etc., at scale.
Strong SQL skills, including data quality and validation practices.
Experience deploying software using CI/CD tools such as Azure DevOps, Jira, Jenkins, etc.
Experience developing microservices that support real-time data ingestion.
Experience developing software applications using both relational and NoSQL databases.
Ability to ensure data integrity across distributed and streaming systems.
Experience with monitoring, testing, and automation in large-scale data environments.

Technologies

Python, Java
AWS: Kinesis, S3, DynamoDB, EventBridge, CloudWatch
CI Autonomy
CI/CD and tooling: Azure DevOps, Jira, Jenkins
SQL, APIs, microservices
Real-time and batch data processing

Location and Work Arrangement

Full-time position working onsite at the Irving, TX office (Dallas).
Domestic relocation assistance is available.
Visa sponsorship is available.

Compensation

Summary pay range: USD 128,470 - 208,770 per year.

Benefits

Medical, dental, and vision benefits.
Paid time off plan (Vacation, Holidays, Volunteer, etc.).
401(k) savings plans.
Health Savings Account (HSA).
Flexible Spending Accounts (FSAs).
Health Lifestyle Programs.
Employee Assistance Program.
Voluntary Benefits and Employee Discounts.
Career Development.
Incentive bonus.
Disability benefits.
Life Insurance.
Parental leave.
Adoption benefits.
Tuition Reimbursement.

Additional Posting Details

Any offer of employment is conditioned upon successful completion of a drug screen.

View full job page →

Cloud Streaming Engineer Jobs

Streaming Data Engineer

Lead Data Engineer – Physical AI Platform

Get Cloud Streaming Engineer Jobs alerts