Senior Data Engineer

Senior Data Engineer role at Robotics Technologies in Atlanta, GA (onsite), focused on building and operating production batch and streaming data pipelines and lakehouse integrations.

Responsibilities

• Build and run production-grade batch and streaming data pipelines across SQL Server, cloud applications, device and event streams, Customer Relationship Management systems, and third-party sources.
• Design ingestion, transformation, and enrichment workflows using Apache Flink, Kafka, Change Data Capture, SQL, and Python/Java/Scala.
• Implement lakehouse layers across landing, raw, conformed, and consumption zones using Apache Iceberg and Trino.
• Develop dimensional and analytical models for entities including buildings, tenants, credentials, devices, and people.
• Handle CDC challenges including late-arriving data, retries, backfills, replay, schema evolution, and data reconciliation.
• Build and maintain lakehouse to Snowflake integrations, including Iceberg-backed/external tables, secure sharing, and downstream data contracts.
• Automate data-quality checks, pipeline monitoring, lineage, freshness alerts, and operational dashboards.
• Troubleshoot and resolve production data issues, pipeline failures, performance bottlenecks, and data-quality problems.
• Optimize SQL and platform efficiency: storage layouts, partitioning, clustering, compute utilization, and overall platform cost.
• Implement security controls including role-based access, row/column-level security, masking, and protections for sensitive data.
• Deliver and support CI/CD with automated testing, infrastructure-as-code, and environment promotion processes.
• Participate in production support, incident resolution, Root Cause Analysis, deployments, and on-call activities.
• Collaborate with Product, Application Engineering, Quality Assurance, DevOps/Site Reliability Engineering, Analytics, and globally distributed engineering teams.
• Contribute to technical design decisions and documentation while staying hands-on with implementation and production delivery.

Requirements

• 6+ years of hands-on experience in data engineering, data warehousing, or data platform development.
• Strong production experience building and supporting end-to-end data pipelines.
• Expert SQL with complex query writing, query-plan analysis, and performance tuning.
• Strong development skills in Python, Java, or Scala.
• Hands-on experience with Apache Iceberg, Delta Lake, or Apache Hudi.
• Experience with distributed query technologies such as Trino, Presto, or Spark SQL.
• Hands-on Snowflake experience including data modeling, performance optimization, external/Iceberg tables, secure data sharing, or access management.
• Experience with streaming and Change Data Capture technologies such as Apache Flink, Kafka, Spark Structured Streaming, or Debezium.
• Strong understanding of dimensional modeling, data quality, schema evolution, and data contracts.
• Experience with Kubernetes, containers, CI/CD, and infrastructure-as-code.
• Experience implementing monitoring, alerting, lineage, reconciliation, and operational support for production data platforms.
• Ability to independently troubleshoot complex production issues and drive them through resolution.

Technologies

• SQL Server, Apache Flink, Kafka, Change Data Capture, SQL
• Python, Java, Scala
• Apache Iceberg, Trino, Snowflake, Apache Hudi, Delta Lake
• Presto, Spark SQL, Spark Structured Streaming, Debezium
• Kubernetes, Continuous Integration, Continuous Deployment, Infrastructure-as-code
• Row/column-level security

Preferred Qualifications

• Experience with multi-tenant Software as a Service data platforms and customer data isolation.
• Experience integrating open lakehouse platforms with Snowflake.
• Azure cloud experience.
• Experience with Airflow, Dagster, dbt, Power BI, Looker, or Superset.
• Experience with data governance, cataloging, masking, and access-control frameworks.
• Exposure to Internet of Things/device telemetry, physical security, commercial real estate, or property technology.
• Experience using Artificial Intelligence-assisted engineering tools to accelerate data development.