Data engineer
Apache Airflow
Automation
Big Data
Bigdata
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Data Warehouse
Cloud Data Warehouse
Cloud Operations
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Integration Tools
Data Lakehouse
Data Management
Data Pipeline
Data Platform
Data Processing
Data Security
Data Warehouse
Data Warehousing
Database
Databases
Databricks
ETL
Informatica
Integration
Pyspark
Snowflake
Spark
SQL
Unity Catalog
Workflow Orchestration
Job Description
Data engineer role focused on ETL/ELT reliability and acquisition and marketing data workflows using Databricks and related tooling.
- ETL/ELT pipelines (acquisition reporting): Assist in building and maintaining pipelines in Databricks using PySpark, SQL, and Unity Catalog, working under the guidance of senior engineers.
- Snowflake to Databricks migration: Support migration of existing Snowflake SQL scripts and transformations into Databricks Unity Catalog by updating queries, validating outputs, and helping apply governance best practices.
- Marketing vendor data ingestion: Contribute to ingestion processes for marketing data, including parsing, normalization, and basic data quality validations.
- Data quality and monitoring: Implement and maintain foundational data quality checks, monitoring alerts, and issue triage workflows using Databricks, Snowflake, Airflow, and internal tooling.
- Data Reliability Engineering support: Partner with the Data Reliability Engineering team for SLA monitoring, simple incident troubleshooting, and logging improvements.
- Stakeholder collaboration: Work with analytics and marketing partners to understand dataset needs and ensure dashboard and reporting accuracy.
- Reliability and optimization: Support performance tuning, logging improvements, and general pipeline reliability efforts.
- Engineering practices: Participate in code reviews, documentation, and contribute to shared frameworks and tools.
Requirements
- Experience: 4+ years as a Data Engineer or similar role.
- SQL: Strong proficiency including analytical SQL, complex joins, and window functions.
- Spark: Hands-on experience with PySpark and/or Spark SQL in production.
- ETL/ELT and modeling: Understanding of data modeling, ETL/ELT design patterns, and distributed data processing.
- Databricks ecosystem: Experience building pipelines in Databricks, including Delta Lake, Unity Catalog, data governance, and Lakehouse patterns.
- AWS: Experience with S3, IAM, EC2, Glue, Lambda, or related services.
- Orchestration: Experience with Airflow or similar orchestration tools.
- DevOps workflow: Experience with Git/GitHub, CI/CD, and modern DevOps practices.
- Communication: Excellent communication skills and ability to work with cross-functional partners.
- Ingestion and semi-structured data: Experience building robust ingestion pipelines and working with JSON, Parquet, and CSV.
Technologies
- Databricks
- PySpark
- SQL
- Unity Catalog (Databricks UC)
- Snowflake
- Airflow
- AWS, including S3, IAM, EC2, Glue, Lambda
- Git, GitHub
- CI/CD
- Delta Lake
- Lakehouse patterns
- JSON, Parquet, CSV
- Spark SQL
Location
- California City, CA (onsite)
How to apply
- Interested candidates: forward CVs to [email protected]
- Contact: Deivy at 856 528 3312 extn 1221