Data Engineer

Robotics Technologies is hiring a Data Engineer in San Jose, CA (onsite) to build scalable pipelines and data processing solutions across multiple sources and cloud platforms.

Responsibilities

• Design, develop, and maintain scalable data pipelines for extract, transform, and load workflows across multiple sources.
• Build ETL/ELT workflows using Python, SQL, PySpark, and Azure Data Factory.
• Develop and optimize data processing solutions with Apache Spark and Databricks.
• Work with structured and unstructured data from relational databases, REST APIs, flat files, and cloud storage.
• Implement data integration solutions that support business intelligence and analytics needs.
• Write complex SQL for large datasets, including stored procedures, views, and functions.
• Create data validation and cleansing routines, including data transformation and data quality checks to improve accuracy.
• Store and manage large data volumes using Azure Data Lake Storage and related cloud data platforms.
• Develop and maintain data warehouses, data marts, and dimensional data models.
• Integrate data from enterprise systems using REST APIs and database connectors.
• Monitor pipeline health, diagnose failures, and resolve performance and processing issues.
• Implement incremental loading, partitioning, and optimization techniques to improve pipeline performance.
• Use Snowflake for loading, transformation, query optimization, and warehouse management.
• Apply data security, access controls, and governance practices to protect sensitive information.
• Use Git and Azure DevOps to manage source control plus CI/CD and deployment activities.
• Collaborate with data analysts, data scientists, and business stakeholders to gather and refine data requirements.
• Participate in Agile/Scrum ceremonies, technical discussions, code reviews, and production support.

Requirements

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Experience in data engineering, ETL development, and data pipeline implementation.
• Strong programming knowledge in Python, SQL, and PySpark.
• Experience with cloud platforms and data warehousing technologies.
• Understanding of data modeling, database concepts, and data integration.
• Strong analytical, problem-solving, debugging, and communication skills.

Skills & Technologies

• Programming: Python, SQL, PySpark
• Big Data: Apache Spark, Hadoop, Databricks
• ETL Tools: Azure Data Factory, Informatica, SSIS
• Cloud Platforms: Microsoft Azure, AWS
• Data Storage: Azure Data Lake, Amazon S3
• Data Warehousing: Snowflake, Azure Synapse Analytics, SQL Server
• Databases: PostgreSQL, MySQL, Oracle, SQL Server
• Data Modeling: Star Schema, Snowflake Schema, Dimensional Modeling
• DevOps: Git, Azure DevOps, CI/CD, Jenkins
• Reporting: Power BI
• Methodologies: Agile, Scrum, SDLC
• APIs & Connectors: REST APIs
• Other Tools/Platforms: Azure Synapse Analytics, Azure Data Lake Storage, Hadoop, Informatica, SSIS