Senior Cloud Data Engineer

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, Python, and SQL.
• Build and optimize ingestion frameworks for ERP, MES, Industrial IoT, quality, supply chain, inventory, finance, and other operational systems.
• Create reusable engineering patterns and standards to improve delivery speed, supportability, and reliability.
• Implement lakehouse solutions using Delta Lake and Medallion Architecture across Bronze, Silver, and Gold layers.
• Design and optimize data models for reporting, analytics, AI, and machine learning use cases.
• Maintain data lineage, metadata, technical documentation, and operational runbooks.
• Integrate Azure Databricks with Azure Data Factory, Azure Data Lake Storage Gen2, Azure Key Vault, Microsoft Fabric, Power BI, and Azure DevOps.
• Support orchestration and data movement between enterprise applications and cloud platforms.
• Monitor and optimize Databricks clusters, jobs, notebooks, and workflows for performance, scalability, and cost efficiency.
• Troubleshoot production issues, identify root causes, and implement sustainable corrective actions.
• Contribute to monitoring, alerting, testing, and operational support practices.
• Apply organizational standards for data quality, governance, privacy, security, and compliance.
• Implement role-based access controls and governed data access using Unity Catalog and Azure security services.
• Build automated validation, auditing, and data quality controls into engineering workflows.
• Partner with business users, analysts, architects, and IT teams to translate requirements into maintainable technical solutions.
• Participate in Agile ceremonies, sprint planning, code reviews, and technical design reviews.
• Mentor junior engineers and promote best practices for coding, testing, documentation, and deployment.

Requirements

• Bachelor's degree in computer science, information systems, data engineering, or a related field; an equivalent combination of education and experience may be considered.
• 3–5 years of experience in data engineering, cloud engineering, or analytics platform development.
• Minimum 2 years hands-on experience developing solutions with Azure Databricks.
• Experience supporting enterprise data solutions in a manufacturing or industrial environment.
• Strong knowledge of Azure Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, lakehouse architecture, data modeling, and data warehousing concepts.
• Experience with Azure Data Lake Storage Gen2, Azure Data Factory, Azure DevOps, Git source control, and CI/CD practices for data platforms.
• Experience with data quality, security, monitoring, troubleshooting, and performance tuning.
• Knowledge of manufacturing data domains such as ERP, MES, production planning, inventory, supply chain, quality management, maintenance, or Industrial IoT.
• Ability to work with operational stakeholders and translate manufacturing processes into reliable data requirements and solutions.

Benefits

• Excellent health and welfare benefits including medical, dental, vision, disability, and a variety of voluntary benefit options.
• 401(k) benefits with annual company match for eligible employees.
• Professional and personal development programs including Career and Learning Paths with opportunities for advancement.

Additional Skills and Certifications

• Experience with Microsoft Fabric, Power BI, or real-time and streaming data workloads.
• Hands-on experience with Unity Catalog and governed lakehouse implementations.
• Exposure to machine learning or AI data preparation and deployment patterns.
• Experience with Infrastructure as Code using Terraform, Bicep, or ARM templates.
• Preferred knowledge of Infor M3, including application design, business data model, and core table structures across manufacturing, supply chain, inventory, finance, and related functional areas.
• Preferred understanding of production, planning, inventory, quality, maintenance, and supply chain processes.
• Microsoft Certified: Azure Data Engineer Associate.
• Databricks Certified Data Engineer Associate or Professional.
• Microsoft Certified: Fabric Analytics Engineer Associate.