Data Engineer, Microsoft Fabric and AI Director

GS1 Global Office is seeking a senior, hands-on Data Engineer Director (individual contributor) to design and operate secure, scalable enterprise data and analytics solutions. The role focuses on Microsoft Fabric and Azure data services, supports Power BI reporting, and delivers AI-enabled pipelines that connect approved enterprise data to large language model platforms such as Claude AI.

Location

Ewing, NJ (onsite)

Compensation

USD 140,000 - 160,000 per year

Experience & Education

• Minimum experience: 5 years
• Education: Bachelor’s degree in computer science, data engineering, information systems or a related field (or equivalent relevant professional experience)

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Microsoft Fabric and Azure data services.
• Build and support Lakehouse, Data Warehouse, and semantic model solutions using appropriate architecture and reusable design patterns.
• Develop reliable data integration using Microsoft Fabric Data Pipelines, Azure Data Factory, APIs, and other approved integration methods.
• Support migration and modernisation initiatives involving Microsoft Fabric and Azure analytics services.
• Optimise data processing performance, scalability, maintainability, and cost efficiency.
• Participate in enterprise data architecture decisions and help evolve shared data models and analytics standards.
• Design and build retrieval-augmented generation (RAG) pipelines that securely connect approved enterprise data to Claude AI and other approved LLM platforms.
• Create data preparation, chunking, embedding, indexing, and retrieval processes to support accurate and traceable AI-enabled search and analytics.
• Support responsible adoption of approved enterprise AI and development tools, including Claude AI and Claude Code, with secure data-handling and access patterns.
• Evaluate and prototype AI-enabled analytics use cases with senior leaders and technical stakeholders, considering business value, architecture, risk, and guardrails.
• Monitor and improve AI pipeline reliability, quality, performance, and cost in line with GS1 governance standards.
• Maintain documentation, traceability, and human oversight for AI-enabled solutions.
• Develop and maintain Power BI dashboards, reports, datasets, and semantic models to deliver actionable insights.
• Support operational reporting, data-quality reporting, and prioritised ad hoc analytics requests.
• Partner with business stakeholders to understand requirements, define acceptance criteria, and translate needs into sustainable reporting solutions.
• Implement data validation, observability, monitoring, and quality controls for pipelines and analytics solutions.
• Support metadata management, data lineage, documentation, retention, and governance requirements.
• Design solutions aligned to GS1 information security, data privacy, access-control, and responsible AI requirements, including role-based access controls, data classification, and auditability.
• Identify and escalate data-quality, security, privacy, model-risk, and governance concerns.
• Develop automated testing and validation for data pipelines, semantic models, reports, and AI-enabled solutions.
• Use Git, source control, CI/CD, and environment-management practices to support deployment across development, test, and production.
• Monitor critical systems, resolve production incidents, troubleshoot pipeline and reporting failures, and address RAG pipeline errors and performance bottlenecks.
• Conduct root-cause analysis and implement preventative improvements.
• Maintain technical documentation, operational runbooks, recovery procedures, and contribute to release validation and business-continuity activities.
• Provide technical leadership on data architecture, solution design, engineering standards, and responsible AI implementation.
• Review code and solution designs, share knowledge, and coach team members in data engineering and analytics practices.
• Collaborate with Product Owners, Data Engineers, QA, Software Engineering, and business stakeholders across a globally distributed organisation.
• Communicate options, dependencies, risks, and costs clearly to technical and non-technical audiences.
• Assess trade-offs and recommend approaches balancing business value, usability, security, scalability, cost, and maintainability.
• Contribute to planning, architecture discussions, continuous improvement, and workload priorities across the BIDA team.

Required Technologies & Skills

• Microsoft Fabric
• Azure data services
• Power BI (dashboards, reports, datasets, semantic models) and DAX
• Azure Data Factory and Microsoft Fabric Data Pipelines
• SQL (including advanced SQL) and Azure SQL
• Lakehouse, Data Warehouse, semantic models
• Git, CI/CD, and production monitoring
• Python and/or PySpark
• RAG (retrieval-augmented generation) and LLM API integration, including Claude AI
• Embeddings, vector search, vector databases, and prompt-based retrieval patterns
• Role-based access controls and privacy/security practices in cloud data and analytics environments

Preferred Experience

• Microsoft Fabric or related Microsoft data-platform certification
• Experience with Microsoft Purview, OneLake, Azure DevOps, or GitHub
• Experience with Power BI administration, tenant governance, or capacity management
• Experience with large-scale analytical datasets and cloud cost optimisation
• Knowledge of AI governance, responsible AI, model evaluation, and enterprise data-privacy practices
• Experience with Claude AI, Claude Code, or comparable enterprise AI coding and knowledge-work tools
• Experience working in a global, federated, or matrixed organisation

Travel & Collaboration

This role may require occasional international travel and regular collaboration across European and United States time zones.