Senior Data Engineer
Job Description
The Senior Data Engineer role will act as the internal owner of Rex Moore’s enterprise Data Lakehouse platform on Microsoft Fabric with OneLake. The position focuses on designing, building, operating, and continuously improving pipelines, models, and the semantic layer used for reporting, workflow automation, and AI capabilities.
Role Overview
Own the ongoing design and operations of the Microsoft Fabric Data Lakehouse, including OneLake storage, Lakehouse items, and Fabric workspaces across development, test, and production environments. Establish and govern enterprise ingestion and transformation patterns, maintain platform reliability and performance, and support downstream analytics and AI use cases through governed data products.
Key Responsibilities
- Own the ongoing design and operation of the Microsoft Fabric Data Lakehouse, including OneLake storage, Lakehouse items, and Fabric workspaces across development, test, and production environments.
- Build and maintain the medallion architecture (bronze, silver, gold) for landing raw data, refining it, and publishing it for enterprise consumption.
- Manage deployment automation and environment promotion using Fabric deployment pipelines and Azure DevOps source control.
- Monitor platform health, pipeline reliability, data freshness, and refresh performance against defined service levels.
- Maintain Fabric capacity performance and evaluate capacity sizing based on actual workload rather than default scale-up.
- Administer the on-premises data gateway and hybrid connectivity to on-premises source systems.
- Design and periodically test backup, recovery, and disaster recovery for platform data and configurations.
- Design, build, and operate ingestion and transformation pipelines using Fabric Data Factory, Dataflows, notebooks, and Spark.
- Integrate enterprise source systems using supported APIs, vendor connectors, database replication, or secure file transfer, favoring repeatable patterns over one-off integrations.
- Build and maintain integrations with OAuth-secured vendor APIs, including authentication, rate limits, pagination, and incremental extraction when change tracking is not reliable.
- Ingest and model semi-structured and document-derived data, including document metadata, OCR-extracted fields, drawing markups, and model coordination issues.
- Develop incremental load strategies, change data capture, and historical retention approaches tailored to each source system.
- Implement data quality validation and reconciliation across finance, job cost, and project data.
- Onboard new source systems as the Rex Moore application portfolio evolves.
- Develop and maintain dimensional models and conformed enterprise dimensions in the gold layer, including Company, Project, Period, Cost Code, Vendor, and Employee.
- Build and tune Direct Lake semantic models supporting financial reporting and project or job cost reporting.
- Implement and test row-level and object-level security in semantic models, including project-scoped access for project and field roles.
- Manage release and change control for shared models and conformed dimensions so updates are tested, communicated, and applied consistently.
- Establish standardized enterprise business logic, KPIs, and measures in the semantic layer so calculations are defined once and reused across Power BI reports.
- Partner with business SMEs to define reporting requirements, identify authoritative data sources, and resolve conflicts across systems.
- Support month-end close data readiness and reconciliation, monitor report usage and adoption, and work with business report champions to measure value.
- Support self-service analytics and Power BI Copilot natural-language Q&A readiness via model metadata and synonym preparation.
- Prepare governed, certified gold-layer data for Microsoft Foundry AI use cases in partnership with AI and application teams, including AI Search indexing and retrieval patterns.
- Implement and maintain data cataloging, lineage, business glossary, ownership, sensitivity labeling, data quality rules, and data loss prevention in Microsoft Purview.
- Design and enforce role-based access aligned with Microsoft Entra identity and least-privilege principles.
- Manage service identities, managed identities, and secure credential and secrets handling for integrations.
- Support data classification and controlled access to sensitive financial, project, and workforce information.
- Partner with business data owners and stewards to certify data products and maintain stewardship over time.
- Maintain current platform documentation, architecture diagrams, source-to-target mappings, data dictionary, and operational runbooks.
- Complete knowledge transfer from the implementation partner, validate delivered work against acceptance criteria, and take ownership of platform documentation, runbooks, and open items.
- Establish repeatable engineering standards, naming conventions, and development practices for future data work.
- Serve as the internal technical authority for the Data Lakehouse platform following consultant transition, and mentor future data team members while directing contractor and vendor work.
- Track and forecast recurring platform cost across Fabric capacity, storage, compute, and data movement.
- Identify opportunities to simplify, consolidate, or improve existing pipelines and models.
- Stay current on Microsoft Fabric and Azure data platform capabilities and evaluate applicability to the Rex Moore environment.
- Participate in training and self-development initiatives to improve technical and business acumen.
Minimum Requirements
- 7+ years of experience in data engineering, business intelligence engineering, or enterprise data platform roles.
- Production experience with Microsoft Fabric, Azure Synapse, Databricks, or a comparable cloud lakehouse platform, including layered or medallion architecture.
- Advanced SQL and hands-on experience with cloud data platforms and distributed processing.
- Dimensional modeling experience (Kimball or similar) with demonstrated ownership of a production data model.
- Power BI development experience, including DAX, semantic model design, and report performance tuning.
- Proficiency with Python and/or PySpark for data transformation.
- Experience integrating ERP data and building API-based ingestion against authenticated REST endpoints.
- Working knowledge of source control, CI/CD practices, and deployment automation for data workloads.
- Understanding of enterprise identity and access management, data classification, and secure handling of sensitive data.
- Bachelor’s degree in Information Technology, Computer Science, Data Engineering, or a related field, or equivalent work experience.
- Strong analytical and problem-solving skills, including the ability to trace a data issue from report back to source.
- Strong documentation discipline, including diagrams, runbooks, and technical specifications.
- Excellent verbal and written communication skills.
- Ability to translate business questions into data requirements and explain technical constraints in business terms.
- Ability to collaborate effectively with IT staff, business SMEs, executives, and external vendors.
- Ability to operate independently and own a production platform end to end.
- Strong attention to detail and commitment to data accuracy and reconciliation to source.
- Ability to prioritize competing demands and work in a fast-paced environment.
- Ability to establish standards and repeatable patterns rather than solving each problem uniquely.
- Interest in growing the Rex Moore data engineering capability over time.
- Ability to verbally communicate detailed and accurate instructions and information.
- Ability to hear with or without correction to receive and interpret detailed information.
- Required to sit at a desk work station for long periods of time.
- Push, pull, lift, carry at least 10 pounds and occasionally lift/move up to 35 pounds.
- Occasionally required to walk and stand for short and long periods of time.
- Required to have visual acuity with or without correction to read contract documents, analyze data, view a computer monitor, and proof-read documents.
- Required to use fingers and hands to type using a computer keyboard and handle, feel, and reach.
- May occasionally be required to stoop, kneel, and crouch.
Preferred Knowledge
Additional experience that may be beneficial:
- Direct Microsoft Fabric and OneLake production experience, including Direct Lake semantic models.
- Microsoft Purview or comparable enterprise data governance tooling.
- Experience as the first or primary data engineer in an organization establishing the function.
- Relevant Microsoft certifications (DP-600, DP-700, AZ-305, AI-102).
- Exposure to Microsoft Foundry, Azure AI Search, or retrieval-augmented generation (RAG) data patterns.
Preferred Knowledge (AEC / Construction Industry)
- Experience supporting data, reporting, or systems within construction, engineering, architecture, specialty trade contracting, or another AEC or project-based industry.
- Familiarity with construction financial and operational concepts including job cost, work breakdown structure (WBS), cost codes and cost types, committed cost, change orders, percent-complete revenue recognition, work-in-progress (WIP) reporting, and retention.
- Understanding of how estimating, project management, field operations, procurement, and accounting data relate across systems in a construction organization.
- Exposure to construction and AEC platforms such as Autodesk Construction Cloud (Build or Forma), Trimble Accubid, Bluebeam, Revizto, or comparable project delivery and estimating systems.
- Experience integrating business platforms such as Paycom, Expensify, or Hilti ON!Track, or comparable tools.
- Familiarity with construction workforce and field data including labor hours, crew and unit productivity, timekeeping, payroll integration, equipment and tool tracking, and prefabrication.
- Awareness of data considerations in document-heavy environments including drawings, submittals, RFIs, and contract documents, including metadata, retention, and governance implications.
- Understanding of reporting needs of project managers, field leadership, finance, and executives in a project-driven business.
Technologies
- Microsoft Fabric, OneLake, Azure DevOps
- Fabric Data Factory, Dataflows, notebooks, Spark
- OAuth, REST
- Direct Lake
- Power BI, DAX
- Microsoft Foundry, AI Search
- Microsoft Purview
- Microsoft Entra
- Kimball
- Python, PySpark
- Azure Synapse, Databricks
Compensation
USD 123,350 - 204,150 per yearly.
Benefits
- Health Benefits
- 401(k) with Company Match
- Life and Disability Benefits
- Paid Holidays & Paid time off
- Access to multiple types of training
- Opportunities for Self-development and Career Progression
- Wellness Benefits & Employee Assistance Program
- Family Leave (Maternity, Paternity)
- Tuition Reimbursement Program
Work Environment
- Hybrid position with a combination of in-office and remote work.
- Fully remote arrangements may be considered for candidates with demonstrated success working remotely and a proven record of driving initiatives independently.
- May be required to travel to other office locations.
Location and Reporting
Location: Sacramento, CA (hybrid)
Reporting structure: Reports to the Information Technology Director, Information Technology Department.