EngineerJobs.io
← Back to all jobs

Job Description

Robotics Technologies is seeking a Senior Data Engineer in Boston, MA (onsite) to support Colorado’s Statewide Longitudinal Data System (SLDS/COdata). The role focuses on building and operating scalable ETL/ELT pipelines into a centralized Snowflake platform, with emphasis on data quality, governance, identity matching, and automation across development and production environments.

Role Focus

Design, develop, optimize, and run reliable data ingestion and transformation workflows that standardize structured data into longitudinal models. The engineer will deliver production-ready Informatica ETL workflows and Snowflake transformation pipelines, while supporting operational continuity during ongoing feed cycles and transition activities.

Responsibilities

  • Design, develop, test, and deploy automated ETL/ELT pipelines using Snowflake and Informatica.
  • Integrate data from state agencies, educational institutions, and third-party source systems.
  • Develop complex SQL queries, transformations, stored procedures, and data processing workflows.
  • Build and maintain source-to-target mappings and data integration processes.
  • Ingest and transform structured data into standardized longitudinal data models.
  • Develop Informatica workflows for automated data ingestion, cleansing, and transformation.
  • Maintain Snowflake tables, views, schemas, and transformation objects.
  • Optimize Snowflake queries, warehouse utilization, and pipeline performance.
  • Monitor data processing workloads, resolve performance bottlenecks, and implement scalable, cost-efficient processing solutions.
  • Maintain RAW, transformation, and analytics data layers.
  • Implement automated data quality checks, including structural and format validation; completeness and duplicate detection; referential integrity; and business logic validation.
  • Perform data cleansing, reconciliation, and exception handling, including troubleshooting failed jobs, pipeline errors, and ingestion issues.
  • Implement record linkage and identity-matching processes across agency datasets.
  • Maintain data lineage, metadata, audit trails, and data governance standards.
  • Ensure compliance with FERPA, COPPA, ADA, and applicable state data privacy regulations.
  • Implement data suppression and de-identification requirements.
  • Monitor scheduled pipelines to ensure successful execution of recurring feed cycles; manage late, corrected, or malformed source files and support data reloads.
  • Reconcile source and target record counts and investigate discrepancies.
  • Maintain operational runbooks, incident logs, and pipeline documentation, and transition production pipeline responsibilities from a third-party implementation partner.
  • Develop and maintain CI/CD pipelines using Azure DevOps and use Git for branching, pull requests, and code reviews.
  • Implement Infrastructure as Code using Terraform, and manage code promotion across Development, Test, UAT, and Production environments.
  • Develop and execute manual and automated test cases using Azure Test Plans, and participate in peer reviews, defect resolution, and release documentation.
  • Collaborate with Data Architects, Data Scientists, Visualization Designers, and Operations Engineers to support data models, Tableau dashboard datasets, and Snowflake warehouse management and permissions.
  • Participate in knowledge-transfer sessions with third-party implementation teams and gradually assume independent ownership of pipelines and production operations.

Technologies

  • Snowflake
  • Informatica
  • Informatica Cloud Data Governance and Catalog (CDGC)
  • Terraform
  • Azure DevOps
  • Git
  • Azure Test Plans
  • SQL
  • Tableau

Requirements

  • Minimum 5 years of experience developing, managing, and optimizing ETL/ELT pipelines and data transformations within Snowflake.
  • Minimum 1 year of hands-on experience with Informatica Cloud Data Governance and Catalog (CDGC), including metadata management, data profiling, cataloging, and data quality validation.
  • Minimum 1 year of experience with Infrastructure as Code (IaC), including provisioning and managing cloud infrastructure.
  • Strong experience developing and maintaining automated CI/CD pipelines for build, testing, and deployment.
  • Strong proficiency with version control practices, branching strategies, pull requests, merge management, and code reviews.
  • Strong SQL development and data transformation skills.
  • Experience designing and implementing complex data ingestion and integration pipelines.
  • Experience with data quality, data validation, cleansing, and reconciliation processes.
  • Ability to troubleshoot production data pipelines and manage operational support activities.

Preferred Qualifications

  • Experience working with state government, public-sector, or education data systems.
  • Familiarity with Statewide Longitudinal Data Systems (SLDS) or longitudinal student and workforce data.
  • Experience integrating data from multiple government agencies and external partner systems.
  • Experience with record linkage, entity resolution, and deterministic or probabilistic identity matching.
  • Knowledge of FERPA, COPPA, student data privacy, and data suppression standards.
  • Experience with Informatica ETL workflows and metadata-driven data integration.
  • Experience with Azure Test Plans and structured software testing processes.
  • Familiarity with Tableau data integration and analytics-layer development.
  • Experience working with education and workforce datasets from organizations such as CDE, CDHE, CDLE, CDEC, and National Student Clearinghouse.

Project-Specific Responsibilities

  • Develop and maintain data ingestion pipelines for Colorado Department of Higher Education (CDHE) course-level data.
  • Support conditional ingestion of postsecondary demographic, Pell Grant eligibility, and disability-related data.
  • Integrate PSAT/SAT assessment data and perform student identity matching.
  • Support existing data feeds, including NSC, K-3, CDEC, and Phase 1 integrations.
  • Participate in knowledge transfer and operational transition from January through April 2027.
  • Support analytics-layer stabilization, data reliability improvements, and performance tuning.
  • Develop de-identified and privacy-compliant data extracts for participating higher education institutions.

Expected Deliverables

  • Production-ready Informatica ETL workflows and Snowflake transformation pipelines.
  • Automated data cleansing and validation modules.
  • Identity matching and record-linkage routines.
  • Snowflake analytics-layer tables, views, and data models.
  • Data mapping, lineage, and dependency documentation.
  • Operational runbooks and incident management procedures.
  • Azure Test Plans, test execution records, and defect logs.
  • Release change logs and production feed reconciliation reports.

Compensation and Location

Location: Boston, MA (onsite). Salary: USD 76 - 76 per hourly. Minimum experience: 5 years.

Similar Jobs