Lead Data Engineer

Kaizen Dynamics is seeking a Lead Data Engineer to support the OCTO Enterprise Data team and its Cloud Data Exchange. In this onsite Washington, DC contract role, you will design, build, and maintain enterprise-grade data pipelines using native Azure tooling to enable ETL, data loading, and transformation work.

You will collaborate across teams and agencies, help translate stakeholder needs into reliable data workflows, and contribute to governance-aligned practices that support accurate, consistent, secure, and dependable data.

Key Responsibilities

• Analyze, design, develop, implement, replicate, and support complex enterprise data projects.
• Interface with other agencies and consult with user departments on system requirements, environment constraints, and operating challenges in the current state.
• Advise on cloud-based solutions to resolve issues and develop and replicate future enhancements to District data systems.
• Apply strong ETL knowledge using frameworks including Azure Data Factory, Synapse, Databricks, and Informatica, gathering requirements or reviewing existing code to deliver enhancements and new development.
• Establish cloud and on-premises connectivity across systems such as ADLS, ADF, and Synapse, including implementations involving Databricks.
• Build end-to-end pipelines to load data from multiple source systems into a landing layer or SQL tables.
• Work with data integration and pipeline tools such as Informatica, Synapse, Apache NiFi, and Apache Airflow.
• Develop, standardize, and optimize data workflow and pipeline processes while adhering to best practices.
• Contribute to enterprise data governance by ensuring data accuracy, consistency, security, and reliability.
• Automate, monitor, manage, and alert on data pipelines and workflows.
• Analyze and evaluate system changes for feasibility, including alternative approaches and backup and rollback procedures.
• Support development of new systems, upgrades, and enhancements to existing systems, ensuring consistency with approved standards.
• Create complex programs and reports using database query languages.
• Handle sensitive data according to its level of sensitivity.
• Convey technical and functional concepts within the technical specialty, and support collaboration with application development activities when coordination is needed.
• Identify improvements to project standards to deliver high-quality services and products.

Requirements

• Bachelor’s degree in Information Technology or related field, or equivalent experience.
• SQL: 10 years (Required).
• Extract-Transform-Load (ETL) processes: 10 years (Required).

Technologies

• Azure Data Factory, Azure Synapse, Databricks
• Informatica
• ADLS, Azure Data Lake Storage Gen2, Blob Storage
• MSSQL Db, Azure SQL DB, SQL Server
• Python
• Apache NiFi, Apache Airflow
• Oracle, DB2, Quickbase
• CSV, XML, Parquet, AVRO

Location: Washington, DC (onsite)
Job Type: Contract (also listed as Full-time)
Compensation: USD 70 - 80 per hour
Minimum Experience: 10 years