EngineerJobs.io
← Back to all jobs

Job Description

Vylor is hiring a full-time Engineer II (Data Engineer II) to build scalable lakehouse and analytics data solutions for Seeds and Biotech domains and an enterprise data platform in Johnston, IA.

Responsibilities

  • Design, develop, and maintain Lakehouse data pipelines, APIs, and data products for seed models, biotech research assets, enterprise applications, dashboards, and analytics
  • Build scalable ETL/ELT pipelines using Python, SQL, dbt, Trino, and shell scripting to process and integrate large-scale agricultural data from enterprise systems, lab platforms, geographic datasets, drones, satellites, and IoT devices
  • Create and maintain Bronze, Silver, and Gold layer data assets by integrating sources including Exasol, ClickHouse, MSSQL Server, and other enterprise systems
  • Apply transformation, validation, reconciliation, and quality-control processes to support analytics and downstream usage
  • Design Lakehouse datasets and transformation pipelines for seed, biotech, pathology, and disease analytics, including integration of laboratory, geographic, and historical research data
  • Develop and optimize incremental and full-refresh dbt models and complex Trino SQL queries to improve performance, scalability, and maintainability
  • Modernize legacy data applications by documenting business rules and source-to-target mappings, migrating ETL logic from legacy and Exasol-based environments to Trino and Lakehouse architectures, and deploying curated datasets to ClickHouse and Gold layer data products
  • Develop and enhance PRISM and FRIP data integrations using Trino connectors and end-to-end comparison pipelines with reusable validation and reconciliation logic across PRISM, FRIP, Lakehouse, Exasol, and downstream application datasets
  • Build reusable frameworks for logging, error handling, scheduling, automated testing, monitoring, deployment, and data-quality validation
  • Troubleshoot application and pipeline integration issues and create analytics-ready datasets for statistical analysis, predictive modeling, regression, classification, clustering, dashboards, and business-intelligence reporting

Requirements

  • Master’s degree (or equivalent) in Computer Science, Statistics, or a related field plus 2 years related experience
  • 24 months experience designing and developing scalable ETL/ELT pipelines with Python, SQL, and shell scripting to extract, transform, validate, reconcile, and integrate large-volume structured, semi-structured, and unstructured data for enterprise R&D, analytical models, dashboards, and reporting systems
  • 24 months experience developing ETL design mappings, source-to-target transformation rules, data reconciliation processes, data-quality controls, and complex SQL transformations to ensure accuracy and consistency across source and target platforms
  • 24 months experience developing and optimizing complex Trino SQL queries and integrations among Lakehouse, Exasol, ClickHouse, MSSQL Server, and other databases to deliver curated, analytics-ready datasets for R&D models and enterprise applications (including phenomic pillar), dashboards, and reporting
  • 24 months experience creating reusable Python- and Linux shell-based pipeline reliability and observability frameworks, including job scheduling, structured logging, exception handling, automated testing, monitoring, data-quality validation, cross-platform reconciliation, and application integration, plus using version control, containerization, and automated-deployment tools to deliver production data products
  • 24 months experience analyzing and modernizing legacy data applications by documenting business rules and data mappings, migrating ETL logic to cloud-based or Lakehouse architecture, deploying curated data products, and using Git and Docker for source control, automated deployment, and containerization

Technologies

  • Lakehouse
  • Python
  • SQL
  • dbt
  • Trino
  • Shell scripting
  • Exasol
  • ClickHouse
  • MSSQL Server
  • Kubernetes
  • Docker
  • Git
  • Linux

Benefits

  • Health benefits for you and your family on your first day of employment
  • Four weeks of paid time off and two weeks of well-being pay per year, plus paid holidays
  • Excellent parental leave with a minimum of 16 weeks for mother and father
  • Competitive retirement savings plan and tuition reimbursement program
  • Numerous development opportunities to build skills
  • Opportunity to contribute to making the world a better place
  • Total rewards package information: Vylor Benefits

Telecommute Benefit

  • Position includes a telecommute benefit within commuting distance to Johnston, IA Pioneer Hi-Bred/Vylor office, as directed

Location: Des Moines, IA (onsite)

Minimum Experience: 2 years

Similar Jobs