Principal Engineer I - Senior Data Engineer

Software Resources, Inc. is seeking a Principal Engineer I - Senior Data Engineer to support the Bank Regulatory Reporting Program by designing, building, and implementing essential components of the enterprise data platform. This hybrid role is based in Phoenix, AZ with 4 days on site and Fridays remote, with options in Phoenix, AZ or Dallas, TX.

In this position, you will apply expert PySpark development skills while delivering cloud data engineering capabilities on Azure. You will also implement Azure DevOps CI/CD pipelines and use Test Driven Development practices to meet bank technical standards and built-in quality expectations.

Responsibilities

• Design, build, and implement critical components of the enterprise data platform aligned to the strategic data and analytics needs of the Bank Regulatory Reporting Program.
• Implement data engineering solutions across the full data lifecycle, including efficient technical hygiene, using Azure DevOps for implementation of the Bank Regulatory Reporting and LFI strategy and roadmap.
• Develop ETL and data pipeline capabilities covering how data is created, transformed, stored, archived, analyzed, and shared across the Bank and partner systems.
• Write dynamic, parameterized, and reusable PySpark code.
• Build shared frameworks, helper functions, and companion notebook structures.
• Perform performance tuning and optimization of Spark workloads.
• Implement Azure DevOps CI/CD pipelines for data solutions in adherence to Bank technical standards.
• Apply Test Driven Development methodology to data solutions designed, built, and implemented.
• Lead implementation efforts tied to outcomes, recommendations, and designs from Data Governance and Enterprise Architecture.
• Conduct data exploration and data profiling using tools such as SSAS.
• Actively participate in data architecture decision-making.
• Provide technical mentorship to team members and other data professionals.
• Collaborate with product owners and business stakeholders to understand business requirements and operational processes.
• Implement built-in quality and compliance-by-design in all data solutions.
• Implement methods to include unstructured data and big data.

Requirements

• MUST be an expert in PySpark.
• 8+ years of data engineering experience focused on ETL concepts and processes, enterprise data warehouse capabilities, database principles, and related tools; strong or expert experience with Azure Data Factory, Azure Synapse Analytics and/or Databricks is preferred.
• 5+ years designing, implementing, and supporting cloud data solutions; Azure Data Lake, Azure Data Factory, Azure Data Services, Azure Synapse, Azure Logic Apps, and Azure DevOps experience is strongly preferred.
• Bachelor’s degree in engineering or a related field.
• Expert level experience with at least one RDBMS and a query language such as T-SQL, PL/SQL, or Spark SQL.
• Ability to design and build reusable, dynamic PySpark components, including companion notebook frameworks, shared helper functions and libraries, and parameter-driven ingestion and transformation patterns.
• Support and influence migration from Azure Synapse to Microsoft Fabric, including modernization of patterns and frameworks.
• Expert level experience in conceptual, logical, and physical data design.
• Azure certifications such as Azure Fundamentals, Azure Fabric Data Engineer, Azure Data Scientist, and/or Azure DevOps Engineer.
• Experience with design tools for conceptual architecture diagrams and data flow diagrams such as Visio, Archimate, or Lucidchart.
• Excellent verbal and written communication skills.

Technologies

• PySpark, Azure DevOps, ETL
• Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Azure Data Services, Azure Logic Apps, Microsoft Fabric
• Databricks, SSAS
• T-SQL, PL/SQL, Spark SQL
• Visio, Archimate, Lucidchart
• SAS, Tableau, PowerBI

Benefits

• Medical, dental, and vision coverage
• A 401(k) with company match
• Short-term disability
• Life insurance with AD&D

What You’ll Need

• Experience in Agile, SAFe, and/or Scrum is preferred.
• Experience integrating with data quality, data catalog, and data lineage tools is preferred.
• Additional cloud data certifications with any major cloud provider (Azure, AWS, GCP) is preferred.
• Banking or financial services industry experience or other highly regulated industry experience is preferred.
• Regulatory Reporting, including Large Financial Institution requirements, is preferred.