Senior Data Engineer
Job Description
National Grid is seeking a Senior Data Engineer to design, build, and modernise a forecasting platform. The role focuses on scalable, end-to-end data pipelines that unify large datasets from multiple sources, with an emphasis on data quality, governance, and engineering capability uplift. This position follows a hybrid working model with office options in Waltham, MA, Brooklyn, NY, or Hicksville, NY.
Role Overview
In this Senior Data Engineer role within Information Technology and Digital, you will help teams access high-quality data and support better, data-driven decision making. You will work in a cross-functional development team, applying agile working practices and strong engineering standards to deliver reliable and accessible data pipelines for forecasting, analytics, and reporting. You will also contribute to evolving the organisation’s data engineering architecture, tools, and ways of working to improve scalability, performance, and delivery speed.
Responsibilities
- Design, build, and support scalable data pipelines, ensuring robustness, testing coverage, and readiness for growing business needs.
- Build and evolve data integrations and APIs to manage increasing data volumes and complexity.
- Champion data quality and governance to ensure data is secure, consistent, reliable, and accessible across the organisation.
- Develop scalable data solutions aligned with architectural roadmaps and standards across integration, storage, reporting, and analytics.
- Partner with analytics and business teams to improve data models, increase accessibility, and enable stronger decision making.
- Design and enhance data integration and quality frameworks, supported by strong unit, integration, and functional testing.
- Build and automate distributed data solutions to collect and process streaming events from multiple sources.
- Investigate and resolve complex data issues using data analysis to identify root causes and implement effective fixes.
- Guide and mentor junior engineers, sharing best practices across coding, engineering standards, and optimisation.
Required Qualifications
- Strong data engineering experience in data engineering, analytics, or business intelligence, ideally around 5 years.
- Strong programming skills, particularly Python, with hands-on experience in PySpark and SQL.
- Strong experience working with Databricks and building scalable data solutions.
- Experience designing and developing integrations across multiple systems and APIs.
- Hands-on experience with Azure data technologies, including Azure Data Lake, Azure Data Factory, Azure DevOps, and Azure Functions.
- Strong SQL expertise, including working with and optimising queries over large, complex datasets.
- Experience with data warehousing and building ETL and/or ELT pipelines.
- Understanding of Agile software development methodologies and modern engineering practices.
- Excellent problem-solving and troubleshooting with a structured, process-driven approach.
- Strong documentation skills focused on maintainable, well-engineered solutions.
- Degree or equivalent experience, ideally in Computer Science, Mathematics, Statistics, or another technical discipline.
Technologies
- Python
- PySpark
- SQL
- Databricks
- Azure Data Lake
- Azure Data Factory
- Azure DevOps
- Azure Functions
- ETL
- ELT
Preferred (It Would Be Great If You Also Have)
- Experience designing data schemas and working with SQL and NoSQL databases.
- Knowledge or experience with Kafka, Flink, Fivetran, or Matillion.
- Exposure to Data Science and Machine Learning.
- A background in software engineering.
- Experience working with Snowflake.
Location and Team
- Team: Information Technology and Digital; IT and Digital
- Primary location: Hicksville, NY (hybrid)
- Other office locations: Waltham, MA; Brooklyn, NY
City addresses listed include: Hicksville (11801), Waltham (02451), and Brooklyn (11217).
Compensation
- Hicksville/Brooklyn: USD 137,000 - 180,000 per year
- Waltham: USD 129,000 - 170,000 per year
The posting also lists a general annual range of USD 129,000 - 180,000.
Notices
- National Grid will not be providing visa sponsorship for this position now or in the future. You must be able to work without current or future visa sponsorship.
- National Grid uses an assessment that evaluates job qualifications/characteristics using AI or statistically based scoring.
- A career path is available with advancement opportunities within and across bands as you develop and evolve in the role.
- Candidates will be assessed and provided offers against minimum qualifications and individual experience.
- National Grid is committed to equal employment opportunities and does not discriminate based on protected characteristics under applicable law.