Senior Data Engineer
Senior
Analytics
API
APIs
Azure Data Lake
Big Data
Bigdata
Business Intelligence
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Integration
Data Lake
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Data Warehousing
Database
Databases
ETL
Hadoop
Hive
Informatica
Integration
Programming Language
Programming Languages
Reporting and Analytics
Spark
SQL
Talend
Job Description
Brooksource is seeking a Senior Data Engineer to design, develop, and maintain scalable data pipelines and architectures that support advanced analytics and business intelligence. This role centers on building robust data frameworks using big data technologies and cloud platforms, with an emphasis on data modeling and reliable data delivery across the organization.
Responsibilities
- Architect and optimize enterprise-grade ETL pipelines using Informatica, Talend, Apache Spark, and Apache Hive
- Design and maintain scalable data architectures on AWS, Azure Data Lake, and other public cloud environments
- Partner with cross-functional teams to capture requirements and translate them into scalable data solutions
- Create and implement dimensional data models for data warehousing and analytics
- Oversee data integration workflows involving linked data, RESTful APIs, and external sources
- Ensure data quality, security, and compliance throughout the data lifecycle
- Enable machine learning model deployment by integrating data pipelines efficiently
- Leverage SQL, Python, Java, and Bash/Shell scripting to automate and optimize processes
- Participate in agile ceremonies and pursue continuous improvement within sprint cycles
- Document data workflows and architecture diagrams to support transparency and knowledge sharing
Requirements
- Demonstrated experience as a Data Engineer or similar role with emphasis on big data systems and cloud architectures
- Deep knowledge of AWS services (S3, Redshift, Glue), Azure Data Lake, Hadoop ecosystem components, and other public cloud offerings
- Strong proficiency in SQL databases including Microsoft SQL Server, Oracle, and cloud-native databases
- Hands-on ETL/ELT development with tools like Informatica or Talend
- Expertise in data modeling techniques, including dimensional modeling for enterprise data warehouses
- Proficiency in programming and scripting languages such as Python, Java, VBA, Bash scripting
- Experience with business intelligence tools like Looker or similar platforms for reporting and analysis
- Understanding of RESTful API integration for linked data access
- Analytical capabilities related to database design, query management, model training, and analytics methodologies
- Experience working in Agile environments to deliver iterative value in fast-paced settings
Technologies
- Informatica
- Talend
- Apache Spark
- Apache Hive
- AWS
- Azure Data Lake
- Hadoop
- S3
- Redshift
- Glue
- Microsoft SQL Server
- Oracle
- SQL
- Python
- Java
- VBA
- Bash scripting
- Shell Scripting
- Looker
- RESTful API
Compensation
Pay: $65.00 - $85.00 per hour
Location
New York, NY (Remote)
Similar Jobs
