This position is no longer accepting applications
Closed on October 30, 2026.
This role is filled — get an email when new Data Processing roles open on EngineerJobs.io:
Senior Data Engineer
Senior
Apache Airflow
Big Data
Bigdata
Data Engineer
Data Integration
Data Pipeline
Data Processing
Jupyter Notebook
Slurm
Spark
Workflow Orchestration
View similar jobs
Get alerted when similar jobs are posted — set up a New Data Processing jobs on EngineerJobs.io alert.
See other roles at Shield Consulting Solutions.
Job Description
Shield Consulting Solutions is hiring a Senior Data Engineer in Annapolis Junction, MD (onsite) to build and optimize end-to-end analytics data pipelines.
Responsibilities
- Apply extensive expertise in dataflow design, data transport mechanisms, and Apache Spark based distributed processing
- Design, implement, and optimize data ingress and egress pathways for efficient, scalable, and reliable processing of analytics workloads
Requirements
- Minimum 7 years of relevant experience
- Bachelor’s degree in a technical discipline
- Experience using the Linux CLI and Linux tools
- Experience developing Bash scripts to automate manual processes
- Recent software development experience using Python and Java
- Experience with Apache Airflow including DAG design, scheduling, operators, and sensors for orchestrating and monitoring complex workflows
- Experience with distributed big data processing engines, including Apache Spark
- Familiarity with SQL technologies including MySQL, MariaDB, and PostgreSQL for querying, joining, and aggregating large datasets
- Experience using Jupyter Notebook
- Experience with data wrangling and preprocessing using pandas and NumPy
- Experience working with structured, semi-structured, and unstructured data formats including Parquet, JSON, CSV, and XML
- Familiarity with data quality concepts, data validation, and anomaly detection
- Experience with Git source control
- Familiarity with HPC job scheduling tools including Slurm
- Experience with the Atlassian Tool Suite including JIRA and Confluence
Technologies
- Apache Spark
- Apache Airflow
- Linux CLI
- Bash
- Python
- Java
- MySQL
- MariaDB
- PostgreSQL
- Jupyter Notebook
- pandas
- NumPy
- Parquet
- JSON
- CSV
- XML
- Git
- Slurm
- JIRA
- Confluence
Compensation
- $230,000 - $240,000 per year (USD)
Benefits
- 25 days PTO
- 11 paid holidays
- 100% employer-paid healthcare for employees and dependents, available day 1
- 8% 401(k) employer match with immediate vesting