Slurm Engineer Jobs

1 job found

Today

Senior Data Engineer

Shield Consulting Solutions Annapolis Junction, MD

Annapolis Junction, MD

Today

$230k - $240k/yr Full time Senior Apache Airflow Big Data

Apply Now

Job Description

Shield Consulting Solutions is hiring a Senior Data Engineer in Annapolis Junction, MD (onsite) to build and optimize end-to-end analytics data pipelines.

Responsibilities

Apply extensive expertise in dataflow design, data transport mechanisms, and Apache Spark based distributed processing
Design, implement, and optimize data ingress and egress pathways for efficient, scalable, and reliable processing of analytics workloads

Requirements

Minimum 7 years of relevant experience
Bachelor’s degree in a technical discipline
Experience using the Linux CLI and Linux tools
Experience developing Bash scripts to automate manual processes
Recent software development experience using Python and Java
Experience with Apache Airflow including DAG design, scheduling, operators, and sensors for orchestrating and monitoring complex workflows
Experience with distributed big data processing engines, including Apache Spark
Familiarity with SQL technologies including MySQL, MariaDB, and PostgreSQL for querying, joining, and aggregating large datasets
Experience using Jupyter Notebook
Experience with data wrangling and preprocessing using pandas and NumPy
Experience working with structured, semi-structured, and unstructured data formats including Parquet, JSON, CSV, and XML
Familiarity with data quality concepts, data validation, and anomaly detection
Experience with Git source control
Familiarity with HPC job scheduling tools including Slurm
Experience with the Atlassian Tool Suite including JIRA and Confluence

Technologies

Apache Spark
Apache Airflow
Linux CLI
Bash
Python
Java
MySQL
MariaDB
PostgreSQL
Jupyter Notebook
pandas
NumPy
Parquet
JSON
CSV
XML
Git
Slurm
JIRA
Confluence

Compensation

$230,000 - $240,000 per year (USD)

Benefits

25 days PTO
11 paid holidays
100% employer-paid healthcare for employees and dependents, available day 1
8% 401(k) employer match with immediate vesting

View full job page →

Get Slurm Engineer Jobs alerts

New Slurm Engineer Jobs, daily.