Data Engineer
Job Description
Silverthorne Advisory Group LLC is hiring a Data Engineer to support Defense sector initiatives. This hybrid role in Washington, D.C. focuses on designing and maintaining data infrastructure and engineering solutions within the Advana environment, with scalable pipelines and reliable integrations across complex datasets.
The work includes ETL development, data quality, troubleshooting, and governance, along with collaboration across engineering, analytics, and stakeholder teams. Palantir Foundry experience is highly preferred to support integration, transformation, pipeline development, and analytics workflows.
Key Responsibilities
- Design, develop, and maintain end-to-end data pipelines using ETL and tools such as Databricks, Python, Spark/Scala, SQL, JavaScript/JSON, and Jupyter Notebooks.
- Build and optimize pipelines from scratch with a focus on scalability, reliability, efficiency, and high performance.
- Support data engineering and integration activities within the Advana platform.
- Use Palantir Foundry (when applicable) for data integration, transformation, pipeline development, and analytics workflows.
- Perform data cleansing, integration, transformation, and quality assurance to maintain accurate datasets.
- Process and analyze large, complex datasets using big data technologies.
- Troubleshoot pipeline issues, identify root causes, and implement effective technical solutions.
- Develop and maintain data models, database structures, and data architectures aligned to business and analytical needs.
- Implement data governance standards, security controls, and best practices to support federal requirements and compliance.
- Collaborate with data scientists, analysts, engineers, and stakeholders to deliver dependable data solutions.
- Maintain technical documentation for pipelines, applications, workflows, and processes.
- Evaluate emerging data engineering technologies and recommend improvements to existing systems.
Required Qualifications
- Secret security clearance or higher required; sponsorship may be available.
- Hands-on experience developing ETL pipelines and working with large datasets.
- Proficiency with Databricks.
- Proficiency with Python.
- Proficiency with Apache Spark / PySpark (including Spark).
- Proficiency with SQL.
- Proficiency with Scala.
- Proficiency with JavaScript / JSON.
- Proficiency with Jupyter Notebooks.
- Experience designing, developing, and optimizing pipelines from scratch.
- Strong understanding of data modeling, database design, data integration, and data governance.
- Experience with big data technologies and complex data environments.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and collaboration skills.
- Ability to work independently and collaboratively in a fast-paced technical environment.
- Strong attention to detail and commitment to delivering high-quality solutions.
Technologies
- Advana, Palantir Foundry
- ETL
- Databricks
- Python
- Apache Spark, PySpark, Spark
- Scala
- SQL
- JavaScript, JSON
- Jupyter Notebooks
- Power BI, Qlik Sense
Highly Preferred Qualifications
- Hands-on experience with Palantir Foundry, including data pipelines, data integration, transformations, and data modeling.
- Experience working within the Advana data environment.
- Previous experience supporting Department of Defense (DoD) or other federal government agencies.
- Familiarity with Palantir Foundry tools such as Pipeline Builder, Code Repositories, and Ontology.
- Experience with cloud-based data platforms and enterprise data architectures.
- Understanding of federal financial management data, reporting, or analytics environments.
- Experience supporting business intelligence and data visualization platforms such as Power BI or Qlik Sense.
Compensation and Benefits
- Salary: USD 120,000 - 140,000 per year
- Flexible work from home options available
- Healthcare benefits: Comprehensive medical, dental, and vision insurance plans for employees and their families
- 401(k) retirement plan: 100% company match on employee contributions up to the first 6%, with access to Vanguard Admiral funds
- Paid time off: Generous paid leave based on length of service
- Performance bonuses: Bonus opportunities for employees who meet and exceed performance goals
- Professional development: Opportunities for training, certifications, and career advancement
- Company retreats and team events: Sponsored trips, team-building activities, and professional conferences
Additional Job Details
- Location: Washington, D.C.
- Work arrangement: On-site/Hybrid (2 days a week)
- Employment type: Full-Time/Salaried
- Industry: Federal Government / Department of Defense (DoD)
- Experience level: All Levels
- Highly preferred: Palantir Foundry, Advana, Databricks experience