Data Engineer

Realign LLC is hiring a Data Engineer for a full-time, permanent onsite role in Charlotte, NC. This position centers on building reliable, secure data solutions with an emphasis on big data processing and cyber security domain knowledge, supporting faster, better, data-informed decision-making across the business. The role offers an opportunity to contribute to end-to-end delivery, pipeline reliability, and technical mentoring within an engineering-focused environment.

Responsibilities

• Deliver consistent design and engineering approaches for complex data solutions across multiple flows and systems.
• Perform research, prototype designs, and exploration including evaluating new toolsets.
• Use multiple architectural components to design and develop client requirements.
• Maintain, improve, clean, and manipulate data for operational and analytics data systems.
• Continuously improve how technical problems are solved, including challenging the status quo when needed.
• Collaborate with development teams to confirm data requirements and ensure data architecture is feasible and implemented accurately.
• Assemble large, complex datasets that meet functional and non-functional requirements.
• Build processes for data transformation, data structures, metadata, data quality controls, and dependency and workload management.
• Define and build data pipelines that enable faster and better business decisions.
• Contribute to integration, regression, and performance test suites; analyze test reports; identify issues/errors; and triage root causes.
• Document and communicate deployment, maintenance, support, and business functionality requirements.
• Follow team delivery and release cadence, including code deployment and release process.
• Identify gaps in data management standards adherence and work with partners to develop plans to close gaps.
• Lead concept and experimentation testing, then synthesize results to validate and improve analytical solutions.
• Define key performance indicators and internal controls.
• Mentor junior Data Engineers and coach team members in delivery and release CI-CD events, including branching timelines, pull requests, issue triage, merge and conflict resolution, and release notes.

Requirements

• Minimum 8+ years hands-on data engineer and/or software development experience, including at least 5 years onsite.
• Experience with big data processing systems such as Hadoop and Cloudera.
• Programming experience with Python, including modular Python programming.
• Strong SQL and UNIX experience.
• Hands-on Hadoop ecosystem experience including Hive, Spark, and Pispark.
• Proven technology leadership experience guiding junior engineers toward project completion and deliveries.
• Cyber security domain experience, preferably vulnerability data management.
• Ability to shift between technology leadership and hands-on coding when required.
• Experience with AWS, Azure, or other cloud services/technologies.
• Experience with data stores such as HIVE, HBase, and Elastic.
• Experience with code versioning systems such as Bitbucket.
• Experience with CI/CD pipelines.
• Good knowledge of data engineering principles and the SDLC process.
• Ability to detect early challenges and articulate requirements gaps with business partners.

Technologies

• Python
• Hadoop
• Cloudera
• Hadoop - Hive
• Spark
• Pispark
• SQL
• UNIX
• HIVE
• HBase
• Elastic
• AWS
• Azure
• Bitbucket
• CICD
• CI-CD

Role Details

Job type: Full Time

Job category: IT

Role: Data Engineer

Location: Charlotte, NC (Onsite)

Employment: Fulltime - Permanent

Objective of role: The candidate is expected to have knowledge in the cyber security domain.

Primary Level of Engagement and Interactions

• Engagement: Works as a team member under supervision from a more senior domain expert.
• Primary interactions: Business partners and stakeholders, Product Owner, Scrum Master, Development Team, Feature Lead, Architect, Domain Community of Practice, Data Scientists.