Data Engineer
Apache Airflow
Big Data
Bigdata
Cloud Data Warehouse
Data
Data Analysis
Data Engineer
Data Integration
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Warehouse
Database
Databases
ETL
Flink
Hadoop
Hbase
Informatica
Information Technology (IT)
Integration
Kafka
Nifi
Nosql Databases
Programming Languages
Spark
SQL
Stream Processing
Job Description
Hybrid Data Engineer role in Knoxville, TN focused on modern big data ingestion, integration, and transformation.
Responsibilities
- Design and implement data engineering components aligned to client requirements and technical solution goals
- Build and maintain batch and real-time ingestion, integration, and transformation workflows
- Develop reliable, efficient data pipelines using large-scale data processing and storage technologies
- Support analytics, search, and aggregation capabilities across data platforms
- Participate in technical design discussions and help maintain solution health and performance
- Operate cloud-based data platform services and optimize data workflows for performance and scalability
- Collaborate with project teams in a hybrid environment, including onsite interviews or equipment pickup when required
Requirements
- 4+ years of hands-on experience with big data technologies
- Experience with Hadoop ecosystem tools such as HDFS, Kafka, Spark, Spark Streaming, Hive, NiFi, Airflow, Flink, Storm, Oozie, or similar platforms
- Strong programming experience in Java, Python, or Scala (Java preferred)
- Working knowledge of NoSQL and MPP data platforms such as HBase, MongoDB, Cassandra, AWS Redshift, Azure SQLDW, or GCP BigQuery
- Experience with AWS data platform services
- Ability to translate business needs into technical design and implementation
- Strong problem-solving skills and ability to work independently during design discussions and solution delivery
Technologies
- HDFS, Kafka, Spark, Spark Streaming, Hive, NiFi, Airflow, Flink, Storm, Oozie
- Java, Python, Scala
- HBase, MongoDB, Cassandra
- AWS Redshift, Azure SQLDW, GCP BigQuery, AWS
- ActiveMQ, RabbitMQ, Solace
- Informatica, Talend
- Oracle, MySQL, SQL Server, Postgres
- Collibra, Alation
Preferred Skills
- Experience with traditional ETL tools such as Informatica or Talend
- Familiarity with database technologies such as Oracle, MySQL, SQL Server, or Postgres
- Knowledge of data governance practices, including security, lineage, and catalog tools like Collibra or Alation
- Exposure to distributed messaging tools such as ActiveMQ, RabbitMQ, or Solace
- Experience with search/indexing and microservices architectures
- Background in performance tuning and data pipeline optimization
- Cloud data certifications or other relevant big data credentials
Additional Information
- Hybrid role based in Knoxville, TN; onsite in Knoxville or East Coast office as needed
- Horizontal Talent may contact you using automated technology, including phone calls, SMS/text messages, or email delivered by a virtual AI recruiter named Alex
- Horizontal supports an inclusive workplace where diverse perspectives are valued
Similar Jobs
S
