W
Senior Software Engineer
Job Description
Wyandotte Technologies is seeking a Senior Splunk Engineer/SME to support mission-critical systems for the Southwestern Power Administration (SWPA), U.S. Department of Energy. This 100% onsite role in Springfield, Missouri centers on hands-on Splunk enterprise architecture and administration within secure, highly restricted environments, including real-time SCADA operations.
In this position, you will serve as a senior technical owner for operational observability, advanced SPL development, data onboarding and normalization, and automation that improves reliability, performance, and troubleshooting speed for distributed Splunk deployments.
Location: Springfield, MO (onsite)
Salary: USD 110,000 - 115,000 per yearly
Experience: 5+ years
Responsibilities
- Act as the senior Splunk SME and technical owner for enterprise and operational Splunk capabilities.
- Design, administer, optimize, and troubleshoot distributed Splunk environments, including search heads, indexers, forwarders, deployment components, and clustered architectures as applicable.
- Develop and optimize advanced SPL for operational intelligence, troubleshooting, analytics, correlation, alerting, and performance at scale.
- Own data onboarding and normalization, including indexes, sourcetypes, props/transforms, field extractions, forwarder configuration, retention, access controls, and platform health.
- Engineer dashboards, reports, and actionable alerts to reduce time to detect, diagnose, and resolve operational issues.
- Design production-grade automation using PowerShell, Python, Bash, REST APIs, and other appropriate technologies to support Splunk and enterprise operations.
- Automate deployments, configuration, validation, monitoring, reporting, remediation, and recurring support activities.
- Integrate Splunk with applications, ServiceNow, automation workflows, infrastructure, security tooling, and other enterprise systems.
- Analyze application, infrastructure, security, and operational telemetry to identify trends, anomalies, failures, performance issues, and root causes.
- Establish reusable engineering standards, documentation, configuration practices, and support procedures.
- Mentor engineers and administrators and provide technical leadership for complex Splunk and observability issues.
- Identify manual, repetitive, and error-prone processes and engineer reliable automated solutions.
- Design, implement, and improve CI/CD pipelines and deployment automation.
- Automate build, test, release, deployment, validation, and rollback processes.
- Develop, enhance, integrate, and support custom enterprise applications.
- Modernize legacy applications and operational processes into secure, supportable architectures.
- Troubleshoot complex production issues and perform detailed root-cause analysis.
- Participate in Agile/Scrum development and sprint activities and create clear technical documentation.
- Support systems operating in secure and highly restricted environments, including real-time operational and SCADA-related systems.
- Participate in a 24/7 on-call rotation and occasional evening/weekend maintenance or deployment activities.
- Work closely with SWPA stakeholders, technical teams, and leadership to translate operational needs into reliable technical solutions.
Requirements
- Bachelor’s degree in Computer Science, Software Engineering, Information Technology, Cybersecurity, or a related technical field plus 7+ years of relevant professional experience; OR 11+ years of relevant Splunk, infrastructure, systems engineering, automation, DevOps, or implementation experience.
- 7+ years of enterprise infrastructure, security, systems, observability, automation, or related engineering experience.
- 5+ years of hands-on Splunk engineering or administration in enterprise or production environments, with demonstrated senior-level ownership.
- Advanced SPL expertise, including complex search development, troubleshooting, correlation, and search optimization.
- Hands-on experience administering core Splunk platform components such as indexers, search heads, universal/heavy forwarders, deployment server, and distributed or clustered environments.
- Demonstrated experience with data onboarding and normalization, including indexes, sourcetypes, props/transforms, field extractions, forwarders, and retention.
- Strong Linux and/or Windows systems troubleshooting experience and the ability to diagnose platform, application, network, and data-ingestion problems.
- Strong scripting and automation experience using PowerShell, Python, Bash, REST APIs, or comparable technologies.
- Experience with Git and CI/CD or configuration/deployment automation.
- Strong communication, client relationship, root-cause analysis, documentation, mentoring, and independent technical ownership skills.
- Splunk Enterprise Certified Admin, Splunk Enterprise Certified Architect, or comparable advanced Splunk certification strongly preferred.
- U.S. Citizenship required due to federal customer/system-access requirements.
- Must successfully complete a 7-year background investigation.
- 100% onsite in Springfield, Missouri. Candidates must be able and willing to work onsite (not remote or hybrid).
- Must participate in an on-call rotation supporting mission-critical systems.
- Must be available for occasional evening or weekend maintenance and deployment activities.
- Important Applicant Screening: Apply only if you meet all three: 5+ years hands-on Splunk engineering/administration, U.S. Citizenship, and 100% onsite work in Springfield, Missouri.
Technologies
- Splunk
- SPL
- PowerShell
- Python
- Bash
- REST APIs
- ServiceNow
- CI/CD
- Git
- Linux
- Windows
Benefits
- 401(k)
- 401(k) matching
- Dental insurance
- Employee assistance program
- Flexible spending account
- Health insurance
- Life insurance
- Paid time off
- Vision insurance
Preferred Qualifications
- Splunk Enterprise Certified Admin, Splunk Enterprise Certified Architect, or comparable advanced Splunk certification.
- Splunk Enterprise Security (ES), security monitoring/SIEM, or advanced operational observability experience.
- Large-scale distributed Splunk architecture, clustering, capacity/performance optimization, upgrade/migration, and platform lifecycle experience.
- Automated remediation, event-driven automation, REST APIs, ServiceNow integrations, and enterprise workflow automation.
- Experience supporting Linux and Windows in secure, restricted, federal, or other highly controlled computing environments.
- SCADA, industrial control systems, OT/IT integration, utilities, energy, real-time operational systems, or other critical-infrastructure experience.
- Technical leadership, architecture ownership, or mentoring of Splunk engineers and administrators.