Splunk Administrator Operational Technology (OT)

Splunk Administrator Operational Technology (OT) Summary : The OT Splunk Administrator is responsible for administering, maintaining, and optimizing the Splunk Enterprise platform supporting Operational Technology (OT) environments, including substations, grid management systems, telecom networks, data centers, and critical OT cyber-security infrastructure. This role supports both security operations and regulatory compliance, enabling accurate log ingestion, detection use-case development, dashboarding, and reporting aligned with NERC CIP, internal controls, and the OT Compliance Center of Excellence. The Splunk Administrator will partner closely with OT Network Security Analysts, Firewall & Network teams, Enterprise Cyber Security, and Compliance teams to ensure log integrity, visibility, and platform reliability across critical OT assets. Key Responsibilities : Splunk Platform Administration Maintain, administer, and troubleshoot Splunk Enterprise (indexers, search heads, forwarders, deployment server, cluster management). Ensure system uptime, performance tuning, capacity planning, and scaling for OT-critical workloads. Install, configure, and maintain Splunk Universal Forwarders on OT systems (Windows, Linux, appliances where applicable). Manage Splunk apps, add-ons, data models, and knowledge objects. OT-Specific Log Ingestion & Content Development Onboard OT data sources such as firewalls, switches / routers, SCADA-adjacent systems, VPN concentrators, RSA SecureID, Tripwire Enterprise, endpoint security, and network monitoring tools. Maintain and validate NERC CIP-relevant log retention and log integrity requirements. Build and maintain dashboards, alerts, correlation searches, and reports for OT security and compliance use cases. Create OT-specific detection content in collaboration with OT Network Security Analysts. Security Operations & Monitoring Support Partner with security operations teams to troubleshoot missing logs, ingestion errors, and detection gaps. Conduct root-cause analysis for log issues impacting OT security visibility. Support incident response by providing Splunk queries, timelines, and data exports. Regulatory Compliance Enablement Support internal and external audits with evidence extracts, dashboards, and log integrity verification. Ensure Splunk configurations meet compliance expectations (e.g., NERC CIP-007, CIP-010, CIP-003 monitoring controls). Maintain documentation for logging architectures, data flows, and procedures aligned with the Compliance Center of Excellence. Automation & Tooling Integration Integrate Splunk with ServiceNow for alerting, ticket creation, and automated workflows. Collaborate with Firewall Governance, PKI, RSA, and VPN lifecycle stakeholders for log enhancement opportunities. Maintain scripting (Python, PowerShell, Bash) for automation of ingestion and monitoring tasks. Operational Support & Governance Maintain work registers, operational runbooks, and architectural documents Provide knowledge transfer Minimum Qualifications 3–5+ years' experience administering Splunk Enterprise, preferably in an industrial, utility, or OT environment. Strong understanding of : Splunk configuration, tuning, and troubleshooting Log ingestion pipelines Windows & Linux server administration Network security concepts (firewalls, VPN, routing, segmentation) OT / ICS protocols (preferred, not required) Ability to obtain and maintain NERC CIP access requirements. Preferred Qualifications Bachelor’s degree in Cybersecurity, Information Systems, Engineering, or related field — or equivalent experience. Experience in utility OT environments (telecom, substations, control centers, power generation, or pipeline operations). Familiarity with : Tripwire, RSA SecureID, SCADA systems, firewall governance NERC CIP requirements Splunk ES or Splunk ITSI Scripting automation (Python, PowerShell, Bash) Experience developing dashboards, correlation searches, and detection content. Key Competencies Strong analytical and troubleshooting capability Clear written documentation and evidence-generation skills Stakeholder partnership and cross-functional communication Ability to work in ambiguous, high-stakes OT environments Accountability, follow-through, and consistency