Splunk

Observability

Synopsis

Creates a Splunk HTTP Event Collector (HEC) target that sends events to one or more Splunk instances. Supports batching, compression, field normalization, and automatic load balancing across multiple endpoints.

Schema

- name: <string>
  description: <string>
  type: splunk
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    endpoints:
      - endpoint: <string>
        auth_type: <string>
        token: <string>
        secret: <string>
    index: <string>
    source_type: <string>
    source: <string>
    batch_size: <numeric>
    timeout: <numeric>
    tcp_routing: <boolean>
    use_compression: <boolean>
    insecure_skip_verify: <boolean>
    field_format: <string>
    interval: <string|numeric>
    cron: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following are the fields used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be splunk
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Endpoint

Field	Required	Default	Description
endpoint	Y	-	Splunk HEC endpoint URL
auth_type	N	token	Authentication type: token or secret
token	N	-	HEC token when using token auth
secret	N	-	Bearer token when using secret auth

Event

Field	Required	Default	Description
index	N	-	Default Splunk index
source_type	N	-	Default sourcetype for events
source	N	-	Default source for events
batch_size	N	10000	Number of events to batch before sending
timeout	N	30	Connection timeout in seconds

Connection

Field	Required	Default	Description
tcp_routing	N	false	Enable TCP routing header
use_compression	N	true	Enable gzip compression
insecure_skip_verify	N	false	Skip TLS certificate verification
field_format	N	-	Data normalization format. See applicable Normalization section

Scheduler

Field	Required	Default	Description
interval	N	realtime	Execution frequency. See Interval for details
cron	N	-	Cron expression for scheduled execution. See Cron for details

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

The Splunk HEC target sends log data to Splunk using the HTTP Event Collector (HEC) protocol. It supports multiple authentication methods, batching, compression, and automatic load balancing between endpoints.

warning

Ensure your HEC tokens have the appropriate permissions and indexes enabled in Splunk. Invalid tokens or insufficient permissions will result in ingestion failures.

Events are automatically batched and compressed by default for optimal performance. When multiple endpoints are configured, the target randomly selects an endpoint for each batch to distribute load evenly across all available Splunk instances.

warning

Setting insecure_skip_verify to true is not recommended for production environments.

HEC Modes: JSON and RAW

The target supports both JSON and RAW HEC ingestion modes. The mode is determined by the endpoint URL path, not by a separate configuration field.

An endpoint URL ending in /services/collector/event (or the default /services/collector) uses JSON mode, where each event is sent as a structured JSON object with metadata fields (index, sourcetype, source) embedded in the payload.

An endpoint URL ending in /services/collector/raw uses RAW mode, where events are sent as plain text with metadata passed as query parameters. RAW mode is useful for forwarding pre-formatted log lines that should not be wrapped in JSON structure. When the incoming pipeline data contains a _raw field, the target extracts its value and sends it as the raw event body.

Named Streams

Endpoint URLs can include a query parameter to define a named stream. For example, https://splunk.example.com:8088/services/collector/raw?paloalto creates a stream named paloalto using RAW mode, while https://splunk.example.com:8088/services/collector/event?winevent creates a stream named winevent using JSON mode. Named streams allow routing different event types through separate HEC paths within a single target configuration.

Load Balancing and Failover

When multiple endpoints are configured, the target uses randomized load balancing. For each batch:

1. Endpoints are randomly shuffled
2. The batch is sent to the first endpoint
3. If it fails, the next endpoint in the shuffled list is tried
4. This continues until successful delivery or all endpoints fail

If only some endpoints fail but delivery eventually succeeds, the batch is cleared and a partial error is logged. If all endpoints fail, the batch is retained for retry and a complete failure error is returned.

Dynamic Routing

The target supports dynamic routing of events to different indexes, sourcetypes, and sources using pipeline processors:

• Set the source field in a pipeline to override the default source
• Set the schema field in a pipeline to override the default sourcetype
• Set the index field in a pipeline to override the default index

This allows sending different event types to appropriate indexes without creating multiple target configurations.

Example pipeline configuration:

pipelines:
  - name: route_by_severity
    processors:
      - set:
          field: source
          value: "production-app"
      - set:
          field: schema
          value: "app:error"
          if: "severity == 'error'"
      - set:
          field: index
          value: "critical-logs"
          if: "severity == 'critical'"

Compression

Compression is enabled by default and uses gzip to reduce network bandwidth. This adds minimal CPU overhead but can significantly improve throughput for high-volume scenarios. Disable compression only if you have bandwidth to spare and want to reduce CPU usage.

Field Normalization

Field normalization helps standardize log data before sending it to Splunk, ensuring consistent data formats that can be easily correlated:

• cim - Common Information Model

Normalization is applied before batching and sending to Splunk.

Examples

Basic

Send events to a single HEC endpoint...

targets: - name: basic_splunk type: splunk properties: endpoints: - endpoint: "https://splunk.example.com:8088/services/collector" auth_type: token token: "YOUR-HEC-TOKEN" index: "main" source_type: "vmetric"

Load Balanced

Configure load balancing and failover across multiple endpoints...

targets: - name: ha_splunk type: splunk properties: endpoints: - endpoint: "https://splunk1.example.com:8088/services/collector" auth_type: token token: "PRIMARY-TOKEN" - endpoint: "https://splunk2.example.com:8088/services/collector" auth_type: token token: "BACKUP-TOKEN" - endpoint: "https://splunk3.example.com:8088/services/collector" auth_type: token token: "TERTIARY-TOKEN" index: "main" source_type: "vmetric" batch_size: 5000

High-Volume

Configure for high throughput with larger batches and extended timeout...

targets: - name: perf_splunk type: splunk properties: endpoints: - endpoint: "https://splunk.example.com:8088/services/collector" auth_type: token token: "YOUR-HEC-TOKEN" index: "metrics" source_type: "vmetric" batch_size: 20000 timeout: 60 use_compression: true tcp_routing: true

With Field Normalization

Using CIM field normalization for better Splunk integration...

targets: - name: normalized_splunk type: splunk properties: endpoints: - endpoint: "https://splunk.example.com:8088/services/collector" auth_type: token token: "YOUR-HEC-TOKEN" index: "main" source_type: "normalized_logs" field_format: "cim"

Secure

Using secret-based auth with TLS verification and custom source...

targets: - name: secure_splunk type: splunk properties: endpoints: - endpoint: "https://splunk.example.com:8088/services/collector" auth_type: secret secret: "YOUR-BEARER-TOKEN" index: "secure" source_type: "vmetric" source: "production_cluster" insecure_skip_verify: false use_compression: true

RAW Mode

Send raw log lines using HEC RAW mode with a named stream...

targets: - name: raw_splunk type: splunk properties: endpoints: - endpoint: "https://splunk.example.com:8088/services/collector/raw?firewall" auth_type: token token: "YOUR-HEC-TOKEN" index: "network" source_type: "paloalto:traffic" source: "pan-fw-01"

No Compression

Disable compression to reduce CPU overhead when bandwidth is not a concern...

targets: - name: uncompressed_splunk type: splunk properties: endpoints: - endpoint: "https://splunk.example.com:8088/services/collector" auth_type: token token: "YOUR-HEC-TOKEN" index: "main" source_type: "vmetric" use_compression: false