Skip to main content

DataStream Stats

Pull

Synopsis

The device subscribes to the Director's internal NATS JetStream stats stream and republishes route, target, device, pipeline, device-resource, queue, and log-severity metrics through standard pipelines and routes. Each device instance attaches its own durable consumer with an independent read position. Output field naming is selectable between ECS, Cribl, and Prometheus conventions.

Schema

- id: <numeric>
name: <string>
description: <string>
type: stats
tags: <string[]>
pipelines: <pipeline[]>
status: <boolean>
properties:
field_format: <string>

Configuration

The following fields are used to define the device:

Device

FieldRequiredDefaultDescription
idY-Unique numeric identifier
nameY-Device name
descriptionN-Optional description
typeY-Must be stats
tagsN-Array of labels for categorization
pipelinesN-Array of preprocessing pipeline references
statusNtrueEnable/disable the device

Output

FieldRequiredDefaultDescription
field_formatNcriblOutput field naming convention. One of ecs, cribl, prometheus.

An unrecognized field_format is not rejected. The device logs a warning at startup and falls through to the internal ECS field names, so a typo produces a working collector whose field names do not match what was asked for.

Details

What Gets Emitted

Records fall into one of seven input_type categories:

  • route — per-route counters including events and bytes in/out, latency, errors, and drops
  • target — per-target counters with the same shape as route records, plus a target identifier
  • device — per-device counters tracking events and bytes processed by each device
  • pipeline — per-pipeline execution_count and execution_time aggregates
  • device_resource — host resource usage with resource_type of volume, cpu, memory, folder, or nats, plus an identifier, item count, and total/utilized sizes
  • queue — queue depth and discard counters including pending and discarded files and bytes
  • log_severity — per-interval deltas of Warning and Error/Critical debug-log entries, attributed to the device or target that recorded them

Every record carries the full base field set. Fields that do not apply to a given row are present with an empty string or a zero value.

An input type the device has no mapping for still converts, but reaches the target as input_type: "unknown" stripped of its metric columns. The collector logs a warning once per batch when this happens.

Output Fields

The tables below give the emitted field name in each of the three conventions. The ECS column is also the internal representation, used verbatim when field_format is ecs or unrecognized.

Base Fields

Present on every record.

ECSCriblPrometheusDescription
timestamp_timetimestamp_msRecord timestamp, RFC3339 with nanosecond precision
event_time_event_timeevent_timeTime the record was produced, RFC3339 with nanosecond precision
route_namecribl_routerouteRoute that processed the events
target_namecribl_outputtargetDestination name
pipeline_namecribl_pipepipelinePipeline name
source_namecribl_inputsourceSource name
output_name_output_nameoutputOutput identifier
vendor_name_vendorvendorVendor name
processor_name_processor_nameprocessorProcessor name
target_type_target_typetarget_typeTarget type identifier
director_namecribl_hostinstanceDirector name
cluster_name_cluster_nameclusterCluster name
collector_hostname_collector_hostnamecollector_hostnameOS hostname of the process that produced the record. In containerized deployments this is the replica or pod name, and the only per-replica discriminator
device_name_device_namedeviceDevice name
device_type_device_typedevice_typeDevice type
device_id_device_iddevice_idDevice identifier
director_id_director_iddirector_idDirector identifier
cluster_id_cluster_idcluster_idCluster identifier
target_id_target_idtarget_idTarget identifier
definition_id_definition_iddefinition_idDefinition identifier
input_type_input_typeinput_typeRecord category. One of the seven listed above, or unknown
type_typedirectionin or out. Absent on log_severity records
object_kind_object_kindobject_kinddevice or target. Present only on log_severity records

Both timestamps are RFC3339 nanosecond strings in all three conventions, including Prometheus, where the field is named timestamp_ms.

type and object_kind are mutually exclusive. Log-severity records omit type rather than carrying a third value in it, so a consumer keyed on type sees an absent key instead of an unexpected enum member.

Route, Target, and Device Fields

ECSCriblPrometheusDescription
events_in_events_inevents_in_totalInbound event count
events_out_events_outevents_out_totalOutbound event count
bytes_inbytes_inbytes_in_totalInbound bytes
bytes_outbytes_outbytes_out_totalOutbound bytes
latency_ns_latency_nslatency_nanosecondsBatch processing wall time, in nanoseconds
error_count_error_counterrors_totalErrors recorded in the interval
dropped_count_dropped_countdropped_totalEvents dropped in the interval
target_count_target_counttarget_countNumber of targets involved
compressed_bytes_compressed_bytescompressed_bytes_totalCompressed byte count on out records. On in records the same column carries the average per-record delivery latency in nanoseconds, or 0 when the producer took no samples
target_identifier_target_identifiertarget_identifierTarget endpoint URL or address. Set on out records
source_hostname_source_hostnamesource_hostnameHostname parsed from the received events. Set on in records

Pipeline Fields

ECSCriblPrometheusDescription
execution_count_execution_countexecution_count_totalPipeline executions in the interval
execution_time_execution_timeexecution_time_nanosecondsAggregate execution time, in nanoseconds

Device Resource Fields

ECSCriblPrometheusDescription
resource_type_resource_typeresource_typevolume, cpu, memory, folder, or nats
resource_identifier_resource_identifierresource_identifierMount point, CPU core, DIMM locator, folder name, or message-store item
resource_count_resource_countresource_countNumber of items of this resource type on the host
total_size_total_sizetotal_size_bytesTotal capacity. See the unit note below
utilized_size_utilized_sizeutilized_size_bytesUtilized capacity. See the unit note below
crash_detected_crash_detectedcrash_detected1 when the process restarted after an unclean exit, otherwise 0
startup_detected_startup_detectedstartup_detected1 on the first resource record after a process start, otherwise 0

total_size and utilized_size are reported in kilobytes for volume, memory, and folder, and in megahertz for cpu. The nats resource type overloads both columns per identifier, reporting connection and message-store counters rather than a capacity. The _bytes suffix the Prometheus convention gives these two fields is a naming convention, not a unit.

On device-resource records source_name and target_name are always empty: the underlying columns carry the resource type and identifier instead.

Queue Fields

ECSCriblPrometheusDescription
queue_pending_files_queue_pending_filesqueue_pending_filesFiles pending in the queue
queue_pending_bytes_queue_pending_bytesqueue_pending_bytesBytes pending in the queue
queue_discarded_files_queue_discarded_filesqueue_discarded_files_totalFiles discarded from the queue
queue_discarded_bytes_queue_discarded_bytesqueue_discarded_bytes_totalBytes discarded from the queue

Log Severity Fields

ECSCriblPrometheusDescription
log_warning_count_log_warning_countlog_warnings_totalWarning entries recorded since the previous emit
log_error_count_log_error_countlog_errors_totalError and Critical entries recorded since the previous emit

The attributed object rides in device_id, device_name, and device_type for both kinds: on a target record those three columns carry the target's identifier, name, and type. object_kind says which kind they describe. A record is emitted only when at least one of the two counts is non-zero.

Output Formats

Three field naming conventions are supported:

  • ecs — flat snake_case internal representation using ECS-aligned field names such as route_name, target_name, latency_ns, and error_count
  • cribl — Cribl Stream internal-metrics naming, with cribl_* system fields (cribl_route, cribl_output, cribl_pipe, cribl_input, cribl_host) and underscore-prefixed custom fields (_time, _type, _events_in, _error_count). bytes_in and bytes_out are first-class Cribl fields and carry no prefix
  • prometheus — Prometheus base-unit naming with _total, _nanoseconds, and _bytes suffixes (events_in_total, latency_nanoseconds, bytes_in_total), and instance and direction as labels

Deprecated Properties

poll_interval and workers are accepted but control nothing. The device stopped polling the stats database on a timer when it moved to a durable JetStream consumer, so neither property has anything left to configure. Unknown device properties are never rejected, so configurations that still set them remain valid, and neither is validated any more.

Delivery

Each device attaches one durable consumer to the stats stream under the role device-<id>, so every stats device reads the stream independently with its own position and catches up from the earliest retained batch on start. The stream retains roughly five minutes of batches: a device stopped for longer resumes from the live edge, losing the interval in between.

Delivery is best-effort. A batch that fails to convert is acknowledged and dropped rather than redelivered, so a single bad batch cannot stall the consumer or arrive twice.

Lifecycle

The device has no external network connectivity. On start it reports ConnectionStateConnected and subscribes to the JetStream stats stream with its per-device durable consumer, retrying with backoff until JetStream is ready. If 120 seconds pass without an internal heartbeat, the device reports ConnectionStateErrorHeartbeat and stops the collector.

Restart Triggers

Changing field_format triggers a clean collector restart. It is read once and handed to the poller at startup. Changes to other configuration fields do not restart the collector.

Examples

Basic Configuration

Creating a basic statistics publisher with default Cribl output naming...

- id: 1
name: stats-publisher
type: stats

Route records arrive with Cribl field naming...

{
"_time": "2026-08-19T14:20:00.000000000Z",
"_event_time": "2026-08-19T14:20:01.412330000Z",
"_input_type": "route",
"_type": "in",
"cribl_route": "main-route",
"cribl_output": "splunk-target",
"cribl_pipe": "parse-pipeline",
"cribl_input": "syslog-input",
"cribl_host": "director-01",
"_collector_hostname": "director-01-7f9c4b",
"_events_in": 14820,
"_events_out": 14820,
"bytes_in": 9437184,
"bytes_out": 9437184,
"_latency_ns": 412000,
"_error_count": 0,
"_dropped_count": 0
}

ECS Format

Configuring flat ECS field naming for the emitted records...

- id: 2
name: stats-ecs
type: stats
properties:
field_format: ecs

Route records use ECS-aligned snake_case field names...

{
"timestamp": "2026-08-19T14:20:00.000000000Z",
"event_time": "2026-08-19T14:20:01.412330000Z",
"input_type": "route",
"type": "in",
"route_name": "main-route",
"target_name": "splunk-target",
"pipeline_name": "parse-pipeline",
"source_name": "syslog-input",
"director_name": "director-01",
"collector_hostname": "director-01-7f9c4b",
"events_in": 14820,
"events_out": 14820,
"bytes_in": 9437184,
"bytes_out": 9437184,
"latency_ns": 412000,
"error_count": 0,
"dropped_count": 0
}

Cribl Format

Configuring Cribl Stream internal-metrics field naming explicitly...

- id: 3
name: stats-cribl
type: stats
properties:
field_format: cribl

Target records use Cribl system fields with underscore-prefixed custom fields...

{
"_time": "2026-08-19T14:20:00.000000000Z",
"_input_type": "target",
"_type": "out",
"cribl_route": "main-route",
"cribl_output": "splunk-target",
"cribl_host": "director-01",
"_target_identifier": "https://splunk.example.com:8088",
"_target_type": "splunkhec",
"_events_out": 14820,
"bytes_out": 9437184,
"_compressed_bytes": 1258291,
"_target_count": 1,
"_error_count": 0,
"_dropped_count": 0
}

Prometheus Format

Configuring Prometheus base-unit field naming for the emitted records...

- id: 4
name: stats-prometheus
type: stats
properties:
field_format: prometheus

Route records use Prometheus naming with _total, _nanoseconds, and _bytes suffixes...

{
"timestamp_ms": "2026-08-19T14:20:00.000000000Z",
"input_type": "route",
"direction": "in",
"route": "main-route",
"target": "splunk-target",
"source": "syslog-input",
"instance": "director-01",
"collector_hostname": "director-01-7f9c4b",
"events_in_total": 14820,
"events_out_total": 14820,
"bytes_in_total": 9437184,
"bytes_out_total": 9437184,
"latency_nanoseconds": 412000,
"errors_total": 0,
"dropped_total": 0
}

Log Severity Records

Collecting per-device Warning and Error counts alongside the throughput metrics...

- id: 5
name: stats-ecs
type: stats
properties:
field_format: ecs

The record identifies the attributed object through object_kind and carries no type field...

{
"timestamp": "2026-08-19T14:20:00.000000000Z",
"input_type": "log_severity",
"object_kind": "device",
"device_id": 42,
"device_name": "fw-01",
"device_type": "syslog",
"director_name": "director-01",
"collector_hostname": "director-01-7f9c4b",
"log_warning_count": 4,
"log_error_count": 2
}

Device Resource Records

Host resource usage arrives as one record per volume, core, or memory module...

{
"timestamp": "2026-08-19T14:20:00.000000000Z",
"input_type": "device_resource",
"type": "in",
"resource_type": "volume",
"resource_identifier": "/var/lib/datastream",
"resource_count": 3,
"total_size": 104806400,
"utilized_size": 52403200,
"crash_detected": 0,
"startup_detected": 1,
"source_name": "",
"target_name": ""
}

Sizes are in kilobytes for volumes, so this reports a 100 GB mount at 50 percent utilization...