NATS

Synopsis

Creates a JetStream consumer that connects to NATS servers and processes messages from specified streams and subjects. Supports authentication, TLS encryption, and multiple workers with automatic message acknowledgment.

For details, see Appendix.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: nats
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    port: <numeric>
    username: <string>
    password: <string>
    auth_type: <string>
    stream: <string>
    consumer: <string>
    subject: <string>
    max_bytes: <numeric>
    timeout: <numeric>
    batch_size: <numeric>
    reuse: <boolean>
    workers: <numeric>
    tls:
      status: <boolean>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      ca_name: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
      insecure_skip_verify: <boolean>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
id	Y	-	Unique numeric identifier
name	Y	-	Device name
description	N	-	Optional description
type	Y	-	Must be nats
tags	N	-	Optional tags
pipelines	N	-	Optional pre-processor pipelines
status	N	true	Enable/disable the device

Connection

Field	Required	Default	Description
address	N	"0.0.0.0"	NATS server address. The console requires it on a new device; a YAML that leaves it out dials 0.0.0.0
port	Y	4222	NATS server port
username	N	-	Authentication username (or JWT for jwt auth, or NKey identity for nkey auth)
password	N	-	Authentication password (or JWT seed for jwt auth, or NKey seed for nkey auth)
auth_type	N	-	Authentication type: basic, jwt, nkey, or none. When omitted with credentials present, jwt is inferred if username contains a ., otherwise basic; with no credentials, none is used

Stream

Field	Required	Default	Description
stream	N	"vmetric"	JetStream stream name
consumer	N	See Consumer	The durable JetStream consumer the director reads the stream through. The director creates it on the NATS server, or sets its subject, when the device starts
subject	Y	-	Subject pattern to subscribe to
max_bytes	N	1048576	Maximum message size in bytes (1 MB default)
timeout	N	5	Fetch timeout in seconds (1-30)
batch_size	N	10000	Number of messages to fetch per request

note

Only the stream must exist on the NATS server before the device can subscribe, and its subjects must include the device's subject: the server accepts a subject outside them, and the device then receives nothing. Create the stream with nats stream add, or set stream to an existing stream; the default is vmetric. The director creates the consumer itself.

warning

Give each device its own consumer, unless two devices are meant to split one subject between them. Devices that read one stream through the same consumer share its messages. When their subjects differ, the device that starts last sets its subject on the consumer: messages outside that subject are never read, and the messages it matches are split between both devices, without any error. See Sharing a Consumer.

Consumer

Which consumer a device reads through depends on how the device is saved, and whether consumer is set:

Saved as	consumer empty	consumer set
A new device, in the console, the API or a Bicep deployment	The platform picks a name for the device and saves it on the device. See The Name the Platform Picks	That name, after the checks in Sharing a Consumer
An update, in the console, the API or a Bicep deployment	The device keeps its current consumer. A device that was saved with no consumer keeps reading through vmetric	That name, after the same checks
A device in a device file on the director	vmetric, the director's default	That name

The platform does not check devices in device files on the director.

The Name the Platform Picks

The name is vmetric_ followed by the device name. The platform lowercases the device name and turns accented letters into plain ones. Every run of other characters becomes one _, and the name is cut to 64 characters. If another NATS device of the tenant already uses the name, the platform adds _2, _3 and so on:

Device name	Consumer
Firewall Logs	vmetric_firewall_logs
firewall-logs, saved after the device above	vmetric_firewall_logs_2
Güvenlik Duvarı	vmetric_guvenlik_duvari
A name with no Latin letters or digits, such as 日志	vmetric_nats

A rename never changes the consumer. A new consumer starts at the beginning of the stream, so the device would read every stored message again, and the old consumer would stay on the NATS server.

Sharing a Consumer

"The same consumer" means the same NATS server (address and port), the same stream and the same consumer name.

• Two devices on the same subject may share a consumer. They split the messages between them, which spreads the load. Each message goes to one device, in batches of up to batch_size messages, and order is not kept across the devices. One device on a Director cluster does this by itself, because every member reads through the same consumer.
• Two devices with different subjects may not share a consumer. The save is refused with NATS_CONSUMER_IN_USE (409), and the message names the other device.
• Names that NATS refuses are refused on save with NATS_CONSUMER_NAME_INVALID (400): a name with a space, ., *, >, / or \, or one longer than 255 characters.
• Devices saved earlier stay editable. An update that leaves the address, port, stream, consumer and subject alone is never refused, even for two devices that already share a consumer with different subjects.

The check cannot see other applications on the NATS server, devices of other tenants, the same NATS server reached under two addresses, or devices in device files on the director. It does not stand in for giving each device its own consumer.

The check also does not look at the NATS account a device signs in to. One NATS server can hold several accounts, and each account can have a stream and a consumer with the same names as another account's. Devices that sign in to different accounts share nothing, but the check still counts them as one consumer. So a second device with a different subject is refused with NATS_CONSUMER_IN_USE, although the two would never share a message. To save it, give its consumer a different name.

warning

A work-queue stream, a stream with WorkQueue retention, lets only one consumer read each subject. NATS refuses to create a second consumer whose subject overlaps one that a consumer of the stream already reads, such as logs.> and logs.firewall. The device then does not start, and the director logs the reason filtered consumer not unique on workqueue stream. A new device gets a consumer of its own, so on a work-queue stream:

• Devices that read the same subject must share one consumer. Set consumer on each of them to the name of the consumer that already reads that subject, rather than to a new name: a new name leaves the old consumer on the server, and NATS refuses the new one because its subject overlaps.
• A consumer that no device or application reads any more still holds its subject. Find it with nats consumer ls <stream> and delete it with nats consumer rm <stream> <consumer>.
• Devices whose subjects overlap but are not the same cannot both read the stream. Change their subjects so that they no longer overlap.

The platform does not check the stream's retention, so neither case is refused on save.

TLS

Field	Required	Default	Description
tls.status	N	false	Enable TLS encryption
tls.cert_name	Y*	cert.pem	Client certificate for mutual TLS
tls.key_name	Y*	key.pem	Client private key for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted private key
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset, the host trust store is used; when set, it replaces the host trust store rather than adding to it.
tls.server_name	N	-	SNI hostname override for the TLS handshake
tls.min_tls_version	N	tls1.2	Minimum accepted TLS version (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	-	Maximum accepted TLS version. When unset, the highest mutually supported version is negotiated.
tls.insecure_skip_verify	N	false	Skip server certificate verification. Use only for testing.

* = Required when tls.status is true.

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

• File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
• Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
• Inline PEM content — used verbatim when the value contains -----BEGIN.
• Environment variable — ${ENV_VAR}.
• Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

Performance

Field	Required	Default	Description
reuse	N	true	Enable multi-worker mode
workers	N	4	Number of worker processes when reuse enabled (capped at the number of available CPU cores)
max_buffer_size	N	128MB	Ingest buffer admission cap. Also sizes this listener's ingest buffers: 3/8 of this value at start, growing to at most 3x

note

max_buffer_size is the ingest-buffer admission cap. Left unset it is 128MB, whose buffers take 48 MiB of heap per listener worker at start and grow to at most 384 MiB as bursts need the room — and with reuse: true a device runs one worker per listener. Size strings are binary, and 32MB and 32MiB are exact synonyms.

See Listener Memory Sizing for the arithmetic, the sizing table, and what happens when the cap is exhausted.

Examples

Basic

Creating a simple NATS consumer using the default stream and consumer...

- id: 1 name: basic_nats type: nats properties: address: "nats.example.com" port: 4222 subject: "logs.>"

Secure

Connecting with basic authentication and TLS encryption...

- id: 2 name: secure_nats type: nats properties: address: "nats.example.com" port: 4222 username: "consumer" password: "secret" auth_type: "basic" stream: "LOGS" consumer: "processor" subject: "logs.secure.>" tls: status: true cert_name: "nats.crt" key_name: "nats.key"

High-Volume

Tuning batch size and timeout for high-throughput consumption...

- id: 3 name: performant_nats type: nats properties: address: "nats.example.com" port: 4222 stream: "LOGS" consumer: "high-perf" subject: "logs.>" max_bytes: 4194304 timeout: 10 batch_size: 50000 reuse: true workers: 4

Subject Filters

Subject-based filtering with wildcards...

- id: 4 name: filtered_nats type: nats properties: address: "nats.example.com" port: 4222 stream: "LOGS" consumer: "filtered" subject: "logs.*.error" reuse: true workers: 2

tip

NATS subjects support * and > as wildcards for single and multiple tokens respectively.

Pipelines

Applying custom processing to messages...

- id: 5 name: pipeline_nats type: nats pipelines: - json_parser - field_extractor properties: address: "nats.example.com" port: 4222 stream: "LOGS" consumer: "processed" subject: "logs.raw.>"

note

Pipelines are processed sequentially, and can modify or drop messages before ingestion.

Shared Stream

Three devices reading one stream, whose subjects include both logs.> and audit.>. The first two read the same subject through the same consumer, and split its messages between them. The third reads another subject, so it has a consumer of its own...

- id: 6 name: logs_reader_a type: nats properties: address: "nats.example.com" port: 4222 stream: "LOGS" consumer: "logs_shared" subject: "logs.>" - id: 7 name: logs_reader_b type: nats properties: address: "nats.example.com" port: 4222 stream: "LOGS" consumer: "logs_shared" subject: "logs.>" - id: 8 name: audit_reader type: nats properties: address: "nats.example.com" port: 4222 stream: "LOGS" consumer: "audit" subject: "audit.>"

note

If audit_reader used logs_shared as well, one of the two subjects would never be read. Saved through the platform, it would be refused with NATS_CONSUMER_IN_USE.