Syslog

Synopsis

Creates a Syslog server that accepts log messages over UDP or TCP connections. Supports both plain and TLS-encrypted connections, with configurable framing and buffering options.

For details, see Appendix.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: syslog
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    protocol: <string>
    address: <string>
    port: <numeric>
    enable_udp: <boolean>
    framing: <string>
    pattern: <string>
    line_delimiter: <string>
    framing_rules:
      - name: <string>
        condition: <string>
        pattern: <string>
        max_event_bytes: <numeric>
        min_raw_length: <numeric>
    max_connections: <numeric>
    timeout: <numeric>
    tenants:
      - tenant_id: <string>
        ip_blocks: <string[]>
    tls:
      status: <boolean>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      min_tls_version: <string>
      max_tls_version: <string>
      client_ca_name: <string>
      client_auth_required: <boolean>
      insecure_skip_verify: <boolean>
    reuse: <boolean>
    workers: <numeric>
    max_message_size: <numeric>
    batch_size: <numeric>
    forwarding:
      - address: <string>
        port: <numeric>
        type: <string>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
id	Y		Unique identifier
name	Y		Device name
description	N	-	Optional description
type	Y		Must be syslog
tags	N	-	Optional tags
pipelines	N	-	Optional pre-processor pipelines
status	N	true	Enable/disable the device

Protocol

Field	Required	Default	Description
protocol	N	"udp"	Transport protocol (udp or tcp).
address	N	"0.0.0.0"	Listen address
port	Y		Listen port

TCP

The following are only applicable when protocol is set to tcp.

Field	Required	Default	Description
framing	N	"delimiter"	Framing mode for TCP (delimiter, octet, regex, or advanced)
pattern	Y*	-	Event-breaker regex pattern; required when framing is regex
line_delimiter	N	"\n"	Line separator for TCP delimiter framing
enable_udp	N	false	Also bind a plaintext UDP listener on the same port, so clients that send via UDP are still accepted. Ignored when protocol is udp.

* = Required when framing is regex

note

delimiter, octet, regex, and advanced are the canonical framing modes. In the GUI, the Framing dropdown labels octet-counting as RFC6587, which writes framing: rfc6587. Syslog transport labels are also accepted as aliases and normalized on load:

Alias	Canonical mode
rfc6587, rfc5425, rfc5424	octet
rfc3195	delimiter

TLS

The following are only applicable when protocol is set to tcp.

Field	Required	Default	Description
tls.status	N	false	Enable TLS encryption
tls.cert_name	Y*	cert.pem	TLS certificate
tls.key_name	Y*	key.pem	TLS private key
tls.passphrase	N	-	Passphrase for an encrypted private key
tls.min_tls_version	N	tls1.2	Minimum accepted TLS version (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	-	Maximum accepted TLS version. When unset, the highest mutually supported version is negotiated.
tls.client_ca_name	N	-	CA bundle used to verify client certificates (mTLS)
tls.client_auth_required	N**	false	Require connecting clients to present a valid certificate. When false, a client certificate is verified only if one is presented.
tls.insecure_skip_verify	N	false	Skip peer certificate verification. Use only for testing.

* = Required when tls.status is true.

** = Requires tls.client_ca_name. If tls.client_auth_required is true or tls.client_ca_name is set and the named CA cannot be loaded, the configuration is rejected and the device fails to start.

note

tls.min_version is a deprecated alias for tls.min_tls_version, honored only when tls.min_tls_version is unset. Use tls.min_tls_version in new configurations.

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

• File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
• Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
• Inline PEM content — used verbatim when the value contains -----BEGIN.
• Environment variable — ${ENV_VAR}.
• Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

note

When enable_udp is true, a secondary UDP listener binds the same port as the TCP listener. The secondary listener is always plaintext — DTLS is not supported, so the UDP fallback carries cleartext even when tls.status is true. It always uses delimiter framing regardless of the TCP framing setting. Changing enable_udp restarts the device collector.

Access List

The access_list property is a source-IP firewall: an ordered list of accept/drop rules over IP blocks, evaluated before the listener parses or authenticates the connection. When it is not set, the firewall is off and adds no overhead.

Field	Required	Default	Description
access_list	N	-	Ordered list of source-IP rules. Omit to disable the firewall.
access_list[].action	Y		accept or drop (synonyms: allow/permit and deny/reject/block). Unrecognized actions are ignored.
access_list[].ip_blocks	Y		One or more IP blocks the rule matches. Comma-separated string or a list. Accepts CIDR (10.0.0.0/24), single IP (172.16.5.10), dotted-netmask (192.168.1.0/255.255.255.0, IPv4 only), and inclusive range (10.5.0.1-10.5.0.50). IPv4 and IPv6.
access_list_default	N	auto	Verdict for a source IP that matches no rule: accept or drop. When omitted, it is derived automatically (see below).

Rules are evaluated top to bottom and the first rule whose blocks contain the source IP decides the outcome. A source IP that matches no rule follows the default policy:

• If any accept rule is present, the list is treated as an allowlist and unmatched IPs are dropped.
• Otherwise the list is treated as a denylist and unmatched IPs are accepted.

Set access_list_default to override this. An unparseable source IP is judged by the default policy, so an allowlist fails closed.

access_list:
  - action: accept
    ip_blocks: "10.0.0.0/8, 192.168.0.0/16, 172.16.5.10"

note

Editing or removing access_list rules applies on the next configuration reconcile with no listener restart. Enabling access_list for the first time on a device that had none takes effect on the next collector restart.

When a source IP is denied over TCP, the connection is accepted and then closed immediately, before framing and before the TLS handshake. Denied UDP datagrams are silently discarded.

Tenants

The tenants property maps a source IP to a tenant: an ordered list of rules matched against the connection's (or datagram's) source address before the record enters the pipeline. Multiple IP blocks can map to the same tenant, and multiple tenants are supported. When it is not set, no tenant is attached.

Field	Required	Default	Description
tenants	N	-	Ordered list of source-IP-to-tenant rules. Omit to disable IP-based tenancy.
tenants[].tenant_id	Y		Tenant identifier attached to records arriving from the matching IP blocks.
tenants[].ip_blocks	Y		One or more IP blocks the rule matches. Comma-separated string or a list. Accepts CIDR (10.0.0.0/24), single IP (172.16.5.10), dotted-netmask (192.168.1.0/255.255.255.0, IPv4 only), and inclusive range (10.5.0.1-10.5.0.50). IPv4 and IPv6.

Rules are evaluated top to bottom and the first rule whose blocks contain the source IP wins. A source IP that matches no rule receives no tenant — the record is still ingested. When a tenant is resolved it is written to _vmetric.event.tenant_id; the client IP always stays at _vmetric.event.request. Use _vmetric.event.tenant_id to route or isolate each customer's data downstream (see Routes).

tenants:
  - tenant_id: acme
    ip_blocks: "10.0.0.0/24, 10.1.2.3, 192.168.10.0/255.255.255.0"
  - tenant_id: globex
    ip_blocks: "10.2.0.0/16, 172.16.5.1-172.16.5.50"

note

Editing or removing tenants rules applies on the next configuration reconcile with no listener restart. Enabling tenants for the first time on a device that had none takes effect on the next collector restart.

The source IP is matched per TCP connection and per UDP datagram.

Advanced Configuration

To enhance performance and achieve better data handling, the following settings are used.

Performance

Field	Required	Default	Description
reuse	N	true	Enable socket address reuse
workers	N	One per 8 CPU cores	Number of listener workers, never below 1 and reduced further on memory-constrained hosts. Forced to 1 when reuse is false.
max_connections	N	10000	Maximum concurrent TCP connections
max_message_size	N	20971520	Maximum message size in bytes (20MB)
timeout	N	300	Connection timeout in seconds
batch_size	N	10000	Number of messages to batch before flushing
max_buffer_size	N	128MB	Ingest buffer admission cap. Also sets this listener's eager heap reservation

note

flush_interval and queue.interval are Director service-level settings configured in vmetric.yml and cannot be overridden per device. max_buffer_size is the exception: it has a service-level default that a device can override.

note

max_buffer_size is the ingest-buffer admission cap, and is distinct from any buffer_size field above, which sizes the network read buffer. Left unset it is 128MB, which commits 384 MiB of heap per listener worker before any data arrives — and with reuse: true a device runs one worker per listener. Size strings are binary, and 32MB and 32MiB are exact synonyms.

See Listener Memory Sizing for the arithmetic, the sizing table, and what happens when the cap is exhausted.

Forwarding

Field	Required	Default	Description
forwarding[].address	Y		Forward destination address
forwarding[].port	N	514	Forward destination port
forwarding[].type	N	"udp"	Forward protocol (udp or tcp)

Framing Rules

Framing rules only apply when protocol is tcp.

Ordered event-breaking rules for TCP connections, used when framing is set to advanced, which requires at least one rule.

At connection open the listener buffers the first N bytes of the stream, where N is the largest min_raw_length across all rules. The first rule whose condition matches that buffer frames every event for the rest of the connection's lifetime. Conditions are a stream classifier, not a per-event filter: they are evaluated once, in order, and never re-checked. All rules use regex-based event breaking.

Field	Required	Default	Description
framing_rules[].name	N	"rule-N"	Descriptive rule name for logs
framing_rules[].condition	N	-	RE2 regex matched unanchored against the buffered prefix to select this rule; empty = unconditional catch-all, permitted on the last rule only
framing_rules[].pattern	Y	-	Event-breaker regex marking the start of each event
framing_rules[].max_event_bytes	N	max_message_size	Per-rule event size cap in bytes; falls back to device-level max_message_size
framing_rules[].min_raw_length	N	256	Bytes to buffer before conditions are evaluated. The largest value across all rules applies to the whole device, capped at the smaller of max_message_size and 1MB

Condition Matching

condition is a Go RE2 regular expression, matched unanchored and case-sensitively against the buffered prefix:

• Plain text behaves like contains: CEF: selects the rule wherever CEF: appears in the prefix. Anchor with ^ for the start of the stream, or (?m)^ for the start of any line within it.
• Metacharacters are live. An unescaped . matches any character, so 10.0.0.1 also matches 10x0y0z1. Write 10\.0\.0\.1 for the literal address.
• Prefix the expression with (?i) for case-insensitive matching.
• Conditions match decoded text. When encoding is set, the prefix is transcoded to UTF-8 before conditions see it, so write them in terms of decoded characters, not wire bytes.
• An invalid expression is a configuration error and the device fails to start.

Condition	Selects the rule when
^CEF:	The stream opens with CEF:
(?m)^CEF:	Any line in the buffered prefix starts with CEF:
(?i)cisco	cisco appears anywhere in the prefix, in any case
10\.0\.0\.1	The literal address 10.0.0.1 appears in the prefix

min_raw_length is also a latency setting: nothing is emitted for a connection until N bytes have arrived or the idle timeout fires. Raise it only as far as the conditions need — a large value delays the first event from slow or low-volume senders. A value above the smaller of max_message_size and 1MB is clamped to that ceiling, and the clamp is reported in the Console Logs.

note

The last rule should carry an empty condition as the unconditional fallback. An unconditional rule matches every stream, so no rule after it can ever be selected: the device still starts, but the shadowed rules are dropped and the loss is reported in the Console Logs. When the last rule does carry a condition, connections matching no rule are closed without being framed.

note

Regex framing is event-start oriented: each match marks the beginning of a new event, and everything between consecutive matches is one complete event. The pattern must not match the empty string.

Examples

The following are commonly used configuration types.

Basic

Creating a simple UDP syslog server...

devices: - id: 1 name: basic_syslog type: syslog properties: port: 514

Checkpoint

The basic UDP Server can be configured to use a checkpoint pre-processing pipeline. This is a pre-processing pipeline that extracts Checkpoint firewall logs from syslog messages:

Creating a simple UDP syslog server with checkpoint...

devices: - id: 2 name: basic_syslog type: syslog tags: - "network_device" pipelines: - checkpoint properties: address: "10.0.0.1" protocol: "udp" port: 1514

note

If the device is a Checkpoint firewall, this pipeline will parse the logs and extract relevant fields for further processing. Otherwise, the pipeline will have no effect on the incoming messages.

High-Volume

Tuning a UDP server for high message volumes...

devices: - id: 3 name: performant_syslog type: syslog properties: protocol: udp port: 514 reuse: true workers: 4 batch_size: 5000

note

The worker count scales as one worker per 8 CPU cores rather than one per core, and is reduced further on memory-constrained hosts. Setting workers without enabling reuse has no effect: the count is forced to 1, because the workers rely on SO_REUSEPORT to share the listening socket.

Framing

TCP server with custom message framing, connection limits, and an idle timeout...

devices: - id: 4 name: tcp_syslog type: syslog properties: protocol: tcp port: 1514 framing: delimiter line_delimiter: "\r\n" max_connections: 5000 timeout: 60

warning

When using TCP with delimiter framing, ensure the line_delimiter matches the client side.

Single Port (TCP + UDP)

Binding a secondary plaintext UDP listener on the same TCP port, accepting clients that send via UDP...

devices: - id: 8 name: dual_syslog type: syslog properties: protocol: tcp port: 514 enable_udp: true

Advanced Framing

TCP syslog server with per-connection event-breaking rules, selected from the initial bytes. Conditions are anchored with ^ so they match the start of the stream, and the last rule carries no condition, making it the fallback...

devices: - id: 5 name: advanced_syslog type: syslog properties: protocol: tcp port: 1514 framing: advanced framing_rules: - name: cef condition: "^CEF:" pattern: "^CEF:" - name: multiline-timestamp condition: "^\\d{4}-" pattern: "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}" - name: fallback pattern: "\\n"

Security

Securing the server with TLS encryption and forwarding to mixed destinations...

devices: - id: 6 name: secure_syslog type: syslog properties: protocol: tcp port: 6514 tls: status: true cert_name: cert.pem key_name: key.pem forwarding: - address: "10.0.0.1" port: 514 type: udp - address: "10.0.0.2" port: 6514 type: tcp

Forwarding

Forwarding replicates incoming messages unmodified to all configured destinations. This is useful for network devices that can only send syslog data to a single destination.

Fan-out: UDP messages received on port 514 are replicated to one UDP and two TCP destinations...

devices: - id: 7 name: forwarder_syslog type: syslog properties: protocol: udp port: 514 forwarding: - address: "10.0.0.50" port: 514 type: udp - address: "10.0.0.51" port: 1514 type: tcp - address: "syslog.example.com" port: 6514 type: tcp

warning

When using TCP forwarding, ensure the destination servers can handle the connection load as each connection is persistent.

Multi-Tenant

One syslog port serving several customers, each identified by its source network. The matched tenant_id is attached for downstream routing:

Mapping source networks to tenants on a single UDP listener...	devices: - id: 9 name: saas_syslog type: syslog properties: port: 514 protocol: udp tenants: - tenant_id: acme ip_blocks: "10.0.0.0/24" - tenant_id: globex ip_blocks: "10.2.0.0/16, 172.16.5.1-172.16.5.50"
A datagram from 10.0.0.7 is tagged with the acme tenant; the source IP is preserved...	{ "_vmetric": { "device": { "id": 9, "name": "saas_syslog", "type": "syslog" }, "event": { "request": "10.0.0.7", "tenant_id": "acme" } } }