Syslog

Synopsis

Creates a Syslog server that accepts log messages over UDP or TCP connections. Supports both plain and TLS-encrypted connections, with configurable framing and buffering options.

For details, see Appendix.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: syslog
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    protocol: <string>
    address: <string>
    port: <numeric>
    framing: <string>
    pattern: <string>
    line_delimiter: <string>
    framing_rules:
      - name: <string>
        condition: <string>
        pattern: <string>
        max_event_bytes: <numeric>
        min_raw_length: <numeric>
    max_connections: <numeric>
    timeout: <numeric>
    tls:
      status: <boolean>
      cert_name: <string>
      key_name: <string>
    reuse: <boolean>
    workers: <numeric>
    buffer_size: <numeric>
    max_message_size: <numeric>
    flush_interval: <numeric>
    batch_size: <numeric>
    queue:
      interval: <numeric>
    forwarding:
      - address: <string>
        port: <numeric>
        type: <string>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
id	Y		Unique identifier
name	Y		Device name
description	N	-	Optional description
type	Y		Must be syslog
tags	N	-	Optional tags
pipelines	N	-	Optional pre-processor pipelines
status	N	true	Enable/disable the device

Protocol

Field	Required	Default	Description
protocol	N	"udp"	Transport protocol (udp or tcp)
address	N	"0.0.0.0"	Listen address
port	Y		Listen port

TCP

The following are only applicable when protocol is set to tcp.

Field	Required	Default	Description
framing	N	"delimiter"	Framing mode for TCP (delimiter, octet, regex, or advanced)
pattern	Y*	-	Event-breaker regex pattern; required when framing is regex
line_delimiter	N	"\n"	Line separator for TCP delimiter framing

* = Required when framing is regex

TLS

The following are only applicable when protocol is set to tcp.

Field	Required	Default	Description
tls.status	N	false	Enable TLS encryption
tls.cert_name	Y		TLS certificate file name (required if TLS enabled)
tls.key_name	Y		TLS private key file name (required if TLS enabled)

note

The TLS certificate and key files must be placed in the service root directory.

Advanced Configuration

To enhance performance and achieve better data handling, the following settings are used.

Performance

Field	Required	Default	Description
reuse	N	true	Enable socket address reuse
workers	N	<dynamic>	Number of worker processes when reuse enabled
max_connections	N	10000	Maximum concurrent TCP connections
max_message_size	N	20971520	Maximum message size in bytes (20MB)
timeout	N	300	Connection timeout in seconds
buffer_size	N	9000	Network read buffer size in bytes

Messages

Field	Required	Default	Description
flush_interval	N	1	Message flush interval in seconds
batch_size	N	1000	Number of messages to batch before flushing
queue.interval	N	1	Queue processing interval in seconds

Forwarding

Field	Required	Default	Description
forwarding[].address	Y		Forward destination address
forwarding[].port	N	514	Forward destination port
forwarding[].type	N	"udp"	Forward protocol (udp or tcp)

Framing Rules

Ordered event-breaking rules for TCP connections, used when framing is set to advanced.

At connection open, the first min_raw_length bytes are buffered. The first rule whose condition matches the buffered bytes is selected for the lifetime of that connection. The last rule should have an empty condition to act as the unconditional fallback. All rules use regex-based event breaking.

Field	Required	Default	Description
framing_rules[].name	N	"rule-N"	Descriptive rule name for logs
framing_rules[].condition	N	-	Regex matched against initial bytes to select this rule; empty = unconditional
framing_rules[].pattern	Y	-	Event-breaker regex marking the start of each event
framing_rules[].max_event_bytes	N	max_message_size	Per-rule event size cap in bytes; falls back to device-level max_message_size
framing_rules[].min_raw_length	N	256	Minimum bytes to buffer before evaluating condition

note

Framing rules only apply when protocol is tcp. Regex framing is event-start oriented: each regex match marks the beginning of a new event. Everything between consecutive matches is one complete event. The pattern must not match the empty string.

Examples

The following are commonly used configuration types.

Basic

A basic configuration can be created easily using "udp" for protocol and "0.0.0.0" for address.

Creating a simple UDP syslog server...

devices: - id: 1 name: basic_syslog type: syslog properties: port: 514

Checkpoint

The basic UDP Server can be configured to use a checkpoint pre-processing pipeline. This is a pre-processing pipeline that extracts Checkpoint firewall logs from syslog messages:

Creating a simple UDP syslog server with checkpoint...

devices: - id: 2 name: basic_syslog type: syslog tags: - "network_device" pipelines: - checkpoint properties: address: "10.0.0.1" protocol: "udp" port: 1514

note

If the device is a Checkpoint firewall, this pipeline will parse the logs and extract relevant fields for further processing. Otherwise, the pipeline will have no effect on the incoming messages.

High-Volume

Performance of a UDP server can be enhanced Volumes using multiple workers, a larger buffer size, a larger batch size, and adjusted flush intervals.

Optimizing for high message volumes...

devices: - id: 3 name: performant_syslog type: syslog properties: protocol: udp port: 514 reuse: true workers: 4 buffer_size: 32768 batch_size: 5000 flush_interval: 5 queue: interval: 2

note

The worker count will be automatically capped at the maximum number of physical cores available on the system.

Framing

For a TCP server with custom message framing, use a custom frame delimiter, connection limits, and an idle timeout:

TCP server with custom message framing...

devices: - id: 4 name: tcp_syslog type: syslog properties: protocol: tcp port: 1514 framing: delimiter line_delimiter: "\r\n" max_connections: 5000 timeout: 60

warning

When using TCP with delimiter framing, ensure the line_delimiter matches the client side.

Advanced Framing

For a TCP syslog server receiving multi-line events from mixed sources, use framing: advanced with framing_rules to select the event-breaking pattern per connection based on the initial bytes:

TCP syslog server with per-connection event-breaking rules...

devices: - id: 5 name: advanced_syslog type: syslog properties: protocol: tcp port: 1514 framing: advanced framing_rules: - name: multiline-timestamp condition: "^\\d{4}-" pattern: "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}" - name: fallback pattern: "\\n"

Security

Security can be enhanced using TLS encryption, multiple forward destinations, and mixed protocols:

Securing the server forwarding the messages...

devices: - id: 6 name: secure_syslog type: syslog properties: protocol: tcp port: 6514 tls: status: true cert_name: cert.pem key_name: key.pem forwarding: - address: "10.0.0.1" port: 514 type: udp - address: "10.0.0.2" port: 6514 type: tcp

Forwarding

For message forwarding use network devices with single syslog output. This can also be implemented on legacy systems that need multiple destinations. The messages can be fanned out to different analysis tools.

note

Forwarding acts as a message replicator, sending exact copies of incoming messages to all configured destinations unmodified. This is particularly useful for network devices that can only send syslog data to a single destination.

The messages are forwarded exactly as received to the UDP server on port 514, and the TCP server on ports 1514 and 6514.

Forwarding incoming messages to multiple destinations...

devices: - id: 7 name: forwarder_syslog type: syslog properties: protocol: udp port: 514 forwarding: - address: "10.0.0.50" port: 514 type: udp - address: "10.0.0.51" port: 1514 type: tcp - address: "syslog.example.com" port: 6514 type: tcp

warning

When using TCP forwarding, ensure the destination servers can handle the connection load as each connection is persistent.