Skip to main content

Syslog

Parse Elastic Compatible

Synopsis​

Parses syslog messages into structured objects containing priority, facility, severity, hostname, application name, process ID, and message content.

note

See Appendix for details of the format.

Schema​

- syslog:
    field: <ident>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>
    target_field: <ident>
    use_table: <boolean>

Configuration​

The following fields are used to define the processor:

FieldRequiredDefaultDescription
fieldY-Field containing the syslog message
descriptionN-Explanatory note
ifN-Condition to run
ignore_failureNfalseSee Handling Failures
ignore_missingNfalseIf true, quietly exit if field doesn't exist
on_failureN-See Handling Failures
on_successN-See Handling Success
tagN-Identifier
disabledNfalseWhen true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration
target_fieldNlog.syslogField to store the parsed syslog object. Ignored when use_table is set
use_tableNfalseWrite the parsed components into the Syslog virtual table instead of building the log.syslog map

Details​

Table Mode​

With use_table: true the parsed components go straight into the Syslog virtual table rather than into a map: Facility, SeverityLevel, HostName, ProcessID, ProcessName and SyslogMessage (falling back to the raw field when the parser finds no message body).

The table must already exist. Run create_table first; the processor fails with virtual table "Syslog" not created rather than falling back to the map, so a misconfigured pipeline is visible instead of silently dropping the parse.

EventTime and HostIP are not parser outputs — set them yourself with a date or set step against $Syslog.*. RFC 5424 structured data is dropped in this mode: the table has no column for a nested parameter bag, so a pipeline that needs the SD block must stay on the map path.

The parse-derived format is published to _vmetric.syslog_format as CEF, LEEF or native.

Supported Formats​

The processor supports:

  • RFC 3164 (BSD-style) syslog messages
  • RFC 5424 (modern format) syslog messages with structured data
  • Common Event Format (CEF) messages
  • Log Event Extended Format (LEEF) messages
warning

The field must contain a valid syslog message string that conforms to one of the supported formats. Invalid messages will cause the processor to fail unless ignore_failure is set to true.

Examples​

Basic​

Parsing a basic syslog message...

{
  "syslog_message": "<190>Jul 23 18:07:21 dhcp6c[10256]: add an address 2a02:cf40:72dc:dd12:7378:913c:b42e:099c/128 on igb0"
}
- syslog:
    field: syslog_message
    target_field: parsed_syslog

extracts message components:

{
  "parsed_syslog": {
    "priority": 190,
    "facility": {
      "code": 23,
      "name": "Local 7"
    },
    "severity": {
      "code": 6,
      "name": "Informational"
    },
    "appname": "dhcp6c",
    "pid": 10256,
    "message": "add an address 2a02:cf40:72dc:dd12:7378:913c:b42e:099c/128 on igb0"
  }
}

Metadata​

Parsing an RFC 5424 format message...

{
  "syslog_message": "<134>1 2022-06-09T14:44:11-06:00 OPNsense.example.com filterlog 76404 - [meta sequenceId=\"1\"] 124,,,fae559338f65e11c53669fc3642c93c2,ixl1_vlan70,match,pass,out"
}
- syslog:
    field: syslog_message
    target_field: parsed_syslog

includes the structured data:

{
  "parsed_syslog": {
    "priority": 134,
    "facility": {
      "code": 16,
      "name": "Local 0"
    },
    "severity": {
      "code": 6,
      "name": "Informational"
    },
    "hostname": "OPNsense.example.com",
    "appname": "filterlog",
    "pid": 76404,
    "message": "124,,,fae559338f65e11c53669fc3642c93c2,ixl1_vlan70,match,pass,out"
  }
}

CEF​

Parsing a Cisco Firepower CEF message...

{
  "syslog_message": "<134>MAR  1 16:23:11 policyuuid CEF:0|Cisco|Firepower|6.0|PV:112:1234:5678|POLICY VIOLATION|5|rt=1687855290000;deviceExternalId=12;act=Alerted;dvchost=10.50.60.100"
}
- syslog:
    field: syslog_message
    target_field: parsed_syslog

extracts the CEF data:

{
  "parsed_syslog": {
    "priority": 134,
    "facility": {
      "code": 16,
      "name": "Local 0"
    },
    "severity": {
      "code": 6,
      "name": "Informational"
    },
    "hostname": "policyuuid",
    "message": "CEF:0|Cisco|Firepower|6.0|PV:112:1234:5678|POLICY VIOLATION|5|rt=1687855290000;deviceExternalId=12;act=Alerted;dvchost=10.50.60.100"
  }
}

Error Messages​

Parsing the error level messages...

{
  "syslog_message": "<27>1 2021-07-03T22:17:01.074560-05:00 pfSense.example.com openvpn 66026 - - TLS Error: cannot locate HMAC in incoming packet from [AF_INET]175.16.199.1:34745"
}
- syslog:
    field: syslog_message
    target_field: parsed_syslog

correctly identifies the severity:

{
  "parsed_syslog": {
    "priority": 27,
    "facility": {
      "code": 3,
      "name": "System"
    },
    "severity": {
      "code": 3,
      "name": "Error"
    },
    "hostname": "pfSense.example.com",
    "appname": "openvpn",
    "pid": 66026,
    "message": "TLS Error: cannot locate HMAC in incoming packet from [AF_INET]175.16.199.1:34745"
  }
}