Suppress

Analytics

Synopsis

Drops duplicate events that share the same derived key within a configurable time window, allowing only the first occurrence to pass through.

Schema

- suppress:
    key_expr: <string>
    window_sec: <integer>
    exclude_filters: <array>
    tag: <string>
    description: <text>
    if: <script>
    disabled: <boolean>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>

Configuration

Field	Required	Default	Description
key_expr	Y		Go template expression evaluated against the log entry to derive the suppression key (e.g., {{SrcIpAddr}}-{{DstIpAddr}}-{{name}})
window_sec	N	30	Time window in seconds during which events with the same key are suppressed
exclude_filters	N		List of filter expressions; events matching any filter bypass suppression entirely
tag	N		Identifier for this processor instance
description	N		Explanatory note
if	N		Condition that must be true for the processor to run
disabled	N	false	When true, the processor is skipped
ignore_failure	N	false	Continue processing if the processor encounters an error. This does not mean "keep filtering and ignore errors" — see the warning below.
ignore_missing	N	false	When true, falls back to key "default" if template evaluation fails due to missing fields
on_failure	N		Processors to run when this processor fails or drops an event
on_success	N		Processors to run when this processor passes an event through

warning

ignore_failure: true silently turns this processor off. The drop is signalled to the pipeline as an error value, and the ignore_failure check runs before that value is inspected — so a matched event is kept instead of dropped, no error is logged, and the pipeline reports success. The processor appears to run normally while filtering nothing.

Use it only if you genuinely want a pass-through. To tolerate real errors without losing the filtering, leave ignore_failure unset and handle the failure with on_failure instead.

Details

The suppress processor tracks events by a key derived from a Go template expression evaluated against each log entry. The first event for a given key starts a suppression window; all subsequent events with the same key that arrive within window_sec seconds are dropped. After the window expires, the next event with that key starts a new window and passes through.

The suppression state is maintained in a per-pipeline in-memory cache. Each cache slot is keyed by suppress: followed by the xxhash64 of the evaluated key string. The cache entry records the time of the first-seen event and a count of dropped events within the current window.

When an event passes through, the processor writes the evaluated key string to _vmetric.suppress_key on the log entry. Dropped events do not receive this field.

If ignore_missing is true and the template expression references fields that do not exist on the log entry, the processor falls back to the literal key value "default" rather than failing. All events that fail template evaluation then share the same default key and are subject to suppression against each other.

The exclude_filters field accepts an array of boolean expressions. Each filter is evaluated in order; if any filter matches, the event bypasses all suppression logic and passes through unconditionally. Exclude filters are evaluated before the suppression cache is consulted, so matching events never affect the cache state.

Examples

Basic

Suppressing repeated connection alerts from the same source/destination pair...	{ "SrcIpAddr": "10.0.0.1", "DstIpAddr": "10.0.0.2", "name": "tcp_weird" } - suppress: key_expr: "{{SrcIpAddr}}-{{DstIpAddr}}-{{name}}" window_sec: 30
First event passes and receives _vmetric.suppress_key; identical events within 30 seconds are dropped...	{ "SrcIpAddr": "10.0.0.1", "DstIpAddr": "10.0.0.2", "name": "tcp_weird", "_vmetric": { "suppress_key": "10.0.0.1-10.0.0.2-tcp_weird" } }

With Exclude Filters

Suppressing duplicates while always passing high-priority events...	{ "SrcIpAddr": "10.0.0.1", "DstIpAddr": "10.0.0.2", "name": "tcp_weird", "priority": "high" } - suppress: key_expr: "{{SrcIpAddr}}-{{DstIpAddr}}-{{name}}" window_sec: 60 exclude_filters: - "priority == 'high'"
Events with priority: high bypass suppression; all other duplicates are dropped within the window...	{ "SrcIpAddr": "10.0.0.1", "DstIpAddr": "10.0.0.2", "name": "tcp_weird", "priority": "high" }

With Missing Field Handling

Handling log entries where key fields may be absent...	{ "SrcIpAddr": "10.0.0.1" } - suppress: key_expr: "{{SrcIpAddr}}-{{DstIpAddr}}-{{name}}" window_sec: 30 ignore_missing: true
Template evaluation fails due to missing fields; processor falls back to key "default" and suppresses subsequent events sharing the same fallback key...	{ "SrcIpAddr": "10.0.0.1", "_vmetric": { "suppress_key": "default" } }

With Chained Processors

Running downstream processors only on the first occurrence of each event key...	- suppress: key_expr: "{{host}}-{{name}}" window_sec: 300 on_success: - set: field: alert.first_seen value: true on_failure: - set: field: alert.suppressed value: true
Passing events receive alert.first_seen; suppressed events trigger the on_failure chain and receive alert.suppressed...