Gsub

Filter Elastic Compatible

Synopsis

Performs pattern-based string replacements using regular expressions.

Schema

- gsub:
    field: <ident>
    pattern: <string>
    replacement: <string>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>
    target_field: <ident>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y	-	Source field containing text to modify
pattern	Y	-	Regular expression pattern to match
replacement	Y	-	Text or pattern to replace matches with
description	N	-	Documentation note
if	N	-	Conditional expression
ignore_failure	N	false	Skip processing errors
ignore_missing	N	false	Skip if input field missing
on_failure	N	-	Error handling processors
on_success	N	-	Success handling processors
tag	N	-	Identifier for logging
disabled	N	false	When true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration
target_field	N	field	Output field for modified text

Details

The processor supports both simple string replacements and complex regex patterns, with the ability to store results in the original field or a new target field.

note

The processor caches compiled regular expressions when dealing with multiple documents with the same pattern. Complex patterns are only compiled once and reused for subsequent matches.

warning

Be careful with complex regular expressions on large text fields, as these may impact performance. Test patterns thoroughly, and consider using simpler patterns when possible.

Regular Expression Semantics

Patterns compile with Go's RE2 engine. A pattern RE2 rejects — one using a lookaround or a backreference — is retried on a .NET-compatible backtracking engine, so both syntaxes are accepted. Every match made by that fallback engine is bounded by a 100ms timeout: a pattern that exceeds it fails the record with an error, and the rest of the pipeline continues.

Matching is unanchored and case-sensitive. A plain string matches anywhere in the field — anchor with ^ and $, and prefix the expression with (?i) for case-insensitive matching.

A pattern is rejected before it compiles when it:

• exceeds 1000 characters,
• uses more than 100 quantifiers (*, +, {n,m}),
• nests groups more than 10 levels deep, or
• takes longer than 100ms to compile.

Examples

Basic

Replacing error code in message...	{ "message": "Error: 404 Not Found" } - gsub: field: message pattern: "404" replacement: "200"
modifies original field:	{ "message": "Error: 200 Not Found" }

Anonymization

Anonymizing IP addresses in logs...	{ "log": "2021-04-15 00:00:00 192.168.1.1 GET /index.html - 80 - 192.168.1.100 Mozilla/5.0" } - gsub: field: log pattern: "\\b(?:\\d{1,3}\\.){3}\\d{1,3}\\b" replacement: "ANONYMIZED"
replaces all:	{ "log": "2021-04-15 00:00:00 ANONYMIZED GET /index.html - 80 - ANONYMIZED Mozilla/5.0" }

Keep Original

Storing the modified text in a new field...	{ "message": "Error: 404 Not Found" } - gsub: field: message pattern: "Error" replacement: "Warning" target_field: new_message
preserves the original field:	{ "message": "Error: 404 Not Found", "new_message": "Warning: 404 Not Found" }

Conditionals

Replacing based on criteria...	{ "message": "Error: Connection failed", "should_process": true } - gsub: field: message pattern: "Error" replacement: "Warning" if: "ctx.should_process == true"
executes the replacement conditionally:	{ "message": "Warning: Connection failed", "should_process": true }

Error Handling

Handling missing fields gracefully...	{ "other_field": "value" } - gsub: field: message pattern: "Error" replacement: "Warning" ignore_missing: true on_failure: - set: field: processing_status value: "field_missing"
continues the execution:	{ "other_field": "value", "processing_status": "field_missing" }