Clean

Mutate String Processing

Trims unwanted characters from the start and end of a string, using configurable modes and character sets.

Schema

- clean:
    field: <ident>
    target_field: <ident>
    mode: <string>
    chars: <string>
    keep_chars: <string>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y	-	Source field to trim
target_field	N	field	Field to store the result. Defaults to field, trimming in place
mode	N	custom	Which characters count as "keep": alphanumeric, numeric, alpha, custom
chars	N	see below	Characters that may be trimmed. custom mode only
keep_chars	N	-	Extra characters that stop the trim, in any mode
description	N	-	Explanatory note
if	N	-	Condition to run processor
ignore_failure	N	false	Continue if processor fails
ignore_missing	N	false	Continue if source field doesn't exist
on_failure	N	-	Processors to run on failure
on_success	N	-	Processors to run on success
tag	N	-	Processor identifier
disabled	N	false	When true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration

Details

The processor trims the ends. It does not remove characters from the middle of a string.

Scanning starts at each end and stops at the first character the mode says to keep; everything between those two points is returned untouched, however many unwanted characters it contains.

This is worth stating plainly because the obvious reading of "cleaning modes" is the opposite one. mode: numeric does not extract the digits from a phone number — it trims whatever sits outside the first and last digit and leaves the punctuation between them exactly as it was.

Runes are decoded whole, so multi-byte characters are never cut in half.

Value types

Value	Behavior
String	Trimmed
Array of strings	Each element trimmed independently
Array containing non-strings	Each element is stringified first, then trimmed
Any other scalar — a number, a boolean, an object	Error. A bare non-string field is not converted

Modes

The mode decides which characters stop the trim. Everything else is eligible to be trimmed off the ends.

Mode	Trimming stops at
alphanumeric	The first letter or digit
numeric	The first digit
alpha	The first letter
custom	The first character not listed in chars

custom is the default. With no chars given it uses a built-in set of quotes, brackets, punctuation and whitespace:

"'`()[]{}/<>\|!@#$%^&*+=~,;: 	

keep_chars is checked before the mode in every case, so a character listed there always stops the trim even if the mode would otherwise discard it.

Examples

Trimming Padding

Stripping decoration from around a value...	{ "username": "***admin_user***" } - clean: field: username mode: alphanumeric
the underscore survives — it is in the middle, and only the ends are trimmed:	{ "username": "admin_user" }

What numeric Does Not Do

numeric trims to the first and last digit; it does not extract digits...	{ "phone": "(555) 123-4567" } - clean: field: phone mode: numeric target_field: phone_trimmed
the interior punctuation is untouched:	{ "phone": "(555) 123-4567", "phone_trimmed": "555) 123-4567" }

note

To strip every non-digit from a phone number, use Gsub with a pattern such as [^0-9]. clean cannot do it.

Default Custom Mode

With no mode and no chars, the built-in delimiter set applies...	{ "raw": " \"payload\"; " } - clean: field: raw target_field: value
quotes, semicolon and whitespace are trimmed from both ends:	{ "raw": " \"payload\"; ", "value": "payload" }

Keeping a Character

keep_chars stops the trim before a character the mode would remove...	{ "path": "//var/log//" } - clean: field: path mode: alphanumeric keep_chars: "/"
so the leading and trailing slashes survive and nothing is trimmed at all:	{ "path": "//var/log//" }

Arrays

Every element is trimmed independently...	{ "tags": [" alpha ", "!beta!", "gamma"] } - clean: field: tags mode: alphanumeric
and the array stays an array:	{ "tags": ["alpha", "beta", "gamma"] }