Rewrite

Data Manipulation Cribl Compatible

Synopsis

Applies an ordered list of match-regex / replace-expression rules to each field listed in fields, replacing regex matches in the field's string value with the result of evaluating a JavaScript expression. rewrite is the native implementation of Cribl's Mask function.

Schema

- rewrite:
    fields:
      - <ident>
    rules:
      - match_regex: <string>
        replace_expr: <string>
    filter: <script>
    tag: <string>
    description: <text>
    if: <script>
    disabled: <boolean>
    ignore_failure: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>

Configuration

Field	Required	Default	Description
fields	Y		List of exact log entry field names the rules are applied to. Not glob patterns — each entry must match a field name exactly.
rules	Y		Ordered list of match-regex / replace-expression rules. Each rule's output feeds the next rule as input.
filter	N		Cribl-style JavaScript truthiness expression evaluated after if. A falsy result silently skips the processor. Distinct from if, which uses the native expression language.
tag	N		Identifier for this processor instance.
description	N		Explanatory note.
if	N		Condition that must be true for the processor to run.
disabled	N	false	When true, the processor is skipped.
ignore_failure	N	false	Continue pipeline processing if a rule's match_regex or replace_expr fails.
on_failure	N		Processors to run when this processor fails.
on_success	N		Processors to run when this processor passes an event through.

Rules

Field	Required	Default	Description
match_regex	Y		Pattern used to find matches in the field's string value. Either a bare pattern (e.g. (.*), applied with no flags) or a /pattern/flags delimited literal. Supported flags: g (replace every match, capped at 1000 replacements per rule/field pair — omit to replace only the first match), i, m, s, u.
replace_expr	Y		JavaScript expression evaluated once per match, producing the replacement text. Evaluated against a scratch environment exposing only the current match's capture groups (g0 for the whole match, g1..gN for numbered groups, and any named group additionally under its own name) — it cannot reference other fields of the event.

Details

rewrite is the native implementation of Cribl's Mask function. DataStream's Cribl-pipeline converter maps a Cribl mask block to this processor — not to DataStream's own Mask processor, which is a separate, unrelated field-redaction processor built around hashing. A pipeline migrated from Cribl's Mask function uses rewrite, never mask.

Each entry in fields is matched exactly against the event's field names; fields does not accept glob or wildcard patterns. A field that is absent from the event, or present with a non-string value, is silently left untouched — this is not treated as a processor failure.

rules is applied in declaration order to each field: the string produced by one rule becomes the input to the next rule. Within a single rule, match_regex locates substrings to replace and replace_expr computes the replacement text. Without the g flag, only the first match is replaced; with g, every match is replaced, up to an internal cap of 1000 replacements per rule/field pair.

replace_expr is compiled once per rule and evaluated separately for every match. Its scratch evaluation environment holds only that match's own capture groups — g0 (the whole match), g1 through gN (numbered groups, in JavaScript left-to-right order), and any named group additionally reachable under its own name. This is a documented v1 limitation: replace_expr cannot see any other field of the event — referencing an identifier that isn't one of the match's own groups evaluates as undefined. If replace_expr itself returns undefined, the matched text is deleted (replaced with an empty string).

replace_expr has access to the C.Mask expression library:

Function	Description
C.Mask.CC(value, unmasked, maskChar)	Masks a Luhn-valid value, leaving unmasked characters visible. Defaults: unmasked -4 (the rightmost four), maskChar "X". A value that is not Luhn-valid is returned unchanged.
C.Mask.IMEI(...), C.Mask.luhn(...)	Identical signature and behavior to C.Mask.CC.
C.Mask.md5(value, len), C.Mask.sha1, C.Mask.sha256, C.Mask.sha512, C.Mask.sha3_256, C.Mask.sha3_512	Hex digest of value. len omitted or 0 returns the full digest; a positive len returns its leftmost N characters, a negative len its rightmost N.
C.Mask.crc32(value)	CRC-32 checksum as eight hex characters.
C.Mask.repeat(len, char)	Repeats char (default "X") len times. A string passed as len contributes its own length, so C.Mask.repeat(g0, '*') masks a match to its original width.
C.Mask.random(len)	Random alphanumeric string of length len.
C.Mask.isCC(value), C.Mask.isIMEI(value)	Luhn validity check, returning a boolean.
C.Mask.REDACTED	The constant string REDACTED. Referenced without call syntax.

A failing match_regex (an invalid pattern, or a match timeout partway through a g-flagged rule's iteration) or a failing replace_expr (a compile or evaluation error) fails the processor with an error of the form rewrite: field "<field>": match_regex "<pattern>": <error> or rewrite: field "<field>": replace_expr "<expr>": <error>. On either error, the field's original value is left completely untouched — no partial or empty result is ever committed.

filter provides the same Cribl-style JavaScript truthiness gate available on other Cribl-compatible processors. It is evaluated after if; a falsy result skips the processor entirely without an error, leaving the event unmodified. This is distinct from if, which is evaluated using DataStream's native expression language — filter is not a universal field available on every processor.

Examples

Masking Credit Card Numbers

Masking a credit card number with C.Mask.CC, leaving the last four digits visible...	{ "cardNumber": "4111111111111111" } - rewrite: fields: ["cardNumber"] rules: - match_regex: "(.*)" replace_expr: "`${C.Mask.CC(g1)}`"
C.Mask.CC confirms the value is Luhn-valid, then masks every digit except the last four (the default unmasked: -4)...	{ "cardNumber": "XXXXXXXXXXXX1111" }

Redacting Repeated Digit Runs with the Global Flag

Replacing every digit run in a field with same-length asterisks using the g flag...	{ "message": "id=123 id=4567" } - rewrite: fields: ["message"] rules: - match_regex: "/\\d+/g" replace_expr: "C.Mask.repeat(g0, '*')"
Every match is replaced (not just the first); C.Mask.repeat reads its len from g0's own string length, so each run is masked to its original width...	{ "message": "id=*** id=****" }

Chaining Rules

Trimming whitespace in the first rule, then hashing the trimmed result in the second...	{ "user": " Alice " } - rewrite: fields: ["user"] rules: - match_regex: "(.*)" replace_expr: "`${g1.trim()}`" - match_regex: "(.*)" replace_expr: "`${C.Mask.sha256(g1, 8)}`"
The first rule's output ("Alice") becomes the second rule's input; the second rule replaces it with the first 8 hex characters of its SHA-256 digest...	{ "user": "3bc51062" }

Missing and Non-String Fields Skipped Silently

Listing a field that doesn't exist alongside a field that isn't a string...	{ "count": 42 } - rewrite: fields: ["note", "count"] rules: - match_regex: "/./g" replace_expr: "'X'"
Neither field is rewritten: note doesn't exist and count is a number, not a string — both are silently left alone rather than failing the processor...	{ "count": 42 }

Gating with the Cribl filter Field

Applying the rule only when a Cribl-style filter expression is truthy...	{ "value": "secret-data", "apply": true } - rewrite: filter: "apply" fields: ["value"] rules: - match_regex: "/.*/" replace_expr: "C.Mask.REDACTED"
filter evaluates truthy, so the rule runs and replaces the entire value with the C.Mask.REDACTED constant; had apply been falsy, the processor would have been skipped and value left untouched...	{ "value": "REDACTED", "apply": true }