Substring

Text Processing String Manipulation

Synopsis

Extracts substrings from string fields.

Schema

- substring:
    field: <ident>
    start: <integer>
    end: <integer>
    length: <integer>
    target_field: <string>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y	-	Source field containing string to extract from
start	N	0	Starting position (0-based index). Omitted, extraction starts at the beginning of the string
end	N	-	Ending position (exclusive, use with start)
length	N	-	Length of substring (use with start instead of end)
target_field	N	Same as field	Target field to store extracted substring
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	Continue processing if extraction fails
ignore_missing	N	false	Skip processing if referenced field doesn't exist
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier
disabled	N	false	When true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration

Details

Extracts a portion of a string based on a starting position and either an ending position or a length. Indexing is zero-based.

Specify the substring with either start + end (end exclusive) or start + length. When both are given, end wins and length is ignored.

note

The processor uses zero-based indexing where the first character is at position 0. When using end, it's exclusive (not included in the result). When using length, it specifies how many characters to extract.

If the range exceeds the string boundaries the available portion is returned: end is clamped to the length of the string.

warning

A start beyond the end of the string returns an empty string, as does an end before start. A negative start is clamped to 0.

warning

Positions are BYTE offsets, not character positions, and negative values do not count from the end.

substring.go measures with len(input) and slices with input[start:endPos], both of which work in bytes. On text outside ASCII the positions do not line up with characters, and a range can cut a multi-byte character in half and produce invalid UTF-8. A negative start is clamped to 0 rather than counting backwards from the end.

Use it on ASCII data — identifiers, codes, fixed-width fields. For character-accurate extraction on arbitrary text, use Regex Extract.

Examples

Basic Substring Extraction

Extracting characters 5-10 from a string...	{ "message": "Hello World DataStream" } - substring: field: message start: 6 end: 11 target_field: extracted
extracts "World":	{ "message": "Hello World DataStream", "extracted": "World" }

Length-based Extraction

Extracting 8 characters starting from position 12...	{ "log_entry": "2024-01-15 DataStream processing started" } - substring: field: log_entry start: 11 length: 10 target_field: component
extracts component name:	{ "log_entry": "2024-01-15 DataStream processing started", "component": "DataStream" }

Prefix Extraction

Extracting first 10 characters as prefix...	{ "transaction_id": "TXN-2024-01-15-14-30-45-ABC123" } - substring: field: transaction_id start: 0 length: 14 target_field: date_prefix
extracts date portion:	{ "transaction_id": "TXN-2024-01-15-14-30-45-ABC123", "date_prefix": "TXN-2024-01-15" }

In-place Trimming

Trimming string to specific length...	{ "long_description": "This is a very long description that needs to be shortened for display purposes" } - substring: field: long_description start: 0 length: 30
trims to 30 characters:	{ "long_description": "This is a very long descriptio" }

URL Path Extraction

Extracting path from URL...	{ "full_url": "https://api.example.com/v1/users/profile" } - substring: field: full_url start: 22 target_field: api_path
extracts API path:	{ "full_url": "https://api.example.com/v1/users/profile", "api_path": "/v1/users/profile" }