From Cribl

Synopsis

Cribl Stream exports a pipeline as YAML with a top-level functions: list, alongside description, groups, output, and asyncFuncTimeout. Conversion reads that export and returns a native DataStream pipeline — a name, a description, and an ordered list of processors.

The functions: array order is the execution order, and it is preserved. Each Cribl function becomes one DataStream processor, so the processor count matches the function count.

Do not hand-translate a Cribl pipeline. The converter handles the function-to-processor mapping, the JavaScript regex and expression dialect, capture-group numbering, and disabled-state handling — all of which are easy to get wrong by hand. Drive the conversion through an AI coding agent connected to the hosted MCP server, as described in Testing.

What Converts

Cribl function (id)	DataStream processor	Notes
comment	Comment	Documentation only.
eval	Eval	add[], keep, and remove are preserved, including their order.
mask	Rewrite	Not DataStream's mask processor. See the naming traps below.
regex_extract	Regex Extract with lang: js	ECMAScript /pattern/flags regex; the g flag selects scan-all over first-match-only.
serde	CSV, JSON, KV, Regex Extract, or Serialize	This is Cribl's Parser function. The target is selected by conf.mode and conf.type — see below.
serialize	Serialize with lang: js	Ordered glob field selection with ! negation. type is json or kvp.
auto_timestamp	Auto Timestamp	Ordered regex and strptime timestamp rules.
lookup	Lookup	file becomes lookup_file; inFields and outFields become lookup_fields and output_fields.
rename	Rename	rename[].currentName and newName become fields[].field and target_field.
regex_filter	Regex Filter	Both platforms drop events whose field matches the regex. The semantics align exactly.
grok	Grok	source becomes field; pattern and patterns[] are merged into patterns.
geoip	Geo IP	file becomes database_file; ipField becomes field; resultField becomes target_field.
dns_lookup	DNS Lookup	Always converts to type: forward. Reverse lookups must be switched by hand.
chain	Pipeline	The function's target pipeline becomes name.
dynamic_sampling	Dynamic Sample	sampleMode, sampleGroupKey, samplePeriodSec, minEvents, and maxSamplingRate map one-to-one.
suppress	Deduplicate	keyExpr becomes fields; suppressPeriodSec becomes ttl.
drop	Drop	Carries the JavaScript filter.
code	Script with lang: javascript	The Cribl Code body runs on the JavaScript engine.
guard	Redact, manually	Degrades to a comment. See below.

Two Naming Traps

Parser is serde. A Cribl function's display name can differ from the id in its conf.yml. The Parser function's id is serde. Convert the real configuration file, which uses the id, not the display name.

Cribl's Mask becomes rewrite, not mask. DataStream has its own Mask processor, which is an unrelated hash-based redaction processor. A pipeline migrated from Cribl's Mask function uses Rewrite.

How serde Resolves

conf.mode defaults to extract and conf.srcField defaults to _raw.

mode	type	Result
extract	json	JSON, with add_to_root: true
extract	kvp or kv	KV
extract	csv, delimited, or unset	CSV
extract	regex	Regex Extract with lang: js
reserialize	any	Serialize with lang: js

Any other extract type — including grok, elff, and clf — is a conversion error.

Conversion Semantics

Cribl filters become filter, never if. Each converted processor keeps its Cribl function's filter as a filter field, which is evaluated as a JavaScript truthiness expression. DataStream's if is a different, native expression language, and the converter never puts Cribl JavaScript there. The two coexist on the same processor: if is evaluated first, then filter.

Disabled steps are preserved, not dropped. A function with disabled: true, or one belonging to a group with disabled: true, converts to a processor with disabled: true. The imported pipeline is a faithful, editable copy, and you can enable those steps in DataStream.

final: true becomes a final processor. An enabled function marked final is followed by a Final processor that stops downstream processing.

Groups, output, and asyncFuncTimeout are dropped. Cribl groups are an organizational device in its interface, not execution semantics. Group-level disabled is the one group property that survives, as per-processor disabled.

The Cribl expression library is available. C.* helpers, __e field access, and the convention that undefined deletes a field all run on the JavaScript engine, so most eval and code functions port directly.

What Does Not Convert

A function with no native equivalent does not abort the conversion. If it is enabled, it becomes a Comment processor carrying the function's complete original configuration, and one warning line is added to the pipeline's description. If it is disabled, it is skipped silently, since it would not have run.

The comment's description takes this shape:

cribl <id> not supported — rewrite as native processors or handle manually. <guidance> Original: {"id": ..., "conf": ...}

Because the original configuration is preserved verbatim, no information is lost — the comment is a work item, not a dead end.

Specialized Functions

These have no native equivalent and always degrade to a comment:

Group	Functions
Metrics and aggregation	aggregations, aggregate_metrics, rollup_metrics, publish_metrics, drop_dimensions
OTLP and other serializers	otlp_logs, otlp_metrics, otlp_traces, snmp_trap_serialize, cef_serializer
Sampling, fan-out, and stateful	sampling, clone, flatten, unroll, json_unroll, xml_unroll, fold_keys, event_breaker, numerify, redis

Two of these are easy to mistake for equivalents that exist:

• sampling is a fixed 1:N rate. Dynamic Sample is adaptive, and is not the same thing.
• clone multiplies events in-stream. It is not Reroute, which redirects an event to a different destination.

guard is a special case: it maps conceptually to Redact, but its scanning rulesets are referenced by identifier and defined outside the pipeline, so the converter cannot resolve them. It degrades to a comment, and the rulesets must be rebuilt by hand as redact patterns.

Conversion Errors

Unlike the cases above, these abort the conversion. Each is a structural problem on an otherwise-supported function.

Condition	Reason
chain with no target pipeline	There is nothing to point the Pipeline processor at.
suppress with numAllowed greater than 1	Deduplicate keeps exactly one event per key.
suppress whose keyExpr has no simple field references	The key cannot be reduced to a field list.
regex_extract with fieldNameExpression	Not supported. The processor uses the regex's own named capture groups.
An unsupported serde extract type	Only json, kvp, csv, delimited, and regex are supported.
A non-trivial filter on a function whose target processor has no filter field	The gate cannot be preserved. Split the filter into a separate step before converting.

The processors that cannot carry a filter, and therefore reject a non-trivial one, are: CSV, JSON, KV, Lookup, Rename, Regex Filter, Grok, Geo IP, DNS Lookup, Pipeline, Dynamic Sample, and Deduplicate. A filter counts as trivial when it is empty or the literal true.

Known Limitations

serialize with type: cef. Under lang: js, Serialize executes json and kvp only. A cef type converts faithfully but fails if the processor is enabled and run.

regex_extract overwrite semantics differ. Cribl's overwrite: false, its default, protects a field that already exists. DataStream's Regex Extract instead appends to it as an array. The two agree when a pattern extracts into new fields, which is the common case, and diverge only when a pattern targets a field that existed before the processor ran.

dns_lookup always converts to a forward lookup. A Cribl reverse lookup must be changed to type: reverse after import.

Examples

Eval

A Cribl eval function adding a field and pruning another, gated by a filter...	functions: - id: eval filter: "sourcetype=='access'" conf: add: - name: severity value: "'high'" remove: - debug_*
The filter lands on filter, not if, and the add and remove lists carry over in order...	- eval: filter: "sourcetype=='access'" add: - field: severity value_expression: "'high'" remove: - debug_*

Mask

A Cribl mask function replacing a card number with its masked form...	functions: - id: mask conf: fields: - cardNumber rules: - matchRegex: "(.*)" replaceExpr: "`${C.Mask.CC(g1)}`"
It becomes a rewrite processor — not DataStream's mask processor. The replace expression and its capture-group references are preserved as written...	- rewrite: fields: - cardNumber rules: - match_regex: "(.*)" replace_expr: "`${C.Mask.CC(g1)}`"

Parser

A Cribl Parser function — note the id is serde — extracting JSON from the raw event...	functions: - id: serde conf: mode: extract type: json srcField: _raw
Extract mode with a JSON type resolves to the json processor, promoting the parsed object to the event root...	- json: field: _raw add_to_root: true

An Unsupported Function

An enabled aggregations function, which has no native equivalent...

functions: - id: aggregations conf: timeWindow: 10s

The conversion continues. The function becomes a comment carrying its original configuration, and a warning is added to the pipeline description naming the work still to be done...

description: | Warning [cribl aggregations]: not supported — rewrite as native processors or handle manually (no native DataStream equivalent — metrics/OTLP/aggregation/data-reduction/stateful/fan-out functions are out of scope; migrate manually) processors: - comment: description: >- cribl aggregations not supported — rewrite as native processors or handle manually. Original: {"id":"aggregations", "conf":{"timeWindow":"10s"}}