From Logstash

Synopsis

Logstash pipelines are written as a .conf file with input, filter, and output sections. Conversion reads that file and returns a native DataStream pipeline.

Only the filter section becomes processors. The input and output sections are preserved as comment processors summarizing what they contained, because DataStream configures its sources and destinations separately — as Devices and Targets, wired together by Routes — rather than inside the pipeline.

Do not hand-translate a Logstash config. Drive the conversion through an AI coding agent connected to the hosted MCP server, as described in Testing.

What Converts

Filter Plugins

Logstash plugin	DataStream processor	Notes
`grok`	Grok	One processor per source field. Adds an implicit `_grokparsefailure` tag on failure.
`dissect`	Dissect	One per `mapping` entry, plus a Convert per `convert_datatype` entry.
`date`	Date	Joda formats are normalized. `UNIX`, `UNIX_MS`, `ISO8601`, and `TAI64N` pass through.
`kv`	KV
`csv`	CSV	`columns` becomes `target_fields`.
`json`	JSON	With no `target`, the parsed object is promoted to the event root.
`xml`	XML
`geoip`	Geo IP	`database` becomes `database_file`; the file must exist on the DataStream side.
`useragent`	User Agent	See the limitations — the output shape differs.
`dns`	DNS Lookup	One per field. `reverse` and `resolve` select the lookup type.
`translate`	Lookup	An inline `dictionary` becomes inline `lookup_content`. A `dictionary_path` does not convert.
`syslog_pri`	Math, Convert, Lookup	Expands into a sequence that derives the severity and facility codes, then labels them.
`fingerprint`	Fingerprint	See the limitations — hashes will differ.
`urldecode`	URL Decode	`all_fields` is not supported.
`prune`	Keep, Remove	`whitelist_names` becomes `keep`; `blacklist_names` becomes `remove`. Regex or value-based pruning does not convert.
`drop`	Drop	An enclosing conditional becomes its `if`.
`mutate`	Varies	See the sub-operation table below.
`ruby`	Varies	Common shapes are lowered to native processors. Anything else becomes a comment.

metrics and the elasticsearch filter have no native equivalent and always become comments.

`mutate` Sub-Operations

Sub-operation	DataStream processor	Notes
`coerce`	Set	Gated on the field being absent.
`rename`	Rename	All pairs land in one processor.
`update`	Set	Gated on the field being present.
`replace`	Set	Ungated.
`convert`	Convert	See the limitations for the `_eu` types.
`gsub`	Gsub	Takes field, pattern, and replacement in triples.
`uppercase`	Uppercase
`capitalize`	Capitalize
`lowercase`	Lowercase
`strip`	Trim
`split`	Split
`join`	Join	Written back in place.
`merge`	Append	An approximation. See the limitations.
`copy`	Set	Uses `copy_from` with `deep_copy`.

These are applied in Logstash's fixed runtime order — the order above — rather than the order they appear in your config. That is what Logstash itself does, so a converted pipeline behaves the same way.

remove_tag has no native equivalent and is dropped, with a warning.

Common Options

Options that any plugin may carry:

Logstash option	Becomes
`add_field`	Set
`add_tag`	Append to `tags`
`remove_field`	Remove
`id`	The processor's `tag`
`tag_on_failure`, `tag_on_exception`	An `on_failure` chain that appends the tag and recovers

Parsing plugins carry an implicit failure tag even when you did not ask for one: _grokparsefailure, _dateparsefailure, _kv_filter_error, _csvparsefailure, _jsonparsefailure, _xmlparsefailure, _geoip_lookup_failure, and _dissectfailure. Setting tag_on_failure => [] suppresses tagging and ignores the failure instead.

Conditionals

A conditional wrapping a single processor becomes that processor's if. A conditional wrapping several becomes a Group with an if, so the condition is evaluated once — matching Logstash's behavior.

else if and else chains are supported, as is nesting. Field references are rewritten from Logstash's bracket syntax to dotted paths, so [host][name] becomes host.name.

What Does Not Convert

These degrade to a Comment carrying the original configuration, with a warning added to the pipeline's description. The conversion continues.

metrics, and the elasticsearch filter
translate with a dictionary_path — the dictionary lives on the Logstash host, so inline it as lookup_content or upload it and set lookup_file
ruby beyond the shapes listed below
Any plugin the converter does not recognize

A ruby filter is lowered to native processors when every statement in the block is a recognized shape — arithmetic, JSON.parse, gsub, type coercion, case conversion, a tags push, a plain event.get copy, or a nil-sweep. If even one statement is unrecognized, the whole block becomes a comment. A script processor is never generated from Ruby.

Known Limitations

in is decided by field name, not by type. Logstash's in operator means membership for arrays and substring for strings. The converter cannot know a field's runtime type, so it treats a field named tags as an array and every other field as a string, rendering contains(). An in test against your own array field will not behave as it did in Logstash.

translate's fallback value is dropped silently, with no warning. If your dictionary relied on a fallback, add it back by hand.

convert => integer_eu and float_eu fold to plain integer and float, silently. European decimal-comma parsing is lost.

fingerprint hashes will differ. Logstash uses key as an HMAC key; the native processor uses it as a salt.

useragent output has a different shape. Logstash writes flat prefixed fields; the native processor writes an object. Downstream references need updating.

mutate merge is approximated with an append. Logstash merges arrays and hashes element-wise. Review the result.

A condition that cannot be rendered leaves its processors ungated. They still run, and a warning is emitted — but they run for every event. Read the warnings.

[@metadata] fields become ordinary event fields. DataStream has no ephemeral-metadata concept, so add a trailing Remove if they must not reach the target.

Environment references are kept literally. A ${VAR} in the source is not resolved; substitute it by hand.

Conversion Errors

Only a malformed source file aborts the conversion: an unbalanced brace, an unbalanced bracket in an array value, or end-of-file reached while reading a value. Everything else degrades.

Examples

A Filter Chain

A typical parse, rename, and timestamp chain...

filter {
  grok {
    match => { "message" => "%{IP:client_ip}" }
  }
  mutate {
    rename => { "client_ip" => "source.ip" }
  }
  date {
    match => [ "timestamp", "ISO8601" ]
  }
}

Each plugin becomes its native counterpart, and the parsing plugins pick up their implicit failure tags...

- grok:
    field: message
    patterns:
      - "%{IP:client_ip}"
- rename:
    fields:
      - field: client_ip
        target_field: source.ip
- date:
    field: timestamp
    formats:
      - ISO8601

A Conditional

A conditional wrapping more than one plugin...

filter {
  if [type] == "apache" {
    mutate { replace => { "service" => "web" } }
    mutate { add_tag => [ "parsed" ] }
  }
}

The body becomes a group carrying the condition, so it is evaluated once for the whole block rather than per processor...

- group:
    name: logstash_if_3_3
    description: "logstash conditional: if [type] == \"apache\""
    if: "type == 'apache'"
    processors:
      - set:
          field: service
          value: web
      - append:
          field: tags
          value: parsed
          allow_duplicates: false

An Unsupported Plugin

A metrics filter, which has no per-event equivalent...

filter {
  metrics {
    meter => "events"
  }
}

The conversion continues. The plugin becomes a comment, and a warning names it in the pipeline description...

description: |
  Converted from Logstash pipeline.
  Warning [3:3 metrics]: metrics (stateful rate metering) has no native
  equivalent — dropped; recreate as a DataStream metric/monitoring
  feature if needed
processors:
  - comment:
      description: >-
        WARN [logstash metrics 3:3]: metrics (stateful rate metering)
        has no native equivalent — dropped; recreate as a DataStream
        metric/monitoring feature if needed

Synopsis​

What Converts​

Filter Plugins​

mutate Sub-Operations​

Common Options​

Conditionals​

What Does Not Convert​

Known Limitations​

Conversion Errors​

Examples​

A Filter Chain​

A Conditional​

An Unsupported Plugin​