Migration: Overview

DataStream converts pipelines written for another telemetry platform into native DataStream pipelines. Conversion is available on two routes — the web interface and VirtualMetric's hosted MCP server — and both call the same converters, so a given source definition produces the same pipeline either way.

Conversion produces a native pipeline: name, description, and an ordered list of processors. There are no vendor-specific processors in the result, and no compatibility layer at runtime. A converted pipeline is an ordinary DataStream pipeline that you can read, edit, and run like any other.

Conversion Routes

Route	Where	Use when
Web interface	Content Management > Pipelines > Create new pipeline, then one of the import creation methods	Converting a pipeline you want created and ready to edit in one pass
Hosted MCP server	An AI coding agent connected to the server	Iterating on the result, or verifying it against sample data before anything is created

The web interface converts and creates in one pass: the converted YAML is presented for review, and completing the wizard stores it as a pipeline. The wizard, its accepted file types and its 5 MB source limit are covered in Management. The same import methods are offered when adding a child pipeline.

The MCP server returns the converted YAML to the agent instead, which can run sample events through it and compare the output against a fixture before anything is created — nothing is written to your installation until you install the result yourself. See Connecting an Agent.

Available Paths

Source platform	Guide
Cribl Stream	From Cribl
Azure Sentinel ASIM parsers, and KQL queries	From KQL
Logstash	From Logstash

The KQL path also runs in reverse: a native DataStream pipeline can be exported back to KQL. Export is available through the MCP server only — the web interface has no equivalent route. See From KQL.

Workflow

Migration is a conversion followed by a verification. The conversion is mechanical; the verification is where you confirm the converted pipeline behaves the way the original did.

1. Convert. Supply the exported source pipeline — pasted or uploaded into the create wizard, or handed to the agent. Both return a DataStream pipeline in YAML.

2. Read the warnings. A source function with no native equivalent does not abort the conversion — it is preserved as a Comment processor carrying its original configuration, and a warning line is added to the pipeline's description. These warnings are the list of things still to do by hand. In the wizard they are visible in the converted pipeline preview on the review step, before the pipeline is created.

3. Test. Run a representative event through the converted pipeline — with the Pipeline Debugger in the web interface, or the test_pipeline tool through an agent. Use the same sample data you tested with on the source platform, so the two outputs can be compared directly.

4. Validate. If you have an expected output, confirm parity with validate_pipeline (pass/fail) or diff_pipeline (a unified diff of expected against actual). Both run through the MCP server; the web interface has no automated parity check.

What Conversion Does Not Do

Conversion translates pipeline logic only. Sources, destinations, and routing are not part of a pipeline export and are not converted — configure them as Devices, Targets, and Routes.

A converted pipeline is a faithful copy of the original, not an optimized one. It reproduces the source pipeline's structure, including any steps that were disabled at the time of export.