From KQL

Synopsis

Azure Sentinel ships its normalization logic as ASIM parsers — a KQL function, wrapped in a YAML file that carries it under a ParserQuery or Query field. Conversion reads that export, or a raw KQL reduction query of the form SourceTable | where ... | extend ... | project ..., and returns a native DataStream pipeline.

You can supply either the full ASIM parser YAML export, in which case the query and the parser name are extracted automatically, or the KQL text on its own.

Do not hand-translate a KQL query. Run the conversion from the create wizard's Import from KQL method, described in Management, or through an AI coding agent connected to the hosted MCP server, described in Connecting an Agent.

This Is Not Federated Search

DataStream uses KQL in two unrelated ways, and it is worth being precise about which one this page describes.

Federated Search runs KQL as a query language. You write a query, and DataStream executes it against your data at query time.

Conversion, described here, treats KQL as a source format. The query is translated once, at import, into a DataStream pipeline — and then it is gone. Nothing about KQL survives into pipeline execution; the result is ordinary YAML that runs like any other pipeline.

The two are complementary rather than overlapping. The operators conversion cannot express — summarize, join, sort, and the rest of the aggregation family — are precisely the ones Federated Search exists to run, because they describe work across a set of events rather than work on a single event.

What Converts

Pipe Operators

KQL operator	DataStream processor	Notes
where	Drop	The condition is negated: events that fail it are dropped. An ASIM where not(disabled) guard is recognized and skipped.
extend	Varies	Each assignment is lowered by its value expression. See Scalar Functions.
project	Keep	Computed columns are emitted first, then the field list is kept.
project-keep	Keep
project-away	Remove	A suffix wildcard that matches no known field is kept verbatim, with a warning.
project-rename	Rename
project-reorder	None	A no-op, with a warning. Field order is not significant in a per-event pipeline.
parse	Dissect or Grok	Dissect for a pure delimiter pattern, Grok when the captures are typed.
parse-kv	KV
lookup	Lookup	Each referenced datatable is registered as inline lookup_content. An unreferenced datatable is pruned.
union	Group	Each branch becomes a group with its own condition. An event matching no branch is dropped, as in KQL.
invoke	Varies	See ASIM Helpers.

Scalar Functions

KQL function	DataStream processor
Literals, bare field references, strcat, strcat_delim, concat, column_ifexists	Set
coalesce	Coalesce
case, iif, iff	Case
Arithmetic (+, -, *, /, %)	Math
tostring, toint, tolong, todouble, toreal, todecimal, tobool, toboolean, todatetime, and the int(), long(), real(), double() literal casts	Convert
unixtime_seconds_todatetime, unixtime_milliseconds_todatetime	Date
toupper	Uppercase
tolower	Lowercase
trim, trim_start, trim_end	Trim
replace_string, replace_regex	Gsub
extract	Grok
split	Split
strcat_array	Join
array_slice	Slice
Array indexing, such as arr[0]	Select
substring	Substring
todynamic, toobject, parse_json	JSON
bag_pack, pack, pack_dictionary	Bag Pack
base64_encode_tostring, encode_base64, base64_encodestring	Base64 Encode
base64_decode_tostring, decode_base64, base64_decodestring	Base64 Decode
parse_url, parse_urlquery	URI Parts
url_decode	URL Decode
parse_ipv4, parse_ipv6, ipv4_is_private	IP Type
hash_sha256, hash_md5, hash_sha1	Fingerprint
parse_user_agent, user_agent	User Agent
The totimespan(F)/time(1s) duration-ratio idiom	Duration

Any other function becomes a Comment carrying the original KQL expression, with a warning.

ASIM Helpers

KQL construct	DataStream processor
invoke _ASIM_ResolveNetworkProtocol(...)	Network Protocol
invoke _ASIM_ResolveDvcFQDN(...) and its Src, Dst, and Target variants	FQDN
_ASIM_GetUsernameType(...)	Username Type

A KQL function defined in the query and then invoked — the ASIM ItemParser idiom — is inlined directly into the pipeline. Any other invoked helper becomes a comment, with a warning.

A literal EventSchema assignment is recognized: the converter prepends a Set writing the corresponding ASIM table name to _vmetric.table.

Conditions

Conditions become DataStream's native if expression language.

KQL	Native if
and, or	&&, ||
==, !=, <, <=, >, >=	Unchanged
contains, contains_cs, icontains	contains()
startswith, hasprefix (and their _cs forms)	startsWith()
endswith, hassuffix (and their _cs forms)	endsWith()
matchesregex	matches()
in, !in	Membership test
isempty, isnotempty, isnull, isnotnull, not()	Equivalent native forms
=~, !~	A case-insensitive comparison, exactly equivalent
has, has_cs, has_any, has_all and their negations	matches() against a word-boundary regex — an approximation. See the limitations below.

Set Operators

These operators describe work across a set of events, which a per-event pipeline cannot do in general. Four of them nonetheless convert in the shapes that happen to be per-event:

Operator	Converts?
summarize	Only the arg_min / arg_max dedup idiom, approximated as keep-first-per-key
distinct	Yes, approximated the same way — keep the first event per key
join	Only a reference join, where the right side is a registered datatable; it lowers onto a lookup. A stream-to-stream join does not convert
mv-apply	Only the reduce forms, where the on-clause ends in a collapsing summarize so the row count is unchanged; a genuine fan-out does not convert
sort, order, top, take, limit	No
mv-expand	No — one event cannot expand into many

Nothing here aborts the conversion. An operator that does not convert becomes a Comment carrying the original KQL, a warning is added to the pipeline's description, and the remaining stages still convert. What you get back is a partial pipeline that you finish by hand — not an error.

For work that genuinely needs aggregation or joins across events, use Federated Search, which runs KQL as a query rather than converting it.

Known Limitations

A successful conversion can be silently incomplete. The KQL parser recovers from syntax it cannot handle rather than failing outright, so an unparseable construct yields a partial pipeline plus a warning — not an error. Always read the warnings in the converted pipeline's description before trusting the result.

union matches KQL's omit semantics. An event that matches no union branch is dropped rather than continuing into the shared tail as a partly-populated row — the converter turns this on for every conversion, so no option is needed.

The underlying library defaults the other way, but that default governs the detection-rule compiler, not this converter.

Hash values change. hash_sha256, hash_md5, and hash_sha1 become a Fingerprint, which produces a deterministic digest — but not the same value KQL produces. Do not rely on migrated hashes matching historical ones.

IP parsing is approximated. parse_ipv4, parse_ipv6, and ipv4_is_private become an IP Type, which is close but not identical.

The has family is approximated. It lowers to a case-insensitive word-boundary regex. KQL's exact term tokenization is not reproduced.

Watchlist-backed let bindings cannot be converted. An expression referencing one is skipped entirely, with a warning.

Detection rules are out of scope. This converter is for normalization parsers and reduction queries. A Sentinel analytics rule is not converted into a pipeline — it is registered, unchanged, with the Smart Engine processor, which uses it to decide what to forward to the SIEM.

Exporting a Pipeline Back to KQL

The conversion also runs in reverse: a native DataStream pipeline can be turned back into KQL. This is useful when you want to hand a pipeline's normalization logic to a team that works in KQL, or produce an ASIM parser for an Azure Sentinel target from a pipeline you built or migrated here.

Export runs through the hosted MCP server only — drive it with the convert_pipeline_to_kql tool from an AI coding agent, as described in Workflows. Unlike the import direction, it has no equivalent in the web interface.

By default the tool emits a bare ASIM ParserQuery. Request the full ASIM parser YAML envelope when you need a drop-in parser file rather than the query on its own.

The goal of the export is runtime equivalence, not textual fidelity: a pipeline exported to KQL and then re-imported produces the same per-event result. The generated text will not match any original hand-written query.

What Does Not Export

Some processors have no faithful KQL form. Each becomes an // UNSUPPORTED comment carrying the original stage, so nothing is dropped silently — you review and finish those by hand:

• Branching and routing — Reroute, because fan-out has no single-query form.
• Stateful processors — caching, deduplication, sampling and suppression, and table creation all depend on state carried across events.
• External I/O — DNS Lookup, threat-intelligence, and AI/LLM processors call out to services at runtime.
• Cross-file pipeline calls — export covers a single pipeline; a call graph spanning multiple pipeline files is not followed.
• Mutating foreach bodies and break / return / on_failure control flow.

A few conversions are lossy by nature and do not round-trip exactly — for example a Fingerprint does not map back to a specific hash_* function.

Examples

A Reduction Query

A KQL query filtering, computing a field, and narrowing the output columns...	NetworkSession | where SrcIpAddr != "" | extend EventVendor = "Fortinet" | project TimeGenerated, SrcIpAddr, EventVendor
where becomes a negated drop, extend becomes a set, and project becomes a keep...	- drop: if: "!(SrcIpAddr != '')" - set: field: EventVendor value: Fortinet - keep: fields: - TimeGenerated - SrcIpAddr - EventVendor

An ASIM Helper

An ASIM parser resolving a network protocol through the standard helper...	| invoke _ASIM_ResolveNetworkProtocol('NetworkProtocol')
The helper has a native equivalent, so it converts directly...	- network_protocol: field: NetworkProtocol

An Aggregation

A summarize, which has no per-event equivalent...	| summarize count() by SrcIpAddr
The conversion continues. The operator is preserved as a comment carrying the original KQL, and a warning names it in the pipeline description...	description: | Converted from KQL (source table: NetworkSession). Warning [summarize]: KQL stage skipped (could not be converted to a native processor — manual migration required): operator "summarize" dropped — not representable per-event; use the KQL->SQL federated-search path processors: - comment: message: >- unconverted KQL stage (manual migration required): |summarizecount()bySrcIpAddr