Skip to main content

CEF to CSL

Transform

Synopsis

Parses a CEF message and maps it directly to Microsoft Sentinel's Common Security Log (CommonSecurityLog) schema in one fused pass, replacing the manual cef + normalize + move (+ enforce_schema) processor chain.

note

For details of CEF, see Appendix.

Schema

- cef_to_csl:
field: <ident>
use_table: <boolean>
use_kv_parser: <boolean>
description: <text>
if: <script>
tag: <string>
disabled: <boolean>
ignore_failure: <boolean>
on_failure: <processor[]>
on_success: <processor[]>

Configuration

The following fields are used to define the processor:

FieldRequiredDefaultDescription
fieldYField containing the raw CEF message to convert. Must be a string starting with CEF:
use_tableNfalseWrite the mapped fields into the typed CommonSecurityLog virtual table instead of the event map. The table must already exist (create_table)
use_kv_parserNfalseParse the CEF extension field with the generic key-value parser instead of the built-in CEF-aware extension scanner
descriptionNExplanatory note
ifNCondition to run
tagNIdentifier
disabledNfalseSkip this processor
ignore_failureNfalseSee Handling Failures
on_failureNSee Handling Failures
on_successNSee Handling Success

Details

cef_to_csl parses the raw CEF message and hands the resulting fields straight to the internal cefcsl mapping in one pass, without ever materializing an intermediate cef object, a separate normalize step, or a move-to-root step. The result already reflects the same CSL schema enforcement that a manual cef + normalize + move + enforce_schema chain would produce—no separate enforce_schema step is needed afterward, and ignore_rules-style schema enforcement is not configurable on this processor; it always applies.

field must resolve to a string starting with CEF:. If the field is missing, is not a string, or does not start with CEF:, the processor fails. There is no ignore_missing option on this processor—unlike the plain cef parsing processor, a missing field is handled the same way as any other failure, through ignore_failure/on_failure.

use_table: false (default)

The mapped CSL fields replace the entire event map. Every field that existed on the event before this processor ran—including the source field itself—is discarded, except system (_vmetric) fields and any virtual tables already created on the entry.

use_table: true

Nothing is written to the event map. Instead, the mapped fields are written directly into the typed CommonSecurityLog virtual table, and the log entry's output source switches to that table. The table is not created by this processor—it must already exist via create_table; the caller owns the table's lifecycle (creation, truncation, dropping). The event map itself is left untouched. Field-for-field, the values written are identical to the use_table: false path—only the storage target differs.

use_kv_parser

Selects a generic key=value parser for the CEF extension field (the part of the message after the seven pipe-delimited header fields) instead of the built-in CEF-aware extension scanner. leef_to_csl has no equivalent option—LEEF's body is always parsed by a single dedicated LEEF tokenizer.

Unmapped fields

CEF extension fields with no corresponding CSL schema column are folded into AdditionalExtensions as key=value pairs joined with ;, in the order they appear in the source message.

Distinct from the cef processor

cef_to_csl is not the CEF parser. The cef processor parses a CEF message into a structured object at target_field (leaving the rest of the event untouched) and supports target_field/ignore_missing; cef_to_csl skips that intermediate object entirely and converts straight to CSL, replacing the event map (or writing to a table) and supporting neither target_field nor ignore_missing. For LEEF, see the sibling LEEF to CSL processor.

Examples

Basic Conversion

Converting a raw CEF message straight to CSL fields...

{
"message": "CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Detect|Address spoofing|Unknown|act=Detect deviceDirection=0 msg=Address spoofing rt=171436202000 spt=50740 dpt=53 ifname=eth1-01.7 loguid={0x663270c7,0x3f,0xf76511ac,0x1b933721} origin=172.17.11.241 originsicname=CN\\=user ,O\\=user.com sequencenum=679 version=5 dst=172.17.101.41 product=VPN-1 & FireWall-1 proto=17 src=192.168.10.9"
}
- cef_to_csl:
field: message

replaces the entire event map with the mapped CSL fields; unmapped extensions fold into AdditionalExtensions...

{
"Activity": "Address spoofing",
"AdditionalExtensions": "ifname=eth1-01.7;loguid={0x663270c7,0x3f,0xf76511ac,0x1b933721};origin=172.17.11.241;originsicname=CN\\=user,O\\=user.com;sequencenum=679;version=5;product=VPN-1 & FireWall-1",
"CommunicationDirection": "0",
"DestinationIP": "172.17.101.41",
"DestinationPort": 53,
"DeviceAction": "Detect",
"DeviceEventClassID": "Detect",
"DeviceProduct": "VPN-1 & FireWall-1",
"DeviceVendor": "Check Point",
"DeviceVersion": "Check Point",
"LogSeverity": "0",
"Message": "Address spoofing",
"Protocol": "17",
"ReceiptTime": "171436202000",
"SimplifiedDeviceAction": "Detect",
"SourceIP": "192.168.10.9",
"SourcePort": 50740
}

Writing into the CommonSecurityLog Table

Creating the typed table first, then converting with use_table: true...

{
"message": "CEF:0|Fortinet|Fortigate|v6.0.3|20503|utm:emailfilter smtp log-only|2|deviceExternalId=FGT5HD3915800610 FTNTFGTlogid=0508020503 cat=utm:emailfilter FTNTFGTsubtype=emailfilter FTNTFGTeventtype=smtp FTNTFGTlevel=information FTNTFGTvd=vdom1 FTNTFGTeventtime=1545939418 FTNTFGTpolicyid=1 externalId=1135 duser=bob src=10.1.100.11 spt=35969 deviceInboundInterface=port12 FTNTFGTsrcintfrole=undefined dst=172.18.62.158 dpt=25 deviceOutboundInterface=port11 FTNTFGTdstintfrole=undefined proto=6 app=SMTP FTNTFGTprofile=test-spam act=log-only suser=testpc1@qa.fortinet.com duser=test1@server88.qa.fortinet.com FTNTFGTsender=testpc1@qa.fortinet.com FTNTFGTrecipient=test1@server88.qa.fortinet.com deviceDirection=1 msg=general email log FTNTFGTsubject=hello_world2 FTNTFGTsize=216 FTNTFGTattachment=no"
}
- create_table:
name: CommonSecurityLog
- cef_to_csl:
field: message
use_table: true

writes the same mapped fields into the CommonSecurityLog table instead of the event map; the event map is left untouched and output now marshals from the table...

{
"Activity": "utm:emailfilter smtp log-only",
"AdditionalExtensions": "FTNTFGTlogid=0508020503;FTNTFGTsubtype=emailfilter;FTNTFGTeventtype=smtp;FTNTFGTlevel=information;FTNTFGTvd=vdom1;FTNTFGTeventtime=1545939418;FTNTFGTpolicyid=1;FTNTFGTsrcintfrole=undefined;FTNTFGTdstintfrole=undefined;FTNTFGTprofile=test-spam;FTNTFGTsender=testpc1@qa.fortinet.com;FTNTFGTrecipient=test1@server88.qa.fortinet.com;FTNTFGTsubject=hello_world2;FTNTFGTsize=216;FTNTFGTattachment=no",
"ApplicationProtocol": "SMTP",
"CommunicationDirection": "1",
"DestinationIP": "172.18.62.158",
"DestinationPort": 25,
"DestinationUserName": "test1@server88.qa.fortinet.com",
"DeviceAction": "log-only",
"DeviceEventCategory": "utm:emailfilter",
"DeviceEventClassID": "20503",
"DeviceExternalID": "FGT5HD3915800610",
"DeviceInboundInterface": "port12",
"DeviceOutboundInterface": "port11",
"DeviceProduct": "Fortigate",
"DeviceVendor": "Fortinet",
"DeviceVersion": "v6.0.3",
"ExtID": "1135",
"LogSeverity": "2",
"Message": "general email log",
"Protocol": "6",
"SimplifiedDeviceAction": "log-only",
"SourceIP": "10.1.100.11",
"SourcePort": 35969,
"SourceUserName": "testpc1@qa.fortinet.com"
}

Error Handling

Handling a field that isn't a valid CEF message...

{
"message": "Invalid CEF message"
}
- cef_to_csl:
field: message
ignore_failure: true
on_failure:
- append:
field: tags
value: cef_to_csl_error

leaves the event map untouched and adds an error tag instead of failing the pipeline...

{
"message": "Invalid CEF message",
"tags": ["cef_to_csl_error"]
}