CEF

The Common Event Format is a standardized security event logging layout created by ArcSight and widely adopted across the industry. A CEF record is a pipe-delimited header followed by an extension section of key=value pairs.

DataStream converts to and from CEF with the Normalize processor.

Message Format

A record consists of seven header fields and an extension section:

CEF:0|Device Vendor|Device Product|Device Version|Device Event Class ID|Name|Severity|key=value key=value ...

Position	Header Field	ECS Source	Written When Absent
1	Version	-	Always CEF:0
2	Device Vendor	observer.vendor	Unknown
3	Device Product	observer.product	Unknown
4	Device Version	observer.version	1.0
5	Device Event Class ID	event.code	0
6	Name	device.name	Unknown Event
7	Severity	event.severity	5

All seven header positions are always written—a missing value is filled with the default above rather than left empty.

The version is written as CEF:0. When reading, the parser accepts any integer in that position and preserves it, so records arriving with another version are not rejected.

Extension pairs are separated by a single space, sorted by key in ascending case-insensitive order.

Extension Fields

The tables below give the CEF key produced for each ECS field.

An ECS field not listed here is still written out. Its dotted path is condensed into a single camel-case key, so network.user becomes networkUser. No field is dropped for being unmapped.

Where two keys are shown, the first is the one written. Both are recognized when reading, which is how alternative vendor spellings are accepted.

Timestamp

ECS Field	CEF Field
@timestamp	receivedTime

Device

ECS Field	CEF Field
device.name	Name
device.host	Host
device.request	Request
device.os	deviceOs
device.osversion	deviceOsVersion
device.zone	deviceZone

Observer

ECS Field	CEF Field
observer.vendor	DeviceVendor
observer.product	DeviceProduct
observer.version	DeviceVersion
observer.name	deviceExternalId
observer.type	deviceIdType
observer.hostname	dvchost
observer.ip	dvc
observer.mac	dvcmac
observer.scope	scope
observer.scope.id	scopeId
observer.domain	deviceDnsDomain
observer.facility	deviceFacility
observer.ntdomain	deviceNtDomain
observer.payload	devicePayloadId
observer.process.id	dvcpid
observer.process.name	deviceProcessName
observer.nat.address	deviceTranslatedAddress
observer.timezone	dtz

Event

ECS Field	CEF Field
event.id	eventId
event.original_uid	eventOriginalUid
event.extid	externalId
event.code	DeviceEventClassID
event.count	cnt
event.action	act
event.severity	Severity
event.outcome	outcome
event.reason	reason
event.created	rt
event.start	start
event.end	end
event.timezone	tz
event.type	type
event.module	module
event.category	cat
event.kind	kind
event.duration	duration
event.hash	eventHash
event.original	rawEvent
event.risk_score	agentSeverity
event.dataset	dataset
event.schema	schema
event.schema_version	schemaVersion
event.owner	owner

Source

ECS Field	CEF Field
source.ip	src
source.port	spt
source.hostname	shost
source.service.name	sourceServiceName
source.nat.ip	sourceTranslatedAddress
source.nat.port	sourceTranslatedPort
source.nat.interface	sourceTranslatedInterface
source.domain	sourceDnsDomain
source.interface	deviceInboundInterface
source.mac	smac
source.user.name	suser
source.user.id	suid, uid
source.process.name	sproc
source.process.id	spid
source.bytes	in
source.packets	sourceTranslatedPackets
source.geo.country	sntry
source.geo.city	sourceGeoCity
source.geo.region	sourceGeoRegion
source.geo.location	sourceGeoLatitude, sourceGeoLongitude
source.user.email	smail
source.user.group	sntdom
source.user.roles	spriv

Destination

ECS Field	CEF Field
destination.name	dstName
destination.ip	dst
destination.port	dpt
destination.service.name	destinationServiceName
destination.nat.ip	destinationTranslatedAddress
destination.nat.port	destinationTranslatedPort
destination.nat.interface	destinationTranslatedInterface
destination.domain	destinationDnsDomain
destination.domain_type	destinationDnsDomainType
destination.interface	deviceOutboundInterface
destination.host	dhostName
destination.hostname	dhost
destination.fqdn	dhostFQDN
destination.mac	dmac
destination.user.name	duser
destination.user.id	duid
destination.process.name	dproc
destination.process.id	dpid
destination.bytes	out
destination.packets	destinationTranslatedPackets
destination.geo.country	dntry
destination.geo.city	destinationGeoCity
destination.geo.region	destinationGeoRegion
destination.geo.location	destinationGeoLatitude, destinationGeoLongitude
destination.user.email	dmail
destination.user.group	dntdom
destination.user.roles	dpriv

Network

ECS Field	CEF Field
network.protocol	app
network.transport	proto
network.type	deviceType
network.application	appName
network.bytes	total
network.packets	pa
network.direction	deviceDirection
network.community_id	deviceCommunityId
network.forwarded	deviceForwardingStatus
network.vlan	deviceVlan

File

ECS Field	CEF Field
file.id	fileId
file.path	filePath
file.type	fileType
file.name	fname
file.size	fsize
file.hash	fileHash
file.created	fileCreateTime
file.mtime	fileModificationTime
file.extension	fileExtension
file.mime_type	fileMimeType
file.directory	fileDirectory
file.hash.md5	fileHash
file.hash.sha1	fileHash
file.hash.sha256	fileHash
file.hash.sha512	fileHash

Old File

ECS Field	CEF Field
old_file.id	oldFileId
old_file.path	oldFilePath
old_file.type	oldFileType
old_file.name	oldFileName
old_file.size	oldFileSize
old_file.hash	oldFileHash
old_file.created	oldFileCreateTime
old_file.mtime	oldFileModificationTime
old_file.extension	oldFileExtension
old_file.mime_type	oldFileMimeType
old_file.directory	oldFileDirectory
old_file.hash.md5	oldFileHash
old_file.hash.sha1	oldFileHash
old_file.hash.sha256	oldFileHash
old_file.hash.sha512	oldFileHash

HTTP and URL

ECS Field	CEF Field
http.request.method	requestMethod
http.response.status_code	responseStatus
http.request.body.content	requestBody
http.response.body.content	responseBody
http.request.referrer	requestContext
http.request.cookies	requestCookies
url.original	requestOriginal
url.category	requestCategory
url.domain	requestClientApplication
url.full	request

Process

ECS Field	CEF Field
process.name	processName
process.pid	processId
process.args	processCommandLine
process.title	processTitle
process.executable	processExecutable
process.working_directory	processPath
process.hash.md5	processHash
process.hash.sha1	processHash
process.hash.sha256	processHash
process.parent.name	processParentName
process.parent.pid	processParentId
process.parent.args	processParentCommandLine

User

ECS Field	CEF Field
user.type	uType
user.id	userId
user.name	user
user.user_type	userType
user.domain	ntdom
user.email	mail
user.group.id	gid
user.group.name	group

User Agent

ECS Field	CEF Field
user_agent.name	HttpUserAgent
user_agent.version	HttpUserAgentVersion

Threat Intelligence

ECS Field	CEF Field
threat.framework	threatFramework
threat.tactic.name	threatDescription
threat.indicator.ip	threatIp
threat.indicator.type	threatType

Registry

ECS Field	CEF Field
registry.key	registryKey
registry.value	registryValue
registry.data	registryData

DNS

ECS Field	CEF Field
dns.answers.class	answerClass
dns.answers.ttl	answerTTL
dns.answers.type	answerType
dns.header_flags	dnsFlags
dns.id	dnsID
dns.op_code	opCode
dns.question.class	queryClass
dns.question.name	queryName
dns.question.registered_domain	registeredDomain
dns.question.subdomain	subdomain
dns.question.top_level_domain	topLevelDomain
dns.question.type	queryType
dns.resolved_ip	resolvedIP
dns.response_code	responseCode
dns.type	dnsType

Rule

ECS Field	CEF Field
rule.author	ruleAuthor
rule.category	ruleCategory
rule.description	ruleDescription
rule.id	ruleID
rule.license	ruleLicense
rule.name	ruleName
rule.reference	ruleReference
rule.ruleset	ruleSet
rule.uuid	ruleUUID
rule.version	ruleVersion

Message

ECS Field	CEF Field
message	msg

Syslog

ECS Field	CEF Field
log.syslog.appname	syslogAppName
log.syslog.facility.code	syslogFacilityCode
log.syslog.facility.name	syslogFacilityName
log.syslog.hostname	syslogHostname
log.syslog.priority	syslogPriority
log.syslog.procid	syslogProcessId
log.syslog.severity.code	syslogSeverityCode
log.syslog.severity.name	syslogSeverityName
log.syslog.structured_data	syslogMsg

Format Identification

ECS Field	CEF Field
leef.version	leefVersion
leef.name	leefName
cef.version	CefVersion
cef.name	CefName

Custom Fields

ECS Field	CEF Field
cef.device_custom_string_1.value	cs1
cef.device_custom_string_1.label	cs1Label
cef.device_custom_string_2.value	cs2
cef.device_custom_string_2.label	cs2Label
cef.device_custom_string_3.value	cs3
cef.device_custom_string_3.label	cs3Label
cef.device_custom_string_4.value	cs4
cef.device_custom_string_4.label	cs4Label
cef.device_custom_string_5.value	cs5
cef.device_custom_string_5.label	cs5Label
cef.device_custom_string_6.value	cs6
cef.device_custom_string_6.label	cs6Label
cef.device_custom_number_1.value	cn1
cef.device_custom_number_1.label	cn1Label
cef.device_custom_number_2.value	cn2
cef.device_custom_number_2.label	cn2Label
cef.device_custom_number_3.value	cn3
cef.device_custom_number_3.label	cn3Label
cef.device_custom_ipv6_address_1.value	c6a1
cef.device_custom_ipv6_address_1.label	c6a1Label
cef.device_custom_ipv6_address_2.value	c6a2
cef.device_custom_ipv6_address_2.label	c6a2Label
cef.device_custom_ipv6_address_3.value	c6a3
cef.device_custom_ipv6_address_3.label	c6a3Label
cef.device_custom_ipv6_address_4.value	c6a4
cef.device_custom_ipv6_address_4.label	c6a4Label
cef.device_custom_floating_point_1.value	cfp1
cef.device_custom_floating_point_1.label	cfp1Label
cef.device_custom_floating_point_2.value	cfp2
cef.device_custom_floating_point_2.label	cfp2Label
cef.device_custom_floating_point_3.value	cfp3
cef.device_custom_floating_point_3.label	cfp3Label
cef.device_custom_floating_point_4.value	cfp4
cef.device_custom_floating_point_4.label	cfp4Label
cef.flex_string_1	flexString1
cef.flex_string_1_label	flexString1Label
cef.flex_string_2	flexString2
cef.flex_string_2_label	flexString2Label
cef.flex_date_1	flexDate1
cef.flex_date_1_label	flexDate1Label
cef.flex_number_1	flexNumber1
cef.flex_number_1_label	flexNumber1Label
cef.flex_number_2	flexNumber2
cef.flex_number_2_label	flexNumber2Label
cef.device_custom_date_1.value	deviceCustomDate1
cef.device_custom_date_1.label	deviceCustomDate1Label
cef.device_custom_date_2.value	deviceCustomDate2
cef.device_custom_date_2.label	deviceCustomDate2Label

Additional

ECS Field	CEF Field
additional.extensions	additionalExtensions

Severity

The Severity header carries the value of event.severity unchanged, defaulting to 5. DataStream applies no numeric scale of its own and validates no range, so a source using names rather than numbers produces names in the header.

The Microsoft Sentinel content pack behaves differently: it moves severity into cs1 with cs1Label=Severity and removes the sev key entirely.

Limitations

• Values are not escaped on write. A value containing |, = or ; is written literally, which can make the record split differently when it is read back. The parser does unescape \| and \=, so writing and reading are not symmetric.
• Several ECS fields share one CEF key. file.hash, file.hash.md5, file.hash.sha1, file.hash.sha256 and file.hash.sha512 all produce fileHash; processHash and oldFileHash behave the same way. Converting back yields the base field, so the specific hash algorithm is not recoverable.
• Geographic coordinates lose longitude. source.geo.location and destination.geo.location list a latitude and a longitude key, and only the first is written.