Checksum

Arithmetic Cryptography

Synopsis

Calculates cryptographic and non-cryptographic checksums of field values.

Schema

- checksum:
    field: <ident>
    algorithm: <string>
    target_field: <string>
    output_format: <string>
    include_fields: <array>
    exclude_fields: <array>
    uppercase: <boolean>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y	-	Source field containing data to checksum
algorithm	Y	-	Checksum algorithm (crc32, crc64, adler32, fnv32, fnv64, fnv128, md5, sha1, sha256, sha512)
target_field	N	{field}_{algorithm}	Target field to store checksum result
output_format	N	hex	hex, decimal or base64. base64 only takes effect for the 32-bit checksums — crc32, crc32c, crc32k and adler32. Every other algorithm returns hex regardless
include_fields	N	-	Array of fields to include for object checksums
exclude_fields	N	-	Array of fields to exclude from object checksums
uppercase	N	false	Output checksum in uppercase format
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	Continue processing if checksum calculation fails
ignore_missing	N	false	Skip processing if referenced field doesn't exist
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier
disabled	N	false	When true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration

Details

Calculates checksums using various algorithms ranging from simple CRC checksums to cryptographic hashes like SHA-256. The processor supports different data types including strings, byte arrays, objects, and arrays.

The processor handles multiple algorithm types including non-cryptographic checksums (CRC32, CRC64, Adler32, FNV variants) and cryptographic hashes (MD5, SHA-1, SHA-2 family). For object checksums, it can selectively include or exclude specific fields.

note

Object fields are formatted as field:value pairs joined with pipes (|). Alphabetical ordering applies only when include_fields is not set — with include_fields, fields are taken in the order you list them, so two configurations listing the same fields in a different order produce different checksums.

Output is hexadecimal by default.

decimal falls back to hexadecimal for any hash wider than 64 bits — the MD5, SHA-1, SHA-2, SHA-3 and FNV-128 algorithms — because the value cannot be held in a 64-bit integer.

base64 is narrower still: it is honoured only by the 32-bit checksums (crc32, crc32c, crc32k, adler32). For the 64-bit CRCs and for every hash algorithm the request is silently ignored and hexadecimal is returned.

warning

MD5 and SHA-1 algorithms are considered cryptographically weak and should only be used for non-security purposes like data integrity verification or deduplication.

Examples

Basic File Checksum

Calculating SHA-256 checksum of file content...	{ "file_content": "Hello, World!" } - checksum: field: file_content algorithm: sha256 target_field: file_hash
generates cryptographic hash:	{ "file_content": "Hello, World!", "file_hash": "dffd6021bb2bd5b0af676290809ec3a53191dd81c7f70a4b28688a362182986f" }

Object Checksum with Field Selection

Creating checksum from specific object fields...	{ "user": { "id": 12345, "name": "john.doe", "email": "john@example.com", "password": "secret123", "last_login": "2024-01-15T10:30:00Z" } } - checksum: field: user algorithm: crc32 include_fields: ["id", "name", "email"] target_field: user_signature output_format: hex uppercase: true
calculates checksum from selected fields:	{ "user": {...}, "user_signature": "555CB275" }

Fast CRC Verification

Using fast CRC32 for data verification...	{ "packet_data": "Lorem ipsum dolor sit amet, consectetur adipiscing elit" } - checksum: field: packet_data algorithm: crc32 output_format: decimal target_field: data_crc
produces numeric CRC value:	{ "packet_data": "Lorem ipsum dolor sit amet, consectetur adipiscing elit", "data_crc": "1821039217" }

Array Content Checksum

Checksumming array data for deduplication...	{ "log_entries": [ "INFO: User logged in", "WARN: High memory usage", "ERROR: Database timeout" ] } - checksum: field: log_entries algorithm: fnv64 target_field: content_hash output_format: base64
the array is joined with no separator, and base64 is ignored on a 64-bit algorithm — the value comes back as hex:	{ "log_entries": [...], "content_hash": "53a82848fae95271" }

Configuration Integrity Check

Verifying configuration integrity with excluded sensitive fields...	{ "config": { "server": "prod-db-01", "port": 5432, "database": "analytics", "username": "analyst", "password": "secret", "ssl": true } } - checksum: field: config algorithm: sha1 exclude_fields: ["password"] target_field: config_checksum uppercase: true
creates integrity hash without sensitive data:	{ "config": {...}, "config_checksum": "16EF5E56036E8A9916B59DCE2DDA633638715794" }