Date Index

Control Flow Elastic Compatible

Synopsis

Creates dynamic index names based on date fields in documents by combining a configurable prefix with formatted dates.

Schema

- date_index_name:
    field: <ident>
    date_rounding: <enum>
    date_formats: <string[]>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    index_name_format: <string>
    index_name_prefix: <string>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>
    time_zone: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y	-	Field containing the date/timestamp to process
date_rounding	N	-	Time unit to round to: y (year), M (month), w (week), d (day), h (hour), m (minute), s (second). Case-sensitive — M is month, m is minute. An omitted or unrecognised value leaves the timestamp unrounded
date_formats	N	see below	Date formats to try, in order. Named formats (RFC3339, ISO8601, UNIX, UNIX_MS, UNIX_NANO, …) and Go layouts are both accepted
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	Continue processing if parsing fails
ignore_missing	N	false	Skip if the date field is missing
index_name_format	N	-	Go time layout for the date part — 2006-01-02, not yyyy-MM-dd. There is no default: leave it out and the output is the prefix alone
index_name_prefix	N	-	String to prepend to the formatted date
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier
disabled	N	false	When true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration
time_zone	N	Director's local zone	IANA name used to interpret the parsed date. Not UTC — an unset value takes the host's local zone, so the same event rounds differently on differently-configured Directors. Set it explicitly

Details

The processor extracts a date from a specified field, applies configured rounding, and generates an index name by combining a prefix with the formatted date.

The result is written to _vmetric.index, which is the routing field targets read to choose a destination index. There is no target_field — the output location is fixed.

With date_formats unset, the processor tries RFC3339NANO, RFC3339, BSD, GO_MS, GO and AUTO. A value made entirely of digits is treated as a UNIX timestamp instead, and one containing a . as UNIX nanoseconds.

The date can be rounded to various time units (year, month, week, day, hour, minute, second) and formatted according to specified patterns. This is particularly useful for time-series data where you want to organize documents into time-based indices.

note

With the weekly rounding (w), the date is rounded to the start of the ISO week (Monday).

warning

The date parsing will fail if none of the specified date formats match the input. Consider setting ignore_failure to true if errors should be tolerated.

Examples

Monthly

Rounding dates to months...	{ "raw-date": "2016-04-25T12:02:01.789Z" } - date_index_name: field: raw-date date_rounding: "M" index_name_prefix: "monthly-" index_name_format: "2006-01-02"
creates a monthly index name:	{ "raw-date": "2016-04-25T12:02:01.789Z", "_vmetric": { "index": "monthly-2016-04-01" } }

Daily

Dates from a nested field...	{ "user": { "login": "2016-04-25T12:02:01.789Z" } } - date_index_name: field: "user.login" date_rounding: "d" index_name_prefix: "logins-" index_name_format: "2006-01-02"
create a daily index name:	{ "user": { "login": "2016-04-25T12:02:01.789Z" }, "_vmetric": { "index": "logins-2016-04-25" } }

Weekly

Rounding to the start of the ISO week...	{ "timestamp": "2016-04-25T12:02:01.789Z" } - date_index_name: field: timestamp date_rounding: "w" index_name_prefix: "weekly-" index_name_format: "2006-01-02"
creates a weekly index name:	{ "timestamp": "2016-04-25T12:02:01.789Z", "_vmetric": { "index": "weekly-2016-04-25" } }