date

Status: stable

The date filter is used for parsing dates from fields and using that date or timestamp as the timestamp for the event.

For example, syslog events usually have timestamps like this: "Apr 17 09:32:01"

You would use the date format "MMM dd HH:mm:ss" to parse this.

The date filter is especially important for sorting events and for backfilling old data. If you don't get the date correct in your event, then searching for them later will likely sort out of order.

In the absence of this filter, logstash will choose a timestamp based on the first time it sees the event (at input time), if the timestamp is not already set in the event. For example, with file input, the timestamp is set to the time of reading.

Synopsis

This is what it might look like in your config file:

filter {
  date {
    /[A-Za-z0-9_-]+/ => ... # array (optional)
    add_field => ... # hash (optional), default: {}
    add_tag => ... # array (optional), default: []
    tags => ... # array (optional), default: []
    type => ... # string (optional), default: ""
  }
}

Details

/[A-Za-z0-9_-]+/

• The configuration attribute name here is anything that matches the above regular expression.
• Value type is array
• There is no default value for this setting.

Config for date is: fieldname => dateformat

The same field can be specified multiple times (or multiple dateformats for the same field) do try different time formats; first success wins.

The date formats allowed are anything allowed by Joda-Time (java time library), generally: java.text.SimpleDateFormat

There are a few special exceptions, the following format literals exist to help you save time and ensure correctness of date parsing.

• "ISO8601" - should parse any valid ISO8601 timestamp, such as 2011-04-19T03:44:01.103Z
• "UNIX" - will parse unix time in seconds since epoch
• "UNIX_MS" - will parse unix time in milliseconds since epoch

For example, if you have a field 'logdate' and with a value that looks like 'Aug 13 2010 00:03:44' you would use this configuration:

logdate => "MMM dd yyyy HH:mm:ss"

add_field

• Value type is hash
• Default value is {}

If this filter is successful, add any arbitrary fields to this event. Example:

filter {
  myfilter {
    add_field => [ "sample", "Hello world, from %{@source}" ]
  }
}

On success, myfilter will then add field 'sample' with the value above and the %{@source} piece replaced with that value from the event.

add_tag

• Value type is array
• Default value is []

If this filter is successful, add arbitrary tags to the event. Tags can be dynamic and include parts of the event using the %{field} syntax. Example:

filter {
  myfilter {
    add_tag => [ "foo_%{somefield}" ]
  }
}

If the event has field "somefield" == "hello" this filter, on success, would add a tag "foo_hello"

tags

• Value type is array
• Default value is []

Only handle events with all of these tags. Note that if you specify a type, the event must also match that type. Optional.

type

• Value type is string
• Default value is ""

The type to act on. If a type is given, then this filter will only act on messages with the same type. See any input plugin's "type" attribute for more. Optional.

---

This is documentation from lib/logstash/filters/date.rb