file

Status: beta

Stream events from files.

By default, each event is assumed to be one line. If you want to join lines, you'll want to use the multiline filter.

Files are followed in a manner similar to "tail -0F". File rotation is detected and handled by this input.

Synopsis

This is what it might look like in your config file:

input {
  file {
    add_field => ... # hash (optional), default: {}
    debug => ... # boolean (optional)
    discover_interval => ... # number (optional), default: 15
    exclude => ... # array (optional)
    format => ... # ["plain", "json", "json_event"] (optional)
    message_format => ... # string (optional)
    path => ... # array (required)
    sincedb_path => ... # string (optional)
    sincedb_write_interval => ... # number (optional), default: 15
    stat_interval => ... # number (optional), default: 1
    tags => ... # array (optional)
    type => ... # string (required)
  }
}

Details

add_field

• Value type is hash
• Default value is {}

Add a field to an event

debug

• Value type is boolean
• There is no default value for this setting.

Set this to true to enable debugging on an input.

discover_interval

• Value type is number
• Default value is 15

How often we expand globs to discover new files to watch.

exclude

• Value type is array
• There is no default value for this setting.

Exclusions (matched against the filename, not full path). Globs are valid here, too. For example, if you have

path => "/var/log/*"

you might want to exclude gzipped files:

exclude => "*.gz"

format

• Value can be any of: "plain", "json", "json_event"
• There is no default value for this setting.

The format of input data (plain, json, json_event)

message_format

• Value type is string
• There is no default value for this setting.

If format is "json", an event sprintf string to build what the display @message should be given (defaults to the raw JSON). sprintf format strings look like %{fieldname} or %{@metadata}.

If format is "json_event", ALL fields except for @type are expected to be present. Not receiving all fields will cause unexpected results.

path (required setting)

• Value type is array
• There is no default value for this setting.

The path to the file to use as an input. You can use globs here, such as "/var/log/*.log" Paths must be absolute and cannot be relative.

sincedb_path

• Value type is string
• There is no default value for this setting.

Where to write the since database (keeps track of the current position of monitored log files). Defaults to the value of environment variable "$SINCEDB_PATH" or "$HOME/.sincedb".

sincedb_write_interval

• Value type is number
• Default value is 15

How often to write a since database with the current position of monitored log files.

stat_interval

• Value type is number
• Default value is 1

How often we stat files to see if they have been modified. Increasing this interval will decrease the number of system calls we make, but increase the time to detect new log lines.

tags

• Value type is array
• There is no default value for this setting.

Add any number of arbitrary tags to your event.

This can help with processing later.

type (required setting)

• Value type is string
• There is no default value for this setting.

Label this input with a type. Types are used mainly for filter activation.

If you create an input with type "foobar", then only filters which also have type "foobar" will act on them.

The type is also stored as part of the event itself, so you can also use the type to search for in the web interface.

---

This is documentation from lib/logstash/inputs/file.rb