elasticsearch_river

Status: beta

This output lets you store logs in elasticsearch. It's similar to the 'elasticsearch' output but improves performance by using an AMQP server, such as rabbitmq, to send data to elasticsearch.

Upon startup, this output will automatically contact an elasticsearch cluster and configure it to read from the queue to which we write.

You can learn more about elasticseasrch at http://elasticsearch.org More about the elasticsearch rabbitmq river plugin: https://github.com/elasticsearch/elasticsearch-river-rabbitmq/blob/master/README.md

Synopsis

This is what it might look like in your config file:

output {
  elasticsearch_river {
    amqp_host => ... # string (required)
    amqp_port => ... # number (optional), default: 5672
    debug => ... # boolean (optional)
    durable => ... # boolean (optional), default: true
    es_bulk_size => ... # number (optional), default: 1000
    es_bulk_timeout_ms => ... # number (optional), default: 100
    es_host => ... # string (required)
    es_port => ... # number (optional), default: 9200
    exchange => ... # string (optional), default: "elasticsearch"
    exchange_type => ... # ["fanout", "direct", "topic"] (optional), default: "direct"
    index => ... # string (optional), default: "logstash-%{+YYYY.MM.dd}"
    index_type => ... # string (optional), default: "%{@type}"
    key => ... # string (optional), default: "elasticsearch"
    name => ... # string (optional), default: "elasticsearch"
    password => ... # string (optional), default: "guest"
    persistent => ... # boolean (optional), default: true
    tags => ... # array (optional), default: []
    type => ... # string (optional), default: ""
    user => ... # string (optional), default: "guest"
    vhost => ... # string (optional), default: "/"
  }
}

Details

amqp_host (required setting)

• Value type is string
• There is no default value for this setting.

Hostname of AMQP server

amqp_port

• Value type is number
• Default value is 5672

Port of AMQP server

debug

• Value type is boolean
• There is no default value for this setting.

durable

• Value type is boolean
• Default value is true

AMQP durability setting. Also used for ElasticSearch setting

es_bulk_size

• Value type is number
• Default value is 1000

ElasticSearch river configuration: bulk fetch size

es_bulk_timeout_ms

• Value type is number
• Default value is 100

ElasticSearch river configuration: bulk timeout in milliseconds

es_host (required setting)

• Value type is string
• There is no default value for this setting.

The name/address of an ElasticSearch host to use for river creation

es_port

• Value type is number
• Default value is 9200

ElasticSearch API port

exchange

• Value type is string
• Default value is "elasticsearch"

AMQP exchange name

exchange_type

• Value can be any of: "fanout", "direct", "topic"
• Default value is "direct"

The exchange type (fanout, topic, direct)

index

• Value type is string
• Default value is "logstash-%{+YYYY.MM.dd}"

The index to write events to. This can be dynamic using the %{foo} syntax. The default value will partition your indeces by day so you can more easily delete old data or only search specific date ranges.

index_type

• Value type is string
• Default value is "%{@type}"

The index type to write events to. Generally you should try to write only similar events to the same 'type'. String expansion '%{foo}' works here.

key

• Value type is string
• Default value is "elasticsearch"

AMQP routing key

name

• Value type is string
• Default value is "elasticsearch"

AMQP queue name

password

• Value type is string
• Default value is "guest"

AMQP password

persistent

• Value type is boolean
• Default value is true

AMQP persistence setting

tags

• Value type is array
• Default value is []

Only handle events with all of these tags. Note that if you specify a type, the event must also match that type. Optional.

type

• Value type is string
• Default value is ""

The type to act on. If a type is given, then this output will only act on messages with the same type. See any input plugin's "type" attribute for more. Optional.

user

• Value type is string
• Default value is "guest"

AMQP user

vhost

• Value type is string
• Default value is "/"

AMQP vhost

---

This is documentation from lib/logstash/outputs/elasticsearch_river.rb