mutate

Status: stable

The mutate filter allows you to do general mutations to fields. You can rename, remove, replace, and modify fields in your events.

TODO(sissel): Support regexp replacements like String#gsub ?

Synopsis

This is what it might look like in your config file:

filter {
  mutate {
    add_field => ... # hash (optional), default: {}
    add_tag => ... # array (optional), default: []
    convert => ... # hash (optional)
    exclude_tags => ... # array (optional), default: []
    gsub => ... # array (optional)
    lowercase => ... # array (optional)
    remove => ... # array (optional)
    remove_tag => ... # array (optional), default: []
    rename => ... # hash (optional)
    replace => ... # hash (optional)
    tags => ... # array (optional), default: []
    type => ... # string (optional), default: ""
    uppercase => ... # array (optional)
  }
}

Details

add_field

• Value type is hash
• Default value is {}

If this filter is successful, add any arbitrary fields to this event. Example:

filter {
  myfilter {
    add_field => [ "sample", "Hello world, from %{@source}" ]
  }
}

On success, myfilter will then add field 'sample' with the value above and the %{@source} piece replaced with that value from the event.

add_tag

• Value type is array
• Default value is []

If this filter is successful, add arbitrary tags to the event. Tags can be dynamic and include parts of the event using the %{field} syntax. Example:

filter {
  myfilter {
    add_tag => [ "foo_%{somefield}" ]
  }
}

If the event has field "somefield" == "hello" this filter, on success, would add a tag "foo_hello"

convert

• Value type is hash
• There is no default value for this setting.

Convert a field's value to a different type, like turning a string to an integer. If the field value is an array, all members will be converted. If the field is a hash, no action will be taken.

Valid conversion targets are: integer, float, string

Example:

filter {
  mutate {
    convert => [ "fieldname", "integer" ]
  }
}

exclude_tags

• Value type is array
• Default value is []

Only handle events without any of these tags. Note this check is additional to type and tags.

gsub

• Value type is array
• There is no default value for this setting.

Convert a string field by applying a regular expression and a replacement if the field is not a string, no action will be taken

this configuration takes an array consisting of 3 elements per field/substitution

be aware of escaping any backslash in the config file

for example:

mutate {

  …
 gsub => [
   "fieldname", "\\/", "_",      #replace all forward slashes with underscore
   "fieldname", "[\\?#-]", "_"   #replace backslashes, question marks, hashes and minuses with underscore
 ]
  …

}

lowercase

• Value type is array
• There is no default value for this setting.

Convert a string to its lowercase equivalent

Example:

mutate {

lowercase => [ "fieldname" ]

}

remove

• Value type is array
• There is no default value for this setting.

Remove one or more fields.

Example:

filter {
  mutate {
    remove => [ "client" ]  # Removes the 'client' field
  }
}

remove_tag

• Value type is array
• Default value is []

If this filter is successful, remove arbitrary tags from the event. Tags can be dynamic and include parts of the event using the %{field} syntax. Example:

filter {
  myfilter {
    remove_tag => [ "foo_%{somefield}" ]
  }
}

If the event has field "somefield" == "hello" this filter, on success, would remove the tag "foo_hello" if it is present

rename

• Value type is hash
• There is no default value for this setting.

Rename one or more fields.

Example:

filter {
  mutate {
    # Renames the 'HOSTORIP' field to 'client_ip'
    rename => [ "HOSTORIP", "client_ip" ]
  }
}

replace

• Value type is hash
• There is no default value for this setting.

Replace a field with a new value. The new value can include %{foo} strings to help you build a new value from other parts of the event.

Example:

filter {
  mutate {
    replace => [ "@message", "%{source_host}: My new message" ]
  }
}

tags

• Value type is array
• Default value is []

Only handle events with all of these tags. Note that if you specify a type, the event must also match that type. Optional.

type

• Value type is string
• Default value is ""

The type to act on. If a type is given, then this filter will only act on messages with the same type. See any input plugin's "type" attribute for more. Optional.

uppercase

• Value type is array
• There is no default value for this setting.

Convert a string to its uppercase equivalent

Example:

mutate {

 uppercase => [ "fieldname" ]

}

---

This is documentation from lib/logstash/filters/mutate.rb