Output filter plugin for modifying each event record for Fluentd

Adding arbitary field to event record without custmizing existence plugin.

For example, generated event from in_tail doesn't contain "hostname" of running machine. In this case, you can use record_modifier to add "hostname" field to event record.

Installation

Use RubyGems:

gem install fluent-plugin-record-modifier

Configuration

<match pattern>
  type record_modifier
  tag foo.filtered

  gen_host ${hostname}
  foo bar
</match>

If following record is passed:

{"message":"hello world!"}

then you got new record like below:

{"message":"hello world!", "gen_host":"oreore-mac.local", "foo":"bar"}

char_encoding

Fluentd including some plugins treats the logs as a BINARY by default to forward. But an user sometimes processes the logs depends on their requirements, e.g. handling char encoding correctly.

char_encoding parameter is useful for this case.

<match pattern>
  type record_modifier

  # set UTF-8 encoding information to string.
  char_encoding utf-8

  # change char encoding from 'UTF-8' to 'EUC-JP'
  char_encoding utf-8:euc-jp
</match>

remove_keys

The logs include needless record keys in some cases. You can remove it by using remove_keys parameter.

<match pattern>
  type record_modifier

  # remove key1 and key2 keys from record
  remove_keys key1,key2
</match>

If following record is passed:

{"key1":"hoge", "key2":"foo", "key3":"bar"}

then you got new record like below:

{"key3":"bar"}

Mixins

TODO

  • Adding following features if needed

    • Use HandleTagNameMixin to keep original tag
    • Replace record value
AuthorMasahiro Nakagawa
CopyrightCopyright (c) 2013- Masahiro Nakagawa
LicenseMIT License