fluent-plugin-latency

Fluentd plugin to measure latency until receiving the messages.

What is this for?

This plugin is to investigate the network latency, in addition, the blocking situation of input plugins.

In the Fluentd mechanism, input plugins usually blocks and will not receive a new data until the previous data processing finishes.

By seeing the latency, you can easily find how long the blocking situation is occuring.

How this works

Fluentd messages include the time attribute which expresses the time of when the message is created, or when it is written in application logs.

This plugin takes the difference between the current time and the time attribute to obtain the latency.

Installation

Use RubyGems:

gem install fluent-plugin-latency

Configuration

Following example measures the max and average latency until receiving messages.

<source>
  type forward
  port 24224
</source>

# Latency plugin output comes here
<match latency>
  type stdout
</match>

# Whatever you want to do
<match rewrite.**>
  type stdout
</match>

# All messages come here once. 
<match **>
  type copy
  <store>
    type latency
    tag latency
    interval 60
  </store>
  <store>
    type rewrite
    add_prefix rewrite
  </store>
</match>

Output will be like

latency: {"max":1.011,"avg":0.002","num":10}

where max and avg are the maximum and average latency, and num is the number of messages.

Option Parameters

interval

The time interval to emit measurement results
tag

The output tag name. Default is latency

ChangeLog

See CHANGELOG.md for details.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request