Fluent::Plugin::Anomalydetect

To detect anomaly for log stream, use this plugin. Then you can find changes in logs casually.

Installation

Add this line to your application’s Gemfile:

gem 'fluent-plugin-anomalydetect'

And then execute:

$ bundle

Or install it yourself as:

$ gem install fluent-plugin-anomalydetect

Usage

<source>
  type file
  ...
  tag access.log
</source>

<match access.**>
  type anomalydetect
  tag anomaly.access
  tick 86400
</match>

<match anomaly.access>
  type file
  ...
</match>

Then the plugin output anomaly log counts in each day.

This plugin watches a value of input record number in the interval set with tick.

If you want to watch a value for a target field <fieldname> in data, write below:

<match access.**>
  type anomalydetect
  tag anomaly.access
  tick 86400
  target fieldname
</match>

more configuration

<match access.**>
  type anomalydetect
  tag anomaly.access
  tick 86400
  target fieldname
  outlier_term 7
  outlier_discount 0.5
  smooth_term 7
  score_term 28
  score_discount 0.01
</match>

If you want to know detail of these parameters, see “Theory”.

<match access.**>
  type anomalydetect
  ...
  store_file /path/to/anomalydetect.dat
</match>

If “store_file” option was specified, a historical stat will be stored to the file at shutdown, and it will be restored on started.

<match access.**>
  type anomalydetect
  ...
  threshold 3
</match>

If “threshold” option was specified, plugin only ouput when the anomalyscore is more than threshold.

Fluent::Plugin::Anomalydetect

Installation

Usage

more configuration

Theory

TODO

threshold

FFT algorithms

Copyright