fluent-plugin-filter_where

Fluentd plugin to filter records with SQL-like WHERE statements

Requirements

fluent-plugin-filter_where supports both v0.14 API and v0.12 API in one gem.

Installation

Use RubyGems:

gem install fluent-plugin-filter_where

Configuration

where

The SQL-like WHERE statements. See SQL-like Syntax for more details.

Example

<filter foo.**>
  @type where
  where string_key = 'string' OR number_key >= 0.1
</filter>

SQL-like Syntax

Example:

where (string_key START_WITH 'str' AND number_key > 1.0) OR ("true_key" = true AND NOT (string_key REGEXP '^reg'))

Literals

Boolean Literal

true or TRUE or false or FALSE are considered as a boolean literal

Number Literal

Characters matching with a regular expression -?[0-9]+(\.[0-9]+)? is considered as a number literal

String Literal

Characters surrounded by ' such as 'foo' is considered as a string literal

Json Literal

Not supported yet

Identifier Literal

Characters matching with a regular expression [a-zA-Z_][a-zA-z0-9_]* such as foobar, and characters surrounded by " such as "foo-bar", "foo.bar", and "foo\"bar" are considred as an identifier literal, that is, fluentd's record key name.

Operators

Boolean Operator

=: Equal operator
!=, <>: Not equal operator

Number Operator (Long and Double)

=: Equal operator
!=, <>: Not equal operator
>: Greater than operator
>=: Greater than or equal operator
<=: Less than or equal operator
<: Less than operator

String Operator

=: Equal operator
!=, <>: Not equal operator
START_WITH
END_WITH
INCLUDE
REGEXP

Json Operator

Not supported yet

Negate Operator

NOT xxx

NULL Operator

IS NULL
IS NOT NULL

ChangeLog

See CHANGELOG.md for details.

Development

Run test:

$ bundle exec rake test

Release:

Modify gemspec and CHANGELOG.md, then

$ bundle exec rake release

Development of SQL-like Syntax

This plugin uses rexical for lexical scanner generator, and racc for parser generator.

If you modify praser.rex or praser.racc, you must compile them as:

$ bundle exec rake compile

The test task runs the compile task before running.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request