Fluentd Server

A Fluentd config distribution server

Demo: http://fluentd-server.herokuapp.com

What You Can Do

With Fluentd Server, you can manage fluentd configuration files centrally with erb.

For example, you may create a config post whose name is worker as:

<source>
  type forward
  port <%= port %>
</source>

<match **>
  type stdout
</match>

Then you can download the config via an API whose uri is like /api/worker?port=24224 where its query parameters are replaced with variables in the erb. The downloaded contents should become as follows:

<source>
  type forward
  port 24224
</source>

<match **>
  type stdout
</match>

How to Use

The include directive of fluentd config supports http, so write just one line on your fluentd.conf as:

# /etc/fluentd.conf
include http://fqdn.to.fluentd-server/api/:name?port=24224

so that it will download the real configuration from the Fluentd Server where :name is the name of your post.

Installation

Prerequisites

SQLite
Ruby 2.0 or later

From Gem package

Easy steps on installation with gem and SQLite.

$ gem install fluentd-server
$ gem install sqlite3
$ fluentd-server new
$ cd fluentd-server
$ fluentd-server init # creates database scheme on SQLite
$ fluentd-server start

Then see http://localhost:5126/.

From Git repository

Install from git repository.

$ git clone https://github.com/sonots/fluentd-server.git
$ cd fluentd-server
$ bundle
$ bundle exec fluentd-server init # creates database scheme on SQLite
$ bundle exec fluentd-server start

Then see http://localhost:5126/.

Configuration

To configure fluentd-server, edit the .env file in the project root directory.

The default configuration is as follows:

PORT=5126
HOST=0.0.0.0
# DATABASE_URL=sqlite3:data/fluentd_server.db
# JOB_DIR=jobs
# LOG_PATH=STDOUT
# LOG_LEVEL=debug
# LOG_SHIFT_AGE=0
# LOG_SHIFT_SIZE=1048576
# LOCAL_STORAGE=false
# DATA_DIR=data
# SYNC_INTERVAL=60

LOCAL FILE STORAGE

Fluentd Server also supports to edit Fluentd configuration files as local files for the case using git (or any VCSs) to manage Fluentd configuration revisions.

To use this feature, enable LOCAL_STORAGE in .env file:

LOCAL_STORAGE=true
DATA_DIR=data
SYNC_INTERVAL=60

The DATA_DIR is the location to place your configuration files locally, and the SYNC_INTERVAL is the interval where a synchronization worker works.

Putting any files whose name ends with .erb in DATA_DIR automatically synchronizes with DB by the sync worker. Removing .erb files is also synchronized with DB.

NOTE: Enabling this feature disables to edit the Fluentd configuration from the Web UI.

HTTP API

See API.md.

ToDo

Automatic deployment (restart) support like the one of chef-server

ChangeLog

See CHANGELOG.md for details.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request