Fluentd: Open-Source Log Collector

Code Climate

Fluentd collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure (Learn more about the Unified Logging Layer).

An event consists of tag, time and record. Tag is a string separated with ’.’ (e.g. myapp.access). It is used to categorize events. Time is a UNIX time recorded at occurrence of an event. Record is a JSON object.

Example Use Cases

Use CaseDescriptionDiagram
Centralizing Apache/Nginx Server LogsFluentd can be used to tail access/error logs and transport them reliably to remote systems.
Syslog AlertingFluentd can “grep” for events and send out alerts.
Mobile/Web Application LoggingFluentd can function as middleware to enable asynchronous, scalable logging for user action events.

Quick Start

$ gem install fluentd
$ fluentd -s conf
$ fluentd -c conf/fluent.conf &
$ echo '{"json":"message"}' | fluent-cat debug.test

Fluentd UI: Admin GUI

Fluentd UI is a graphical user interface to start/stop/configure Fluentd.

More Information

  • Website: http://fluentd.org/
  • Documentation: http://docs.fluentd.org/
  • Source repository: http://github.com/fluent
  • Discussion: http://groups.google.com/group/fluentd
  • Slack / Community: http://slack.fluentd.org
  • Newsletters: http://get.treasuredata.com/Fluentd_education
  • Author: Sadayuki Furuhashi
  • Copyright: (c) 2011 FURUHASHI Sadayuki
  • License: Apache License, Version 2.0

Contributors:

Patches contributed by great developers.