LogStashLogger
This gem implements a subclass of Ruby's Logger class that logs directly to logstash. It writes to a logstash listener over a TCP connection, in logstash JSON format. This is an improvement over writing to a file or syslog since logstash can receive the structured data directly.
Features
- Writes directly to logstash over a UDP or TCP connection.
- Always writes in logstash JSON format.
- Logger can take a string message, a hash, a LogStash::Event, or a logstash-formatted json string as input.
- Events are automatically populated with message, timestamp, host, and severity.
Installation
Add this line to your application's Gemfile:
gem 'logstash-logger'
And then execute:
$ bundle
Or install it yourself as:
$ gem install logstash-logger
Basic Usage
First set up a logstash agent to receive input over a UDP or TCP port. Then in ruby, create a LogStashLogger that writes to that port.
require 'logstash-logger'
# Defaults to UDP
logger = LogStashLogger.new('localhost', 5228)
logger.info 'test'
# Writes the following to UDP port 5228:
# {"@source":"server-host-name","@tags":[],"@fields":{"severity":"INFO"},"@message":"test","@timestamp":"2013-04-08T18:56:23.767273+00:00"}
# Specify UDP or TCP explicitly
udp_logger = LogStashLogger.new('localhost', 5228, :udp)
tcp_logger = LogStashLogger.new('localhost', 5229, :tcp)
Rails integration
Add the following to your config/environments/production.rb:
config.logger = ActiveSupport::TaggedLogging.new(LogStashLogger.new('localhost', 5228))
To get Rails to nicely output its logs in structured logstash format, try one of the following gems:
Currently these gems output a JSON string, which LogStashLogger then parses. Future versions of these gems could potentially have deeper integration with LogStashLogger (i.e. by writing LogStash::Event objects).
UDP vs TCP
Should you write to a UDP or TCP listener? It depends on your specific needs, but most applications should use the default (UDP).
- UDP is faster because it's asynchronous (fire-and-forget). However, this means that log messages could get dropped. This is okay for most applications.
- TCP verifies that every message has been received via two-way communication . This could slow your app down to a crawl if the TCP listener is under heavy load.
For a more detailed discussion of UDP vs TCP, I recommend reading this article: UDP vs. TCP
Ruby compatibility
Verified to work with:
- Ruby 1.9.3
- Ruby 2.0.0
- JRuby 1.7+ (1.9 mode)
Ruby 1.8.7 is not supported because LogStash::Event is not compatible with Ruby 1.8.7. This might change in the future.
The specs don't pass in Rubinius yet, but the logger does work.
Breaking change in version 0.3+
Earlier versions of this gem (<= 0.2.1) only implemented a TCP connection. Newer versions (>= 0.3) also implement UDP, and use that as the new default. Please be aware if you are using the default constructor and still require TCP, you should add an additional argument:
# Now defaults to UDP instead of TCP
logger = LogStashLogger.new('localhost', 5228)
# Explicitly specify TCP instead of UDP
logger = LogStashLogger.new('localhost', 5228, :tcp)
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create new Pull Request