Observed::Logstash

Allows you to use the number of results returned by an Elasticsearch query as an Observed healthcheck.

Expected use case is searching server logs stored in Logstash format.

You provide an Elasticsearch query and a timespan, and the plugin will search for logs that match your query. If there are too few hits, or too many, it will record an error event.

Example use cases

  • If your web server returned more than X "500 Internal Server Error" responses in the last few minutes, it's probably unhealthy.

  • If it returned fewer than Y "200 OK" responses in the last few minutes, it's probably unhealthy.

Installation

Add this line to your application's Gemfile:

gem 'observed-logstash'

And then execute:

$ bundle

Or install it yourself as:

$ gem install observed-logstash

Test

Some tests expect an Elasticsearch instance to be running on Localhost.

$ git clone https://github.com/cb372/observed-logstash.git
$ cd observed-logstash
$ bundle install
$ elasticsearch
$ bundle exec rspec

Usage

Configuration parameters

NameRequired?Default valueDescription
hostNolocalhost:9200ES server hostname and port
index_name_formatNologstash-%Y.%m.%d (Logstash daily format)Naming format of ES indices
queryYesA hash representing an ES query, e.g. { :term => { :status => 404 } }
timespan_in_secondsYesSearch for logs from the last N seconds
max_hitsNo1000000Maximum number of matching logs in the last N seconds. If there are more than these, an error will be recorded.
min_hitsNo0

Example configuration

observe 'myapp.404', via: 'logstash', with: {
    host: 'localhost:9200',
    index_name_format: 'observed-logstash-test-%Y.%m.%d',
    query: { :term => { :status => 404 } },
    timespan_in_seconds: 3600,
    max_hits: 10
}

Example reporting

report /myapp.404/, via: 'stdout', with: {
    format: -> tag, time, data {
      case data[:status]
      when :success
        "Looks OK! #{data[:message]}"
      else
        "Oh noes! #{data[:message]}"
      end
    }
}