neeto-monitor-ruby

neetoMonitor - A ruby gem for monitoring application's uptime and performance.

Installation:

1. Add the gem

gem "neeto-monitor-ruby", git: "https://github.com/bigbinary/neeto-monitor-ruby.git"

2. Install the gems

bundle install

3. Add configuration

The neetoMonitor can be configured at multiple levels. i.e. using env variables, adding a config file or manually passing the configuration hash to NeetoMonitorRuby.init!.

3.1 Setting the config via NeetoMonitorRuby.init!

NeetoMonitorRuby.init!({
  api_key: your_app_api_key,
  environment: production

  ...

})

3.2 Using a config file

# config/neetomonitor.yml
---
api_key: [API_KEY]
ping_timeout: 5

Following are the available configuration options with their default values.

base_url: (default: https://neetomonitor.com)
# The monitor server base URL. In `development` or `staging` we can set this to local/staging URL.

api_key: (default: nil)
# A unique api key for an organization.

environment: (default: nil)
# The monitoring environment name for an application

ping_timeout: (default: 8 seconds)
# The request timeout for ping requests.

sidekiq_enabled:
# Enables sidekiq plugin to report worker/job events.

config_path: (default: "config/neetomonitor.yml")
# The config file path

key_prefix_enabled: (default: true)
# This allows to enable/disable prefix for monitor key.

key_prefix: (default: application's name e.g. NeetoKb if integrated in neetoKb)
# The prefix added to the monitor key. e.g. if the check name is ArticleWorker, the result will be NeetoKb::ArticleWorker.

monitors:
  # more on this in next section

3.3 Using ENV variables:

The ENV variables must start with NEETO_MONITOR e.g. NEETO_MONITOR_API_KEY.

Similarly, other ENV variables can also be set by prefixing with NEETO_MONITOR_.

The precedence of the configuration is as follows:

Options passed to NeetoMonitorRuby.init! > Config file > ENV variables > default values

4. Configuring monitors

The monitors can be added in bulk using a config file. e.g.

  api_key: [API_KEY]
  monitors:
    checks:
      - name: "neeto-monitor"
        request_attributes:
          endpoint: "http://app.neetomonitor.test/health_check/"
          kind: "http"
          verb: "get"
          interval: 10
          timeout: 20
        assertions_attributes:
          - kind: "response_code"
            verb: "equal"
            value: 200
    jobs:
      - name: "neeto-job"
        assertions_attributes:
        - kind: max_duration
          verb: less_than
          value: 10
      - name: "hard-job",
        schedule: "* * * * *"
    heartbeats:
      - name: "neeto-heartbeat"
        schedule: "* * * * *"

The above config will create total four monitors i.e. 2 jobs and 1 check and 1 heartbeat.

4.1 Checks monitors (Checks)

The checks are used for monitoring website's uptime. The website requests are made at regular intervals which is configured via `interval` attribute.

name: # The name of check monitor e.g. "neeto-auth"
request_attributes: # This specifies configuring options for the website request.
  endpoint: # The request URL endpoint

  kind: # The request protocol

  verb: # The request verb/method i.e. get/post/head etc.

  interval: # The request frequency

  timeout: # The request timeout

assertions_attributes: # This specifies the assertion to be made when a request is made.
  kind:# The kind of assertion to be made
    # available options => response_code, response_time, response_body, response_json, response_header, ssl_expiry, min_duration, max_duration

  verb:# The operation verb for assertion
    # available options => less_than, less_than_equal, equal, not_equal, greater_than, greater_than_equal, contains, not_contains

  key:# The key from the response body json or headers
    # e.g. if response body = { "success": true }, key can be "success"

  value: # Any value to be checked for the assertion

4.2 Heartbeat monitors (Heartbeats)

The heartbeats are used for monitoring the website/service uptime. The basic difference between checks and heartbeats is that the heartbeats requests are sent from the host application. Where as the checks requests are made from the neetoMonitor server application.

The heartbeats can be added via the config file. When it is mentioned in the config, the heartbeats will be created in neetoMonitor upon the app initialization.

When schedule is defined for the heartbeats, the gem will send the heartbeats using the schedule frequency and neetoMonitor can send alerts when heartbeats are not received in this frequency.

The heartbeats events are sent using the HeartbeatRunnerJob. This sends the ping at time intervals based on the schedule. E.g. following heartbeat sends a ping every minute.

monitors:
  heartbeats:
    - name: "neeto-heartbeat"
      schedule: "* * * * *" # heartbeat frequency

Trigger heartbeats from application manually The heartbeats can be triggered manually using a periodic job. If you are using sidekiq, you can use sidekiq-cron to trigger heartbeats at desired frequency.

Here is an example of how you can configure heartbeats manually.

    # Worker for sending heartbeat
    class HeartbeatsWorker < BaseWorker
      sidekiq_options neeto_monitor_disabled: true

      def perform
        super

        NeetoMonitor::Monitor.new("test-heartbeat").ping
      end
    end

    # config/schedule.yml
    heartbeat:
      cron: "* * * * *"
      class: "HeartbeatsWorker"

4.3 Job monitors (Jobs)

The jobs are used for checking reliability and performance of background workers.

All the monitors can be added either via the web application or from the mentioned config file.

The jobs can be created by other methods as well i.e. without adding jobs to config section or web application UI. This is done using Sidekiq plugin included in the gem. When any new worker starts execution e.g. ArticleJob, the job monitor's run event is sent automatically. The complete event is sent when the job execution is successfully completed. Similarly, fail event is sent when job execution fails. This ArticleJob monitor is created at neetoMonitor application if it is not created already. If we want schedule field i.e. periodic job events to be sent to neetoMonitor application, we must add such jobs to a config file. e.g.

monitors:
  jobs:
    - name: "NeetoWorker"
      schedule: "* * * * *" # frequency of periodic jobs
    - name: "neeto-job"
      assertions_attributes:
        - kind:
          # available values => max_duration, min_duration
          verb:
          # available values => less_than_or_equal, less_than, greater_than, greater_than_or_equal, equal
          value:
          # allowed values => integer/float value

The job events are automatically sent from the sidekiq plugin when sidekiq_enabled is set to true. This flag is set to true by default. The job events are not sent when this plugin is not enabled. The gem currently supports only Sidekiq for background job monitoring. If we are using any other library for background jobs, we can send the events manually. E.g.

  class ArticleJob
    ..

    ...

    def perform
      job_monitor = NeetoMonitorRuby::Monitor.new("ArticleJob")
      series_stamp = Time.now.utc.to_f
      job_monitor.job_ping(state: "run", series: series_stamp)

      process_article_job # job business logic

      job_monitor.job_ping(state: "complete", series: series_stamp)
    rescue StandardError => error
      job_monitor.job_ping(state: "fail", series: series_stamp, message: error.message)
    end

    private

    ..
  end

Sometimes we don't want to monitor some Sidekiq workers. We have an option to disable the monitoring for such workers.

class MonitorDisabledWorker
  include Sidekiq::Worker
  sidekiq_options neeto_monitor_disabled: true

  ...

end

The default monitor key for jobs are the class name of the Sidekiq worker itself. We can change the key of the monitored job by setting neeto_monitor_key in sidekiq_options.

class CustomMonitorWorker
  include Sidekiq::Worker
  sidekiq_options neeto_monitor_key: "super_worker"

  ...
end