Class: Rubix::AutoSender

Inherits:
Object
  • Object
show all
Includes:
Logs
Defined in:
lib/rubix/auto_sender.rb

Overview

Sender

Rubix comes with a zabbix_pipe script which, coupled with the href="http://www.zabbix.com/documentation/1.8/manpages/zabbix_sender"> utility, allows for writing data to Zabbix.

By the design of Zabbix, all data written via zabbix_sender (and therefore zabbix_pipe) must be written to items with type “Zabbix trapper”. This type instructs Zabbix to accept values written by zabbix_sender.

zabbix_pipe can consume data from STDIN, a file on disk, or a named pipe. It consumes data one line at a time from any of these sources and uses the zabbix_sender utility to send this data to a Zabbix server.

Here’s an example of starting zabbix_pipe and sending some data onto Zabbix. (Credentials and addresses for the Zabbix server and the Zabbix API can all be customized; see the --help option.)

# Send one data point -- this is tab separated!
$ echo "foo.bar.baz	123" | zabbix_pipe --host='My Zabbix Host' --host_groups='My Zabbix Servers,Some Other Group'
# Send a bunch of data points in a file
$ cat my_data.tsv | zabbix_pipe --host='My Zabbix Host' --host_groups='My Zabbix Servers,Some Other Group'

You can also pass the file directly:

# Send a bunch of data points in a file
$ zabbix_pipe --host='My Zabbix Host' --host_groups='My Zabbix Servers,Some Other Group' my_data.tsv

You can also listen from a named pipe. This is useful on a “production” system in which many processes may want to simply and easily write somewhere without worrying about what happens.

# In the first terminal
$ mkfifo /dev/zabbix
$ zabbix_pipe --pipe=/dev/zabbix --host='My Zabbix Host' --host_groups='My Zabbix Servers,Some Other Group'

# In another terminal
$ echo "foo.bar.baz	123" > /dev/zabbix
$ cat my_data > /dev/zabbix

In any of these modes, you can also send JSON data directly.

$ echo '{"data": [{"key": "foo.bar.baz", "value": 123}, {"key": "foo.bar.baz", "value": 101}]}' > /dev/zabbix

This simple block of JSON doesn’t really add any power to zabbix_pipe. What becomes more interesting is that we can wedge in data for diffferent hosts, items, &c. when using JSON input:

$ echo '{"host": "My first host", "host_groups": "My Zabbix Servers,Some Other Group", "data": [{"key": "foo.bar.baz", "value": 123}, {"key": "foo.bar.baz", "value": 101}]}' > /dev/zabbix
$ echo '{"host": "My second host", "host_groups": "My Zabbix Servers,Some Other Group", "data": [{"key": "foo.bar.baz", "value": 123}, {"key": "foo.bar.baz", "value": 101}]}' > /dev/zabbix
$ echo '{"host": "My third host", "host_groups": "My Zabbix Servers,Some Other Group", "data": [{"key": "foo.bar.baz", "value": 123}, {"key": "foo.bar.baz", "value": 101}]}' > /dev/zabbix

Rubix will switch hosts on the fly.

Auto-vivification

By default, for every item written into zabbix_pipe, Rubix will first check, using the Zabbix API, whether an item with the given key exists for the given host. If not, Rubix will create the host and the item. You can pass a few details along with your data (when writing in JSON format) or at startup of the zabbix_pipe to tune some of the properites of newly created hosts and items:

$ echo '{"host": "My host", "hostgroups": "Host Group 1,Host Group 2", "templates": "Template 1,Template 2", "applications": "App 1, App2", "data": [{"key": "foo.bar.baz", "value": 123}, {"key": "foo.bar.baz", "value": 101}]}' > /dev/zabbix

If the host ‘My host’ does not exist, Rubix will create it, putting in each of the host groups ‘Host Group 1’ and ‘Host Group 2’ (which will also be auto-vivified), and attach it to templates ‘Template 1’ and ‘Template 2’ (auto-vivified). If the item does not exist for the host ‘My host’, then it will be created and put inside applications ‘App1’ and ‘App2’ (auto-vivified). The value type of the item (unsigned int, float, character, text, &c.) will be chosen dynamically based on the value being written. The created items will all be of the type “Zabbix trapper” so that they can be written to.

Auto-vivification is intended to make it easy to register a lot of dynamic hosts and items in Zabbix. This is perfect for cloud deployments where resources to be monitored are often dynamic.

Failed writes

By the design of Zabbix, a newly created item of type ‘Zabbix trapper’ will not accept data for some interval, typically a minute or so, after being created. This means that when writing a series of values for some non-existent item ‘foo.bar.baz’, the first write will cause the item to be created (auto-vivified), the next several writes will fail as the Zabbix server is not accepting writes for this item yet, and then all writes will begin to succeed as the Zabbix server catches up. If it is absolutely essential for all writes to succeed, including the first, then zabbix_pipe needs to go to sleep for a while after creating a new item in order to give the Zabbix server time to catch up. This can be configured with the --create_item_sleep option. By default this is set to 0.

Skipping auto-vivification

Attempting to auto-vivify keys on every single write is expensive and does not scale. It’s recommended, therefore, to run zabbix_pipe with the --fast flag in production settings. In this mode, zabbix_pipe will not attempt to auto-vivify anything – if items do not exist, writes will just fail, as they do with zabbix_sender itself.

A good pattern is to set up a production pipe (in --fast) mode at /dev/zabbix and to do all development/deployment with a separate instance of zabbix_pipe. Once development/deployment is complete and all hosts, groups, templates, applications, and items have been created, switch to writing all data in “production mode” using the --fast pipe at /dev/zabbix.

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Logs

#debug, #error, #fatal, #info, #warn

Constructor Details

#initialize(settings) ⇒ AutoSender

Create a new sender with the given settings.

Parameters:

  • settings (Hash, Configliere::Param)
  • settings (String) —

    host the name of the Zabbix host to write data for

  • settings (String) —

    host_groups comma-separated names of Zabbix host groups the host should belong to

  • settings (String) —

    templates comma-separated names of Zabbix templates the host should belong to

  • settings (String) —

    applications comma-separated names of applications created items should be scoped under

  • settings (String) —

    server URL for the Zabbix server – not the URL for the Zabbix API

  • settings (String) —

    configuration_file path to a local Zabbix configuration file as used by the zabbix_sender utility

  • settings (true, false) —

    verbose be verbose during execution

  • settings (true, false) —

    fast auto-vivify (slow) or not (fast)

  • settings (String) —

    pipe path to a named pipe to be read from

  • settings (Fixnum) —

    pipe_read_sleep seconds to sleep after an empty read from the a named pipe

  • settings (Fixnum) —

    create_item_sleep seconds to sleep after creating a new item



152
153
154
155
156
157
158
159
160
161
162
163
164
 
# File 'lib/rubix/auto_sender.rb', line 152

def initialize settings
  @settings = settings
  confirm_settings
  if fast?
    info("Forwarding for #{settings['host']}...") if settings['verbose']
  else
    initialize_host_groups
    initialize_templates
    initialize_host
    initialize_applications
    info("Forwarding for #{host.name}...") if settings['verbose']
  end
end

Instance Attribute Details

#applications ⇒ Array<Rubix::Application>

Returns The applications used to create items.

Returns:



132
133
134
 
# File 'lib/rubix/auto_sender.rb', line 132

def applications
  @applications
end

#host ⇒ Rubix::Host

Returns the host the Sender will send data for.

Returns:

  • (Rubix::Host) —

    the host the Sender will send data for



123
124
125
 
# File 'lib/rubix/auto_sender.rb', line 123

def host
  @host
end

#host_groups ⇒ Array<Rubix::HostGroup>

Returns the hostgroups used to create this host.

Returns:



126
127
128
 
# File 'lib/rubix/auto_sender.rb', line 126

def host_groups
  @host_groups
end

#settings ⇒ Hash

Returns settings.

Returns:

  • (Hash) —

    settings



120
121
122
 
# File 'lib/rubix/auto_sender.rb', line 120

def settings
  @settings
end

#templates ⇒ Array<Rubix::Template>

Returns the templates used to create this host.

Returns:



129
130
131
 
# File 'lib/rubix/auto_sender.rb', line 129

def templates
  @templates
end

Instance Method Details

#auto_vivify? ⇒ true, false

Will this sender auto-vivify hosts, groups, items, &c.?

Returns:

  • (true, false) 


177
178
179
 
# File 'lib/rubix/auto_sender.rb', line 177

def auto_vivify?
  !fast?
end

#fast? ⇒ true, false

Is this sender running in ‘fast’ mode? If so, it will not auto-vivify any hosts, groups, items, &c.

Returns:

  • (true, false) 


170
171
172
 
# File 'lib/rubix/auto_sender.rb', line 170

def fast?
  settings['fast']
end

#process_line(line) ⇒ Object

Process a single line of text.

Parameters:

  • line (String) 


341
342
343
344
345
346
347
 
# File 'lib/rubix/auto_sender.rb', line 341

def process_line line
  if looks_like_json?(line)
    process_line_of_json_in_new_pipe(line)
  else
    process_line_of_tsv_in_this_pipe(line)
  end
end

#run ⇒ Object

Run this sender.

Will read from the correct source of data and exit the Ruby process once the source is consumed.



264
265
266
267
268
269
270
271
272
273
274
275
276
 
# File 'lib/rubix/auto_sender.rb', line 264

def run
  case
  when settings['pipe']
    process_pipe
  when settings.rest.size > 0
    settings.rest.each do |path|
      process_file(path)
    end
  else
    process_stdin
  end
  exit(0)
end