Module: Hastur

Extended by:
Hastur
Included in:
Hastur
Defined in:
lib/hastur/api.rb,
lib/hastur/version.rb

Overview

Hastur API gem that allows services/apps to easily publish correct Hastur-commands to their local machine’s UDP sockets. Bare minimum for all JSON packets is to have :type key/values to map to a hastur message type, which the router uses for sink delivery.

Constant Summary collapse

SECS_2100 = 
4102444800
MILLI_SECS_2100 = 
4102444800000
MICRO_SECS_2100 = 
4102444800000000
NANO_SECS_2100 = 
4102444800000000000
SECS_1971 = 
31536000
MILLI_SECS_1971 = 
31536000000
MICRO_SECS_1971 = 
31536000000000
NANO_SECS_1971 = 
31536000000000000
PLUGIN_INTERVALS = 
[ :five_minutes, :thirty_minutes, :hourly, :daily, :monthly ]
START_OPTS = 
[
  :background_thread
]
VERSION = 
"1.2.8"

Class Attribute Summary collapse

Instance Method Summary collapse

Class Attribute Details

.mutex ⇒ Object

Returns the value of attribute mutex.



25
26
27
 
# File 'lib/hastur/api.rb', line 25

def mutex
  @mutex
end

Instance Method Details

#add_default_labels(new_default_labels) ⇒ Object

Add default labels which will be sent back with every Hastur message sent by this process. The labels will be sent back with the same constant value each time that is specified in the labels hash.

This is a useful way to send back information that won’t change during the run, or that will change only occasionally like resource usage, server information, deploy environment, etc. The same kind of information can be sent back using info_process(), so consider which way makes more sense for your case.

Parameters:

  • new_default_labels (Hash) —

    A hash of new labels to send.



201
202
203
204
205
 
# File 'lib/hastur/api.rb', line 201

def add_default_labels(new_default_labels)
  @default_labels ||= {}

  @default_labels.merge!
end

#app_name ⇒ String Also known as: application

Attempts to determine the application name, or uses an application-provided one, if set. In order, Hastur checks:

  • User-provided app name via Hastur.app_name=

  • HASTUR_APP_NAME environment variable

  • ::HASTUR_APP_NAME Ruby constant

  • Ecology.application, if set

  • File.basename($0)

Returns:

  • (String) —

    The application name, or best guess at same



153
154
155
156
157
158
159
160
161
162
163
164
165
 
# File 'lib/hastur/api.rb', line 153

def app_name
  return @app_name if @app_name

  return @app_name = ENV['HASTUR_APP_NAME'] if ENV['HASTUR_APP_NAME']

  top_level = ::HASTUR_APP_NAME rescue nil
  return @app_name = top_level if top_level

  eco = Ecology rescue nil
  return @app_name = Ecology.application if eco

  @app_name = File.basename $0
end

#app_name=(new_name) ⇒ Object Also known as: application=

Set the application name that Hastur registers as.

Parameters:

  • new_name (String) —

    The new application name.



173
174
175
176
177
178
179
180
181
182
183
184
 
# File 'lib/hastur/api.rb', line 173

def app_name=(new_name)
  old_name = @app_name

  @app_name = new_name

  if @process_registration_done
    err_str = "You changed the application name from #{old_name} to " +
      "#{new_name} after the process was registered!"
    STDERR.puts err_str
    Hastur.log err_str
  end
end

#background_thread? ⇒ Boolean

TODO:

Debug this.

Returns whether the background thread is currently running.

Returns:

  • (Boolean) 


102
103
104
 
# File 'lib/hastur/api.rb', line 102

def background_thread?
  @bg_thread && !@bg_thread.alive?
end

#counter(name, value = 1, timestamp = :now, labels = {}) ⇒ Object

Sends a ‘counter’ stat to Hastur. Counters are linear, and are sent as deltas (differences). Sending a value of 1 adds 1 to the counter.

Parameters:

  • name (String) —

    The counter name

  • value (Fixnum) (defaults to: 1) —

    Amount to increment the counter by

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



494
495
496
497
498
499
500
 
# File 'lib/hastur/api.rb', line 494

def counter(name, value=1, timestamp=:now, labels={})
  send_to_udp :type      => :counter,
              :name      => message_name_prefix + (name || ""),
              :value     => value,
              :timestamp => epoch_usec(timestamp),
              :labels    => default_labels.merge(labels)
end

#deliver_with(&block) ⇒ Object

Set delivery method to the given proc/block. The block is saved and called with each message to be sent. If no block is given or if this method is not called, the delivery method defaults to sending over the configured UDP port.



447
448
449
 
# File 'lib/hastur/api.rb', line 447

def deliver_with(&block)
  @__delivery_method__ = block
end

#epoch_usec(timestamp = Time.now) ⇒ Fixnum Also known as: timestamp

Best effort to make all timestamps be Hastur timestamps, 64 bit numbers that represent the total number of microseconds since Jan 1, 1970 at midnight UTC. Accepts second, millisecond or nanosecond timestamps and Ruby times. You can also give :now or nil for Time.now.

Parameters:

  • timestamp (defaults to: Time.now) —

    The timestamp as a Fixnum, Float or Time. Defaults to Time.now.

Returns:

  • (Fixnum) —

    Number of microseconds since Jan 1, 1970 midnight UTC

Raises:

  • RuntimeError Unable to validate timestamp format



116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
 
# File 'lib/hastur/api.rb', line 116

def epoch_usec(timestamp=Time.now)
  timestamp = Time.now if timestamp.nil? || timestamp == :now

  case timestamp
  when Time
    (timestamp.to_f*1000000).to_i
  when DateTime
    # Ruby 1.8.7 doesn't have to DateTime#to_time or DateTime#to_f method.
    # For right now, declare failure.
    raise "Ruby DateTime objects are not yet supported!"
  when SECS_1971..SECS_2100
    timestamp * 1000000
  when MILLI_SECS_1971..MILLI_SECS_2100
    timestamp * 1000
  when MICRO_SECS_1971..MICRO_SECS_2100
    timestamp
  when NANO_SECS_1971..NANO_SECS_2100
    timestamp / 1000
  else
    raise "Unable to validate timestamp: #{timestamp}"
  end
end

#event(name, subject = nil, body = nil, attn = [], timestamp = :now, labels = {}) ⇒ Object

Sends an event to Hastur. An event is high-priority and never buffered, and will be sent preferentially to stats or heartbeats. It includes an end-to-end acknowledgement to ensure arrival, but is expensive to store, send and query.

‘Attn’ is a mechanism to describe the system or component in which the event occurs and who would care about it. Obvious values to include in the array include user logins, email addresses, team names, and server, library or component names. This allows making searches like “what events should I worry about?” or “what events have recently occurred on the Rails server?”

Parameters:

  • name (String) —

    The name of the event (ex: “bad.log.line”)

  • subject (String) (defaults to: nil) —

    The subject or message for this specific event (ex “Got bad log line: @#$#@garbage@#$#@”)

  • body (String) (defaults to: nil) —

    An optional body with details of the event. A stack trace or email body would go here.

  • attn (Array) (defaults to: []) —

    The relevant components or teams for this event. Web hooks or email addresses would go here.

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



539
540
541
542
543
544
545
546
547
 
# File 'lib/hastur/api.rb', line 539

def event(name, subject=nil, body=nil, attn=[], timestamp=:now, labels={})
  send_to_udp :type => :event,
              :name => message_name_prefix + (name || ""),
              :subject => subject.to_s[0...3_072],
              :body => body.to_s[0...3_072],
              :attn => [ attn ].flatten,
              :timestamp => epoch_usec(timestamp),
              :labels  => default_labels.merge(labels)
end

#every(interval) { ... } ⇒ Object

Runs a block of code periodically every interval. Use this method to report statistics at a fixed time interval.

Parameters:

  • interval (Symbol) —

    How often to run. One of [:five_secs, :minute, :hour, :day]

Yields:

  • A block which will send Hastur messages, called periodically



718
719
720
721
722
723
724
725
726
727
728
729
730
731
 
# File 'lib/hastur/api.rb', line 718

def every(interval, &block)
  if @prevent_background_thread
    log("You called .every(), but background threads are specifically prevented.")
  end

  unless @intervals.include?(interval)
    raise "Interval must be one of these: #{@intervals}, you gave #{interval.inspect}"
  end

  # Don't add to existing array.  += will create a new array.  Then
  # when we save a reference to the old array and iterate through
  # it, it won't change midway.
  Hastur.mutex.synchronize { @scheduled_blocks[interval] += [ block ] }
end

#gauge(name, value, timestamp = :now, labels = {}) ⇒ Object

Sends a ‘gauge’ stat to Hastur. A gauge’s value may or may not be on a linear scale. It is sent as an exact value, not a difference.

Parameters:

  • name (String) —

    The mark name

  • value —

    The value of the gauge as a Fixnum or Float

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



512
513
514
515
516
517
518
 
# File 'lib/hastur/api.rb', line 512

def gauge(name, value, timestamp=:now, labels={})
  send_to_udp :type      => :gauge,
              :name      => message_name_prefix + (name || ""),
              :value     => value,
              :timestamp => epoch_usec(timestamp),
              :labels    => default_labels.merge(labels)
end

#heartbeat(name = "application.heartbeat", value = nil, timeout = nil, timestamp = :now, labels = {}) ⇒ Object

Sends a heartbeat to Hastur. A heartbeat is a periodic message which indicates that a host, application or service is currently running. It is higher priority than a statistic and should not be batched, but is lower priority than an event does not include an end-to-end acknowledgement.

Plugin results are sent as a heartbeat with the plugin’s name as the heartbeat name.

Parameters:

  • name (String) (defaults to: "application.heartbeat") —

    The name of the heartbeat.

  • value (defaults to: nil) —

    The value of the heartbeat as a Fixnum or Float

  • timeout (Float) (defaults to: nil) —

    How long in seconds to expect to wait, at maximum, before the next heartbeat. If this is nil, don’t worry if it doesn’t arrive.

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



687
688
689
690
691
692
693
 
# File 'lib/hastur/api.rb', line 687

def heartbeat(name="application.heartbeat", value=nil, timeout = nil, timestamp=:now, labels={})
  send_to_udp :name => message_name_prefix + (name || ""),
              :type => :hb_process,
              :value => value,
              :timestamp => epoch_usec(timestamp),
              :labels    => default_labels.merge(labels)
end

#info_agent(tag, data = {}, timestamp = :now, labels = {}) ⇒ Object

This sends back freeform data about the agent or host that Hastur is running on. Sample uses include what libraries or packages are installed and available, the total installed memory

Any number of these can be sent as information changes or is superceded. However, if information changes constantly or needs to be graphed or alerted on, send that separately as a metric or event. Info_agent messages are freeform and not readily separable or graphable.

Parameters:

  • tag (String) —

    The tag or title of this chunk of process info

  • data (Hash) (defaults to: {}) —

    The detailed data being sent

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



635
636
637
638
639
640
641
 
# File 'lib/hastur/api.rb', line 635

def info_agent(tag, data = {}, timestamp = :now, labels = {})
  send_to_udp :type      => :info_agent,
              :tag       => tag,
              :data      => data,
              :timestamp => epoch_usec(timestamp),
              :labels    => default_labels.merge(labels)
end

#info_process(tag, data = {}, timestamp = :now, labels = {}) ⇒ Object

Sends freeform process information to Hastur. This can be supplemental information about resources like memory, loaded gems, Ruby version, files open and whatnot. It can be additional configuration or deployment information like environment (dev/staging/prod), software or component version, etc. It can be information about the application as deployed, as run, or as it is currently running.

The default labels contain application name and process ID to match this information with the process registration and similar details.

Any number of these can be sent as information changes or is superceded. However, if information changes constantly or needs to be graphed or alerted on, send that separately as a metric or event. Info_process messages are freeform and not readily separable or graphable.

Parameters:

  • tag (String) —

    The tag or title of this chunk of process info

  • data (Hash) (defaults to: {}) —

    The detailed data being sent

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



611
612
613
614
615
616
617
 
# File 'lib/hastur/api.rb', line 611

def info_process(tag, data = {}, timestamp = :now, labels = {})
  send_to_udp :type      => :info_process,
              :tag       => tag,
              :data      => data,
              :timestamp => epoch_usec(timestamp),
              :labels    => default_labels.merge(labels)
end

#kill_background_thread ⇒ Object

This should ordinarily only be for testing. It kills the background thread so that automatic heartbeats and .every() blocks don’t happen. If you restart the background thread, all your .every() blocks go away, but the process heartbeat is restarted.



94
95
96
 
# File 'lib/hastur/api.rb', line 94

def kill_background_thread
  __kill_bg_thread__
end

#log(subject = nil, data = {}, timestamp = :now, labels = {}) ⇒ Object

Sends a log line to Hastur. A log line is of relatively low priority, comparable to stats, and is allowed to be buffered or batched while higher-priority data is sent first.

Severity can be included in the data field with the tag “severity” if desired.

Parameters:

  • subject (String) (defaults to: nil) —

    The subject or message for this specific log (ex “Got bad input: @#$#@garbage@#$#@”)

  • data (Hash) (defaults to: {}) —

    Additional JSON-able data to be sent

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



562
563
564
565
566
567
568
 
# File 'lib/hastur/api.rb', line 562

def log(subject=nil, data={}, timestamp=:now, labels={})
  send_to_udp :type => :log,
              :subject => subject.to_s[0...7_168],
              :data => data,
              :timestamp => epoch_usec(timestamp),
              :labels => default_labels.merge(labels)
end

#mark(name, value = nil, timestamp = :now, labels = {}) ⇒ Object

Sends a ‘mark’ stat to Hastur. A mark gives the time that an interesting event occurred even with no value attached. You can also use a mark to send back string-valued stats that might otherwise be guages – “Green”, “Yellow”, “Red” or similar.

It is different from a Hastur event because it happens at stat priority – it can be batched or slightly delayed, and doesn’t have an end-to-end acknowledgement included.

Parameters:

  • name (String) —

    The mark name

  • value (String) (defaults to: nil) —

    An optional string value

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



476
477
478
479
480
481
482
 
# File 'lib/hastur/api.rb', line 476

def mark(name, value = nil, timestamp=:now, labels={})
  send_to_udp :type      => :mark,
              :name      => message_name_prefix + (name || ""),
              :value     => value,
              :timestamp => epoch_usec(timestamp),
              :labels    => default_labels.merge(labels)
end

#message_name_prefix ⇒ Object 



264
265
266
 
# File 'lib/hastur/api.rb', line 264

def message_name_prefix
  @message_name_prefix || ""
end

#message_name_prefix=(value) ⇒ Object

Set a message-name prefix for all message types that have names. It will be prepended automatically for those message types’ names. A nil value will be treated as the empty string. Plugin names don’t count as message names for these purposes, and will not be prefixed.



260
261
262
 
# File 'lib/hastur/api.rb', line 260

def message_name_prefix=(value)
  @message_name_prefix = value
end

#no_background_thread! ⇒ Object

Prevents starting a background thread under any circumstances.



45
46
47
 
# File 'lib/hastur/api.rb', line 45

def no_background_thread!
  @prevent_background_thread = true
end

#register_plugin(name, plugin_path, plugin_args, plugin_interval, timestamp = :now, labels = {}) ⇒ Object

Sends a plugin registration to Hastur. A plugin is a program on the host machine which can be run to determine status of the machine, an application or anything else interesting.

This registration tells Hastur to begin scheduling runs of the plugin and report back on the resulting status codes or crashes.

Parameters:

  • name (String) —

    The name of the plugin, and of the heartbeat sent back

  • plugin_path (String) —

    The path on the local file system to this plugin executable

  • plugin_args (Array) —

    The array of arguments to pass to the plugin executable

  • plugin_interval (Symbol) —

    The interval to run the plugin. The scheduling will be slightly approximate. One of: PLUGIN_INTERVALS

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



657
658
659
660
661
662
663
664
665
666
667
668
 
# File 'lib/hastur/api.rb', line 657

def register_plugin(name, plugin_path, plugin_args, plugin_interval, timestamp=:now, labels={})
  unless PLUGIN_INTERVALS.include?(plugin_interval)
    raise "Interval must be one of: #{PLUGIN_INTERVALS.join(', ')}"
  end
  send_to_udp :type        => :reg_pluginv1,
              :plugin_path => plugin_path,
              :plugin_args => plugin_args,
              :interval    => plugin_interval,
              :plugin      => name,
              :timestamp   => epoch_usec(timestamp),
              :labels      => default_labels.merge(labels)
end

#register_process(name = app_name, data = {}, timestamp = :now, labels = {}) ⇒ Object

Sends a process registration to Hastur. This indicates that the process is currently running, and that heartbeats should be sent for some time afterward.

Parameters:

  • name (String) (defaults to: app_name) —

    The name of the application or best guess

  • data (Hash) (defaults to: {}) —

    The additional data to include with the registration

  • timestamp (defaults to: :now) —

    The timestamp as a Fixnum, Float, Time or :now

  • labels (Hash) (defaults to: {}) —

    Any additional data labels to send



580
581
582
583
584
585
 
# File 'lib/hastur/api.rb', line 580

def register_process(name = app_name, data = {}, timestamp = :now, labels = {})
  send_to_udp :type      => :reg_process,
              :data      => { "language" => "ruby", "version" => Hastur::VERSION }.merge(data),
              :timestamp => epoch_usec(timestamp),
              :labels    => default_labels.merge(labels)
end

#remove_default_label_names(*default_label_keys) ⇒ Object

Remove default labels which will be sent back with every Hastur message sent by this process. This cannot remove the three automatic defaults (application, pid, tid). Keys that have not been added cannot be removed, and so will be silently ignored (no exception will be raised).

Parameters:

  • default_label_keys (Array<String> or multiple strings) —

    Keys to stop sending



216
217
218
219
220
 
# File 'lib/hastur/api.rb', line 216

def remove_default_label_names(*default_label_keys)
  keys_to_remove = default_label_keys.flatten

  keys_to_remove.each { |key| @default_labels.delete(key) }
end

#reset ⇒ Object

Reset Hastur module for tests. This removes all settings and kills the background thread, resetting Hastur to its initial pre-start condition.



238
239
240
241
242
243
244
245
246
247
248
249
250
251
 
# File 'lib/hastur/api.rb', line 238

def reset
  __kill_bg_thread__
  @app_name = nil
  @prevent_background_thread = nil
  @process_registration_done = nil
  @udp_port = nil
  @__delivery_method__ = nil
  @scheduled_blocks = nil
  @last_time = nil
  @intervals = nil
  @interval_values = nil
  @default_labels = nil
  @message_name_prefix = nil
end

#reset_default_labels ⇒ Object

Reset the default labels which will be sent back with every Hastur message sent by this process. After this, only the automatic default labels (process ID, thread ID, application name) will be sent, plus of course the ones specified for the specific Hastur message call.



229
230
231
 
# File 'lib/hastur/api.rb', line 229

def reset_default_labels
  @default_labels = {}
end

#start(opts = {}) ⇒ Object

Start Hastur’s background thread and/or do process registration or neither, according to what options are set.

Parameters:

  • opts (Hash) (defaults to: {}) —

    The options for features

Options Hash (opts):

  • :background_thread (boolean) —

    Whether to start a background thread



60
61
62
63
64
65
66
67
68
69
70
71
 
# File 'lib/hastur/api.rb', line 60

def start(opts = {})
  bad_keys = opts.keys - START_OPTS
  raise "Unknown options to Hastur.start: #{bad_keys.join(", ")}!" unless bad_keys.empty?

  unless @prevent_background_thread ||
      (opts.has_key?(:background_thread) && !opts[:background_thread])
    start_background_thread
  end

  @process_registration_done = true
  register_process Hastur.app_name, {}
end

#start_background_thread ⇒ Object

Starts a background thread that will execute blocks of code every so often.



76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/hastur/api.rb', line 76

def start_background_thread
  if @prevent_background_thread
    raise "You can't start a background thread!  Somebody called .no_background_thread! already."
  end

  return if @bg_thread

  @intervals = [:five_secs, :minute, :hour, :day]
  @interval_values = [5, 60, 60*60, 60*60*24 ]
  __reset_bg_thread__
end

#time(name, timestamp = nil, labels = {}) ⇒ Object

Run the block and report its runtime back to Hastur as a gauge.

Examples:

Hastur.time "foo.bar" { fib 10 }
Hastur.time "foo.bar", Time.now, :from => "over there" do fib(100) end

Parameters:

  • name (String) —

    The name of the gauge.



703
704
705
706
707
708
709
 
# File 'lib/hastur/api.rb', line 703

def time(name, timestamp=nil, labels={})
  started = Time.now
  ret = yield
  ended = Time.now
  gauge name, ended - started, timestamp || started, labels
  ret
end

#udp_port=(new_port) ⇒ Object

Set the UDP port. Defaults to 8125

Parameters:

  • new_port (Fixnum) —

    The new port number.



456
457
458
 
# File 'lib/hastur/api.rb', line 456

def udp_port=(new_port)
  @udp_port = new_port
end