Module: Failbot

Extended by:
Failbot, Compat, SensitiveDataScrubber
Included in:
Failbot
Defined in:
lib/failbot.rb,
lib/failbot/compat.rb,
lib/failbot/version.rb,
lib/failbot/haystack.rb,
lib/failbot/backtrace.rb,
lib/failbot/exit_hook.rb,
lib/failbot/middleware.rb,
lib/failbot/file_backend.rb,
lib/failbot/http_backend.rb,
lib/failbot/json_backend.rb,
lib/failbot/memory_backend.rb,
lib/failbot/waiter_backend.rb,
lib/failbot/console_backend.rb,
lib/failbot/sensitive_data_scrubber.rb,
lib/failbot/default_backtrace_parser.rb,
lib/failbot/exception_format/haystack.rb,
lib/failbot/exception_format/structured.rb

Overview

This file exists so that the unhandled exception hook may easily be injected into programs that don’t register it themselves. It also provides a lightweight failbot interface that doesn’t bring in any other libraries until a report is made, which is useful for environments where boot time is important.

To use, set RUBYOPT or pass an -r argument to ruby:

RUBYOPT=rfailbot/exit_hook some-program.rb

Or:

ruby -rfailbot/exit_hook some-program.rb

Your program can also require this library instead of ‘failbot’ to minimize the amount of up-front processing required and automatically install the exit hook.

require 'failbot/exit_hook'

The ‘failbot’ lib is loaded in full the first time an actual report is made.

Defined Under Namespace

Modules: Compat, DefaultBacktraceParser, ExceptionFormat, SensitiveDataScrubber Classes: Backtrace, ConsoleBackend, FileBackend, HTTPBackend, Haystack, JSONBackend, MemoryBackend, Rescuer, WaiterBackend

Constant Summary collapse

EXCEPTION_DETAIL = 
'exception_detail'
MAXIMUM_CAUSE_DEPTH =

We’ll include this many nested Exception#cause objects in the needle context. We limit the number of objects to prevent excessive recursion and large needle contexts.

2
EXCEPTION_FORMATS =

Enumerates the available exception formats this gem supports. The original format and the default is :haystack and the newer format is :structured

{
  :haystack => ExceptionFormat::Haystack,
  :structured => ExceptionFormat::Structured
}
VERSION = 
"2.5.5"

Constants included from SensitiveDataScrubber

SensitiveDataScrubber::BASIC_AUTH_REGEX, SensitiveDataScrubber::FILTERED, SensitiveDataScrubber::MAX_DEPTH, SensitiveDataScrubber::QUERY_STRING_REGEX, SensitiveDataScrubber::SENSITIVE_KEYWORDS

Instance Attribute Summary collapse

Attributes included from Compat

#backend

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Compat

backend!, backend_name, cast, config, config_file, default_options, default_options=, environment, fail, haystack, raise_errors=, raise_errors?, report_errors=, report_errors?, setup_deprecated

Methods included from SensitiveDataScrubber

scrub, scrub_url, scrub_urls, transform_values

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(method, *args, &block) ⇒ Object

Tap into any other method invocation on the Failbot module (especially report) and lazy load and configure everything the first time.



75
76
77
78
79
 
# File 'lib/failbot/exit_hook.rb', line 75

def method_missing(method, *args, &block)
  return super if @failbot_loaded
  require 'failbot'
  send(method, *args, &block)
end

Instance Attribute Details

#already_reporting ⇒ Object

prevent recursive calls to Failbot.report!



50
51
52
 
# File 'lib/failbot.rb', line 50

def already_reporting
  @already_reporting
end

#backtrace_parser ⇒ Object (readonly)

Returns the value of attribute backtrace_parser.



111
112
113
 
# File 'lib/failbot.rb', line 111

def backtrace_parser
  @backtrace_parser
end

#instrumenter ⇒ Object

Public: Set an instrumenter to be called when exceptions are reported.

class CustomInstrumenter
  def instrument(name, payload = {})
    warn "Exception: #{payload["class"]}\n#{payload.inspect}"
  end
end

Failbot.instrumenter = CustomInstrumenter

The instrumenter must conform to the ‘ActiveSupport::Notifications` interface, which defines `#instrument` and accepts:

name - the String name of the event (e.g. “report.failbot”) payload - a Hash of the exception context.



47
48
49
 
# File 'lib/failbot.rb', line 47

def instrumenter
  @instrumenter
end

#source_root ⇒ Object

Returns the value of attribute source_root.



60
61
62
 
# File 'lib/failbot.rb', line 60

def source_root
  @source_root
end

Class Method Details

.backtrace_parser=(callable) ⇒ Object

Set a callable that is responsible for parsing and formatting ruby backtraces. This is only necessary to set if your app deals with exceptions that are manipulated to contain something other than actual stackframe strings in the format produced by ‘caller`. The argument passed must respond to `call` with an arity of 1. The callable expects to be passed Exception instances as its argument.



101
102
103
104
105
106
107
108
109
 
# File 'lib/failbot.rb', line 101

def self.backtrace_parser=(callable)
  unless callable.respond_to?(:call)
    raise ArgumentError, "backtrace_parser= passed #{callable.inspect}, which is not callable"
  end
  if callable.method(:call).arity != 1
    raise ArgumentError, "backtrace_parser= passed #{callable.inspect}, whose `#call` has arity =! 1"
  end
  @backtrace_parser = callable
end

.exception_classname_from_hash(hash) ⇒ Object 



121
122
123
 
# File 'lib/failbot.rb', line 121

def self.exception_classname_from_hash(hash)
  @exception_formatter.exception_classname_from_hash(hash)
end

.exception_format=(identifier) ⇒ Object

Set the current exception format.



85
86
87
88
89
 
# File 'lib/failbot.rb', line 85

def self.exception_format=(identifier)
  @exception_formatter = EXCEPTION_FORMATS.fetch(identifier) do
    fail ArgumentError, "#{identifier} is not an available exception_format (want one of #{EXCEPTION_FORMATS.keys})"
  end
end

.exception_message_from_hash(hash) ⇒ Object

Helpers needed to parse hashes included in e.g. Failbot.reports.



117
118
119
 
# File 'lib/failbot.rb', line 117

def self.exception_message_from_hash(hash)
  @exception_formatter.exception_message_from_hash(hash)
end

Instance Method Details

#before_report(&block) ⇒ Object

Public: your last chance to modify the context that is to be reported with an exception.

The key value pairs that are returned from your block will get squashed into the context, replacing the values of any keys that were already present.

Example:

Failbot.before_report do |exception, context|

# context is { "a" => 1, "b" => 2 }
{ :a => 0, :c => 3 }

end

context gets reported as { “a” => 0, “b” => “2”, “c” => 3 }



257
258
259
 
# File 'lib/failbot.rb', line 257

def before_report(&block)
  @before_report = block
end

#clear_before_report ⇒ Object

For tests



262
263
264
 
# File 'lib/failbot.rb', line 262

def clear_before_report
  @before_report = nil
end

#context ⇒ Object

Stack of context information to include in the next failbot report. These hashes are condensed down into one and included in the next report. Don’t mess with this structure directly - use the #push and #pop methods.



197
198
199
 
# File 'lib/failbot.rb', line 197

def context
  @context ||= [{'server' => hostname}]
end

#disable(&block) ⇒ Object

Public: Disable exception reporting. This is equivalent to calling ‘Failbot.setup(“FAILBOT_REPORT” => 0)`, but can be called after setup.

Failbot.disable do
  something_that_might_go_kaboom
end

block - an optional block to perform while reporting is disabled. If a block

is passed, reporting will be re-enabled after the block is called.



391
392
393
394
395
396
397
398
399
400
401
402
 
# File 'lib/failbot.rb', line 391

def disable(&block)
  original_report_errors = @report_errors
  @report_errors = false

  if block
    begin
      block.call
    ensure
      @report_errors = original_report_errors
    end
  end
end

#enable ⇒ Object

Public: Enable exception reporting. Reporting is enabled by default, but this can be called if it is explicitly disabled by calling ‘Failbot.disable` or setting `FAILBOT_REPORTING => “0”` in `Failbot.setup`.



407
408
409
 
# File 'lib/failbot.rb', line 407

def enable
  @report_errors = true
end

#exception_info(e) ⇒ Object

Extract exception info into a simple Hash.

e - The exception object to turn into a Hash.

Returns a Hash.



475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
 
# File 'lib/failbot.rb', line 475

def exception_info(e)
  res = @exception_formatter.call(e)

  if exception_context = (e.respond_to?(:failbot_context) && e.failbot_context)
    res.merge!(exception_context)
  end

  if original = (e.respond_to?(:original_exception) && e.original_exception)
    remote_backtrace  = []
    remote_backtrace << original.message
    if original.backtrace
      remote_backtrace.concat(Array(original.backtrace)[0,500])
    end
    res['remote_backtrace'] = remote_backtrace.join("\n")
  end

  res
end

#hostname ⇒ Object 



502
503
504
 
# File 'lib/failbot.rb', line 502

def hostname
  @hostname ||= Socket.gethostname
end

#install_unhandled_exception_hook! ⇒ Object

Installs an at_exit hook to report exceptions that raise all the way out of the stack and halt the interpreter. This is useful for catching boot time errors as well and even signal kills.

To use, call this method very early during the program’s boot to cover as much code as possible:

require 'failbot'
Failbot.install_unhandled_exception_hook!

Returns true when the hook was installed, nil when the hook had previously been installed by another component.



51
52
53
54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/failbot/exit_hook.rb', line 51

def install_unhandled_exception_hook!
  # only install the hook once, even when called from multiple locations
  return if @unhandled_exception_hook_installed

  # the $! is set when the interpreter is exiting due to an exception
  at_exit do
    boom = $!
    if boom && !@raise_errors && !boom.is_a?(SystemExit)
      report(boom, 'argv' => ([$0]+ARGV).join(" "), 'halting' => true)
    end
  end

  @unhandled_exception_hook_installed = true
end

#logger ⇒ Object 



494
495
496
 
# File 'lib/failbot.rb', line 494

def logger
  @logger ||= Logger.new($stderr)
end

#logger=(logger) ⇒ Object 



498
499
500
 
# File 'lib/failbot.rb', line 498

def logger=(logger)
  @logger = logger
end

#pop ⇒ Object

Remove the last info hash from the context stack.



216
217
218
 
# File 'lib/failbot.rb', line 216

def pop
  context.pop if context.size > 1
end

#push(info = {}) ⇒ Object

Add info to be sent in the next failbot report, should one occur.

info - Hash of name => value pairs to include in the exception report. block - When given, the info is removed from the current context after the

block is executed.

Returns the value returned by the block when given; otherwise, returns nil.



208
209
210
211
212
213
 
# File 'lib/failbot.rb', line 208

def push(info={})
  context.push(info)
  yield if block_given?
ensure
  pop if block_given?
end

#remove_from_report(key) ⇒ Object

Loops through the stack of contexts and deletes the given key if it exists.

key - Name of key to remove.

Examples

remove_from_report(:some_key)

remove_from_report("another_key")

Returns nothing.



236
237
238
239
240
241
 
# File 'lib/failbot.rb', line 236

def remove_from_report(key)
  context.each do |hash|
    hash.delete(key.to_s)
    hash.delete(key.to_sym)
  end
end

#report(e, other = {}) ⇒ Object

Public: Sends an exception to the exception tracking service along with a hash of custom attributes to be included with the report. When the raise_errors option is set, this method raises the exception instead of reporting to the exception tracking service.

e - The Exception object. Must respond to #message and #backtrace. other - Hash of additional attributes to include with the report.

Examples

begin
  my_code
rescue => e
  Failbot.report(e, :user => current_user)
end

Returns nothing.



317
318
319
320
321
322
323
324
 
# File 'lib/failbot.rb', line 317

def report(e, other = {})
  if @raise_errors
    squash_contexts(context, exception_info(e), other) # surface problems squashing
    raise e
  else
    report!(e, other)
  end
end

#report!(e, other = {}) ⇒ Object 



326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
 
# File 'lib/failbot.rb', line 326

def report!(e, other = {})
  return unless @report_errors

  if already_reporting
    logger.warn "FAILBOT: asked to report while reporting!" rescue nil
    logger.warn e.message rescue nil
    logger.warn e.backtrace.join("\n") rescue nil
    return
  end
  self.already_reporting = true

  begin
    data = squash_contexts(context, exception_info(e), other)

    if !data.has_key?("rollup")
      data = data.merge("rollup" => @rollup.call(e, data))
    end

    if @before_report
      data = squash_contexts(data, @before_report.call(e, data))
    end

    if @app_override
      data = data.merge("app" => @app_override)
    end

    data = scrub(sanitize(data))
  rescue Object => i
    log_failure("processing", data, e, i)
    self.already_reporting = false
    return
  end

  start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  instrumentation_data = {
    "report_status" => "error",
  }
  begin
    if @enable_timeout
      Timeout.timeout(@timeout_seconds) do
        backend.report(data)
      end
    else
      backend.report(data)
    end
    instrumentation_data["report_status"] = "success"
  rescue Object => i
    log_failure("reporting", data, e, i)
    instrumentation_data["exception_type"] = i.class.name
  ensure
    instrumentation_data["elapsed_ms"] = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time) * 1000).to_i
    instrument("report.failbot", data.merge(instrumentation_data)) rescue nil
    self.already_reporting = false
  end
end

#reports ⇒ Object

Public: exceptions that were reported. Only available when using the memory and file backends.

Returns an Array of exceptions data Hash.



415
416
417
 
# File 'lib/failbot.rb', line 415

def reports
  backend.reports
end

#reset! ⇒ Object

Reset the context stack to a pristine state.



221
222
223
 
# File 'lib/failbot.rb', line 221

def reset!
  @context = [context[0]]
end

#rollup(&block) ⇒ Object

Specify a custom block for calculating rollups. It should accept:

exception - The exception object context - The context hash

The block must return a String.

If a ‘rollup` attribute is supplied at the time of reporting, either via the `failbot_context` method on an exception, or passed to `Failbot.report`, it will be used as the rollup and this block will not be called.



276
277
278
 
# File 'lib/failbot.rb', line 276

def rollup(&block)
  @rollup = block
end

#sanitize(attrs) ⇒ Object 



440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
 
# File 'lib/failbot.rb', line 440

def sanitize(attrs)
  result = {}

  attrs.each do |key, value|
    result[key] =
      case value
      when Time
        value.iso8601
      when Date
        value.strftime("%F") # equivalent to %Y-%m-%d
      when Numeric
        value
      when String, true, false
        value.to_s
      when Array
        if key == EXCEPTION_DETAIL
          # special-casing for the exception_detail key, which is allowed to
          # be an array with a specific structure.
          value
        else
          value.inspect
        end
      else
        value.inspect
      end
  end

  result
end

#setup(settings = {}, default_context = {}) ⇒ Object

Shim into Failbot.setup and store config information off for the first time a real method is invoked.



126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
 
# File 'lib/failbot.rb', line 126

def setup(settings={}, default_context={})
  deprecated_settings = %w[
    backend host port haystack
    raise_errors
  ]

  if settings.empty? ||
    settings.keys.any? { |key| deprecated_settings.include?(key) }
    warn "%s Deprecated Failbot.setup usage. See %s for details." % [
      caller[0], "https://github.com/github/failbot"
    ]
    return setup_deprecated(settings)
  end

  if default_context.respond_to?(:to_hash) && !default_context.to_hash.empty?
    context[0] = default_context.to_hash
  end

  populate_context_from_settings(settings)
  @enable_timeout = false
  if settings.key?("FAILBOT_TIMEOUT_MS")
    @timeout_seconds = settings["FAILBOT_TIMEOUT_MS"].to_f / 1000
    @enable_timeout = (@timeout_seconds > 0.0)
  end

  @connect_timeout_seconds = nil
  if settings.key?("FAILBOT_CONNECT_TIMEOUT_MS")
    @connect_timeout_seconds = settings["FAILBOT_CONNECT_TIMEOUT_MS"].to_f / 1000
    # unset the value if it's not parsing to something valid
    @connect_timeout_seconds = nil unless @connect_timeout_seconds > 0
  end

  self.backend =
    case (name = settings["FAILBOT_BACKEND"])
    when "memory"
      Failbot::MemoryBackend.new
    when "waiter"
      Failbot::WaiterBackend.new
    when "file"
      Failbot::FileBackend.new(settings["FAILBOT_BACKEND_FILE_PATH"])
    when "http"
      Failbot::HTTPBackend.new(URI(settings["FAILBOT_HAYSTACK_URL"]), @connect_timeout_seconds)
    when 'json'
      Failbot::JSONBackend.new(settings["FAILBOT_BACKEND_JSON_HOST"], settings["FAILBOT_BACKEND_JSON_PORT"])
    when 'console'
      Failbot::ConsoleBackend.new
    else
      raise ArgumentError, "Unknown backend: #{name.inspect}"
    end

  @raise_errors  = !settings["FAILBOT_RAISE"].to_s.empty?
  @report_errors = settings["FAILBOT_REPORT"] != "0"

  # allows overriding the 'app' value to send to single haystack bucket.
  # used primarily on ghe.io.
  @app_override = settings["FAILBOT_APP_OVERRIDE"]

  # Support setting exception_format from ENV/settings
  if settings["FAILBOT_EXCEPTION_FORMAT"]
    self.exception_format = settings["FAILBOT_EXCEPTION_FORMAT"].to_sym
  end
end

#squash_contexts(*contexts_to_squash) ⇒ Object

Combines all context hashes into a single hash converting non-standard data types in values to strings, then combines the result with a custom info hash provided in the other argument.

other - Optional array of hashes to also squash in on top of the context

stack hashes.

Returns a Hash with all keys and values.



427
428
429
430
431
432
433
434
435
436
437
438
 
# File 'lib/failbot.rb', line 427

def squash_contexts(*contexts_to_squash)
  squashed = {}

  contexts_to_squash.flatten.each do |hash|
    hash.each do |key, value|
      value = (value.call rescue nil) if value.kind_of?(Proc)
      squashed[key.to_s] = value
    end
  end

  squashed
end

#use_default_rollup ⇒ Object 



294
295
296
 
# File 'lib/failbot.rb', line 294

def use_default_rollup
  rollup(&DEFAULT_ROLLUP)
end