Class: RightSupport::Net::RequestBalancer

Inherits:
Object
  • Object
show all
Includes:
Log::Mixin
Defined in:
lib/right_support/net/request_balancer.rb

Overview

Utility class that allows network requests to be randomly distributed across a set of network endpoints. Generally used for REST requests by passing an Array of HTTP service endpoint URLs.

Note that this class also serves as a namespace for endpoint selection policies, which are classes that actually choose the next endpoint based on some criterion (round-robin, health of endpoint, response time, etc).

The balancer does not actually perform requests by itself, which makes this class usable for various network protocols, and potentially even for non- networking purposes. The block passed to #request does all the work; the balancer merely selects a suitable endpoint to pass to its block.

PLEASE NOTE that it is VERY IMPORTANT that the balancer is able to properly distinguish between fatal and non-fatal (retryable) errors. Before you pass a :fatal option to the RequestBalancer constructor, carefully examine its default list of fatal exceptions and default logic for deciding whether a given exception is fatal! There are some subtleties.

Constant Summary

DEFAULT_RETRY_PROC =
lambda do |ep, n|
  n < ep.size
end
DEFAULT_FATAL_EXCEPTIONS =

Built-in Ruby exceptions that should be considered fatal. Normally one would be inclined to simply say RuntimeError or StandardError, but because gem authors frequently make unwise choices of exception base class, including these top-level base classes could cause us to falsely think that retryable exceptions are fatal.

A good example of this phenomenon is the rest-client gem, whose base exception class is derived from RuntimeError!!

[
  NoMemoryError, SystemStackError, SignalException, SystemExit,
  ScriptError,
  #Subclasses of StandardError, which we can't mention directly
  ArgumentError, IndexError, LocalJumpError, NameError, RangeError,
  RegexpError, ThreadError, TypeError, ZeroDivisionError
]
DEFAULT_FATAL_PROC =
lambda do |e|
  if DEFAULT_FATAL_EXCEPTIONS.any? { |c| e.is_a?(c) }
    #Some Ruby builtin exceptions indicate program errors
    true
  elsif e.respond_to?(:http_code) && (e.http_code != nil)
    #RestClient's exceptions all respond to http_code, allowing us
    #to decide based on the HTTP response code.
    #Any HTTP 4xx code EXCEPT 408 (Request Timeout) counts as fatal.
    (e.http_code >= 400 && e.http_code < 500) && (e.http_code != 408)
  else
    #Anything else counts as non-fatal
    false
  end
end
DEFAULT_HEALTH_CHECK_PROC =
Proc.new do |endpoint|
  true
end
DEFAULT_OPTIONS =
{
    :policy       => nil,
    :retry        => DEFAULT_RETRY_PROC,
    :fatal        => DEFAULT_FATAL_PROC,
    :on_exception => nil,
    :health_check => DEFAULT_HEALTH_CHECK_PROC
}

Constants included from Log::Mixin

Log::Mixin::Decorator

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Log::Mixin

default_logger, default_logger=, included

Constructor Details

#initialize(endpoints, options = {}) ⇒ RequestBalancer

Constructor. Accepts a sequence of request endpoints which it shuffles randomly at creation time; however, the ordering of the endpoints does not change thereafter and the sequence is tried from the beginning for every request.

Parameters

endpoints(Array)

a set of network endpoints (e.g. HTTP URLs) to be load-balanced

Options

retry

a Class, array of Class or decision Proc to determine whether to keep retrying; default is to try all endpoints

fatal

a Class, array of Class, or decision Proc to determine whether an exception is fatal and should not be retried

on_exception(Proc)

notification hook that accepts three arguments: whether the exception is fatal, the exception itself,

and the endpoint for which the exception happened

health_check(Proc)

callback that allows balancer to check an endpoint health; should raise an exception if the endpoint

is not healthy

on_health_change(Proc)

callback that is made when the overall health of the endpoints transition to a different level;

its single argument contains the new minimum health level


94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
# File 'lib/right_support/net/request_balancer.rb', line 94

def initialize(endpoints, options={})
  @options = DEFAULT_OPTIONS.merge(options)

  unless endpoints && !endpoints.empty?
    raise ArgumentError, "Must specify at least one endpoint"
  end

  @options[:policy] ||= RightSupport::Net::Balancing::RoundRobin
  @policy = @options[:policy]
  @policy = @policy.new(endpoints, options) if @policy.is_a?(Class)
  unless test_policy_duck_type(@policy)
    raise ArgumentError, ":policy must be a class/object that responds to :next, :good and :bad"
  end

  unless test_callable_arity(options[:retry], 2)
    raise ArgumentError, ":retry callback must accept two parameters"
  end

  unless test_callable_arity(options[:fatal], 1)
    raise ArgumentError, ":fatal callback must accept one parameter"
  end

  unless test_callable_arity(options[:on_exception], 3, false)
    raise ArgumentError, ":on_exception callback must accept three parameters"
  end

  unless test_callable_arity(options[:health_check], 1, false)
    raise ArgumentError, ":health_check callback must accept one parameter"
  end

  unless test_callable_arity(options[:on_health_change], 1, false)
    raise ArgumentError, ":on_health_change callback must accept one parameter"
  end

  @endpoints = endpoints
end

Class Method Details

.request(endpoints, options = {}, &block) ⇒ Object



73
74
75
# File 'lib/right_support/net/request_balancer.rb', line 73

def self.request(endpoints, options={}, &block)
  new(endpoints, options).request(&block)
end

Instance Method Details

#get_statsObject

Provide an interface so one can query the RequestBalancer for statistics on it's endpoints. Merely proxies the balancing policy's get_stats method. If no method exists in the balancing policy, a hash of endpoints with "n/a" is returned.

Examples

A RequestBalancer created with endpoints [1,2,3,4,5] and using a HealthCheck balancing policy may return:

=> "yellow-3", 1 => "red", 2 => "yellow-1", 3 => "green", 4 => "yellow-2"

A RequestBalancer created with endpoints [1,2,3,4,5] and specifying no balancing policy or using the default RoundRobin balancing policy may return:

=> "n/a", 1 => "n/a", 3 => "n/a"



223
224
225
226
227
228
# File 'lib/right_support/net/request_balancer.rb', line 223

def get_stats
  stats = {}
  @endpoints.each { |endpoint| stats[endpoint] = 'n/a' }
  stats = @policy.get_stats if @policy.respond_to?(:get_stats)
  stats
end

#requestObject

Perform a request.

Block

This method requires a block, to which it yields in order to perform the actual network request. If the block raises an exception or provides nil, the balancer proceeds to try the next URL in the list.

Raise

ArgumentError

if a block isn't supplied

NoResult

if every URL in the list times out or returns nil

Return

Return the first non-nil value provided by the block.

Raises:

  • (ArgumentError)


144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
# File 'lib/right_support/net/request_balancer.rb', line 144

def request
  raise ArgumentError, "Must call this method with a block" unless block_given?

  exceptions = []
  result     = nil
  complete   = false
  n          = 0

  loop do
    if complete
      break
    else
      do_retry = @options[:retry] || DEFAULT_RETRY_PROC
      do_retry = do_retry.call(@endpoints, n) if do_retry.respond_to?(:call)
      break if (do_retry.is_a?(Integer) && n >= do_retry) || [nil, false].include?(do_retry)
    end

    endpoint, need_health_check  = @policy.next

    raise NoResult, "No endpoints are available" unless endpoint
    n += 1
    t0 = Time.now

    # Perform health check if necessary. Note that we guard this with a rescue, because the
    # health check may raise an exception and we want to log the exception info if this happens.
    if need_health_check
      begin
        unless @policy.health_check(endpoint)
          logger.error "RequestBalancer: health check failed to #{endpoint} because of non-true return value"
          next
        end
      rescue Exception => e
        logger.error "RequestBalancer: health check failed to #{endpoint} because of #{e.class.name}: #{e.message}"
        next
      end

      logger.info "RequestBalancer: health check succeeded to #{endpoint}"
    end

    begin
      result   = yield(endpoint)
      @policy.good(endpoint, t0, Time.now)
      complete = true
      break
    rescue Exception => e
      if to_raise = handle_exception(endpoint, e, t0)
        raise(to_raise)
      else
        @policy.bad(endpoint, t0, Time.now)
        exceptions << e
      end
    end

  end

  return result if complete

  exceptions = exceptions.map { |e| e.class.name }.uniq.join(', ')
  msg = "No available endpoints from #{@endpoints.inspect}! Exceptions: #{exceptions}"
  logger.error "RequestBalancer: #{msg}"
  raise NoResult, msg
end