Class: RightSupport::Net::RequestBalancer

Inherits:

Object

Object
RightSupport::Net::RequestBalancer

show all

Includes:: Log::Mixin

Defined in:: lib/right_support/net/request_balancer.rb

Overview

Utility class that allows network requests to be randomly distributed across a set of network endpoints. Generally used for REST requests by passing an Array of HTTP service endpoint URLs.

Note that this class also serves as a namespace for endpoint selection policies, which are classes that actually choose the next endpoint based on some criterion (round-robin, health of endpoint, response time, etc).

The balancer does not actually perform requests by itself, which makes this class usable for various network protocols, and potentially even for non- networking purposes. The block passed to #request does all the work; the balancer merely selects a suitable endpoint to pass to its block.

PLEASE NOTE that it is VERY IMPORTANT that the balancer is able to properly distinguish between fatal and non-fatal (retryable) errors. Before you pass a :fatal option to the RequestBalancer constructor, carefully examine its default list of fatal exceptions and default logic for deciding whether a given exception is fatal! There are some subtleties.

Constant Summary collapse

DEFAULT_RETRY_PROC =

Deprecated.

please do not refer to this constant; it will be removed in RightSupport 3.0

lambda do |ep, n|
  n < ep.size
end

FATAL_RUBY_EXCEPTIONS =

Deprecated.

please do not refer to this constant; it will be removed in RightSupport 3.0

Built-in Ruby exceptions that should be considered fatal. Normally one would be inclined to simply say RuntimeError or StandardError, but because gem authors frequently make unwise choices of exception base class, including these top-level base classes could cause us to falsely think that retryable exceptions are fatal.

A good example of this phenomenon is the rest-client gem, whose base exception class is derived from RuntimeError!!

[
  # Exceptions that indicate something is seriously wrong with the Ruby VM.
  NoMemoryError, SystemStackError, SignalException, SystemExit,
  ScriptError,
  # Subclasses of StandardError. We can't include the base class directly as
  # a fatal exception, because there are some retryable exceptions that derive
  # from StandardError.
  ArgumentError, IndexError, LocalJumpError, NameError, RangeError,
  RegexpError, ThreadError, TypeError, ZeroDivisionError
]

FATAL_TEST_EXCEPTIONS =

Deprecated.

please do not refer to this constant; it will be removed in RightSupport 3.0

[]

DEFAULT_FATAL_EXCEPTIONS =

Deprecated.

please do not refer to this constant; it will be removed in RightSupport 3.0

Well-considered exceptions that should count as fatal (non-retryable) by the balancer. Used by default, and if you provide a :fatal option to the balancer, you should probably consult this list in your overridden fatal determination!

FATAL_RUBY_EXCEPTIONS + FATAL_TEST_EXCEPTIONS

DEFAULT_FATAL_PROC =

Deprecated.

please do not refer to this constant; it will be removed in RightSupport 3.0

lambda do |e|
  if DEFAULT_FATAL_EXCEPTIONS.any? { |c| e.is_a?(c) }
    #Some Ruby builtin exceptions indicate program errors
    true
  elsif e.respond_to?(:http_code) && (e.http_code != nil)
    #RestClient and Net::HTTP exceptions all respond to http_code, allowing us
    #to decide based on the HTTP response code.
    #Any HTTP 3xx counts as fatal, in order to force the client to handle it
    #Any HTTP 4xx code EXCEPT 408 (Request Timeout) counts as fatal.
    (e.http_code >= 300 && e.http_code < 500) && (e.http_code != 408)
  else
    #Anything else counts as non-fatal
    false
  end
end

DEFAULT_HEALTH_CHECK_PROC =

Proc.new do |endpoint|
  true
end

DEFAULT_OPTIONS =

{
    :policy       => nil,
    :retry        => DEFAULT_RETRY_PROC,
    :fatal        => DEFAULT_FATAL_PROC,
    :on_exception => nil,
    :health_check => DEFAULT_HEALTH_CHECK_PROC
}

Constants included from Log::Mixin

Log::Mixin::Decorator, Log::Mixin::UNDELEGATED

Instance Attribute Summary collapse

#endpoints ⇒ Object readonly

Returns the value of attribute endpoints.

Class Method Summary collapse

.request(endpoints, options = {}, &block) ⇒ Object

Instance Method Summary collapse

#get_stats ⇒ Object

Provide an interface so one can query the RequestBalancer for statistics on its endpoints.
#initialize(endpoints, options = {}) ⇒ RequestBalancer constructor

Constructor.
#lookup_hostname(endpoint) ⇒ Object

Un-resolve an IP address.
#request ⇒ Object

Perform a request.
#resolved_endpoints ⇒ Array

Return the actual, potentially DNS-resolved endpoints that are used for requests.

Methods included from Log::Mixin

default_logger, default_logger=, included

Constructor Details

#initialize(endpoints, options = {}) ⇒ `RequestBalancer`

Constructor. Accepts a sequence of request endpoints which it shuffles randomly at creation time; however, the ordering of the endpoints does not change thereafter and the sequence is tried from the beginning for every request.

If you pass the :resolve option, then the list of endpoints is treated as a list of hostnames (or URLs containing hostnames) and the list is expanded out into a larger list with each hostname replaced by several entries, one for each of its IP addresses. If a single DNS hostname is associated with multiple A records, the :resolve option allows the balancer to treat each backing server as a distinct endpoint with its own health state, etc.

Parameters

endpoints(Array): a set of network endpoints (e.g. HTTP URLs) to be load-balanced

Options

retry: a Class, array of Class or decision Proc to determine whether to keep retrying; default is to try all endpoints
fatal: a Class, array of Class, or decision Proc to determine whether an exception is fatal and should not be retried
resolve(Integer): how often to re-resolve DNS hostnames of endpoints; default is nil (never resolve)
on_exception(Proc): notification hook that accepts three arguments: whether the exception is fatal, the exception itself,

and the endpoint for which the exception happened

health_check(Proc): callback that allows balancer to check an endpoint health; should raise an exception if the endpoint

is not healthy

on_health_change(Proc): callback that is made when the overall health of the endpoints transition to a different level;

its single argument contains the new minimum health level

# File 'lib/right_support/net/request_balancer.rb', line 164

def initialize(endpoints, options={})
  @options = DEFAULT_OPTIONS.merge(options)

  unless endpoints && !endpoints.empty?
    raise ArgumentError, "Must specify at least one endpoint"
  end

  @options[:policy] ||= RightSupport::Net::LB::RoundRobin
  @policy = @options[:policy]
  @policy = @policy.new(options) if @policy.is_a?(Class)

  unless test_policy_duck_type(@policy)
    raise ArgumentError, ":policy must be a class/object that responds to :next, :good and :bad"
  end

  unless test_callable_arity(options[:retry], 2)
    raise ArgumentError, ":retry callback must accept two parameters"
  end

  unless test_callable_arity(options[:fatal], 1)
    raise ArgumentError, ":fatal callback must accept one parameter"
  end

  unless test_callable_arity(options[:on_exception], 3, false)
    raise ArgumentError, ":on_exception callback must accept three parameters"
  end

  unless test_callable_arity(options[:health_check], 1, false)
    raise ArgumentError, ":health_check callback must accept one parameter"
  end

  unless test_callable_arity(options[:on_health_change], 1, false)
    raise ArgumentError, ":on_health_change callback must accept one parameter"
  end

  @endpoints = endpoints

  if @options[:resolve]
    # Perform initial DNS resolution
    resolve
  else
    # Use endpoints as-is
    @policy.set_endpoints(@endpoints)
  end
end

Instance Attribute Details

#endpoints ⇒ `Object` (readonly)

Returns the value of attribute endpoints.



125
126
127

# File 'lib/right_support/net/request_balancer.rb', line 125

def endpoints
  @endpoints
end

Class Method Details

.request(endpoints, options = {}, &block) ⇒ `Object`



135
136
137

# File 'lib/right_support/net/request_balancer.rb', line 135

def self.request(endpoints, options={}, &block)
  new(endpoints, options).request(&block)
end

Instance Method Details

#get_stats ⇒ `Object`

Provide an interface so one can query the RequestBalancer for statistics on its endpoints. Merely proxies the balancing policy’s get_stats method. If no method exists in the balancing policy, a hash of endpoints with “n/a” is returned.

Examples

A RequestBalancer created with endpoints [1,2,3,4,5] and using a HealthCheck balancing policy may return:

=> “yellow-3”, 1 => “red”, 2 => “yellow-1”, 3 => “green”, 4 => “yellow-2”

A RequestBalancer created with endpoints [1,2,3,4,5] and specifying no balancing policy or using the default RoundRobin balancing policy may return:

=> “n/a”, 1 => “n/a”, 3 => “n/a”

# File 'lib/right_support/net/request_balancer.rb', line 334

def get_stats
  stats = {}
  @endpoints.each { |endpoint| stats[endpoint] = 'n/a' }
  stats = @policy.get_stats if @policy.respond_to?(:get_stats)
  stats
end

#lookup_hostname(endpoint) ⇒ `Object`

Un-resolve an IP address.

Parameters

endpoint: a network endpoint (e.g. HTTP URL) to be un-resolved

Return

Return the first hostname that resolved to the IP (there should only ever be one)



217
218
219

# File 'lib/right_support/net/request_balancer.rb', line 217

def lookup_hostname(endpoint)
  @resolved_hostnames.select{ |k,v| v.addresses.include?(endpoint) }.shift[0]
end

#request ⇒ `Object`

Perform a request.

Block

This method requires a block, to which it yields in order to perform the actual network request. If the block raises an exception or provides nil, the balancer proceeds to try the next URL in the list.

Raise

ArgumentError: if a block isn’t supplied
NoResult: if every URL in the list times out or returns nil

Return

Return the first non-nil value provided by the block.

Raises:

(ArgumentError)

# File 'lib/right_support/net/request_balancer.rb', line 234

def request
  raise ArgumentError, "Must call this method with a block" unless block_given?

  resolve if need_resolve?

  exceptions = {}
  result     = nil
  complete   = false
  n          = 0

  loop do
    if n > 0
      do_retry = @options[:retry] || DEFAULT_RETRY_PROC
      do_retry = do_retry.call((@ips.nil? || @ips.empty?) ? @endpoints : @ips, n) if do_retry.respond_to?(:call)
      break if (do_retry.is_a?(Integer) && n >= do_retry) || [nil, false].include?(do_retry)
    end

    endpoint, need_health_check  = @policy.next
    break unless endpoint

    n += 1
    t0 = Time.now

    # Perform health check if necessary. Note that we guard this with a rescue, because the
    # health check may raise an exception and we want to log the exception info if this happens.
    if need_health_check
      begin
        unless @policy.health_check(endpoint)
          logger.error "RequestBalancer: health check failed to #{endpoint} because of non-true return value"
          next
        end
      rescue Exception => e
        logger.error "RequestBalancer: health check failed to #{endpoint} because of #{e.class.name}: #{e.message}"
        if fatal_exception?(e)
          # Fatal exceptions should still raise, even if only during a health check
          raise
        else
          # Nonfatal exceptions: keep on truckin'
          next
        end
      end

      logger.info "RequestBalancer: health check succeeded to #{endpoint}"
    end

    begin
      result   = yield(endpoint)
      @policy.good(endpoint, t0, Time.now)
      complete = true
      break
    rescue Exception => e
      if to_raise = handle_exception(endpoint, e, t0)
        raise(to_raise)
      else
        @policy.bad(endpoint, t0, Time.now)
        exceptions[endpoint] ||= []
        exceptions[endpoint] << e
      end
    end

  end

  return result if complete

  # Produce a summary message for the exception that gives a bit of detail
  msg = []
  stats = get_stats
  exceptions.each_pair do |endpoint, list|
    summary = []
    list.each { |e| summary << e.class }
    health = stats[endpoint] if stats[endpoint] != 'n/a'
    if @resolved_hostnames
      hostname = lookup_hostname(endpoint)
      msg << "'#{hostname}' (#{endpoint}#{", "+health if health}) => [#{summary.uniq.join(', ')}]"
    else
      msg << "'#{endpoint}' #{"("+health+")" if health} => [#{summary.uniq.join(', ')}]"
    end
  end
  message = "Request failed after #{n} tries to #{exceptions.keys.size} endpoints: (#{msg.join(', ')})"

  logger.error "RequestBalancer: #{message}"
  raise NoResult.new(message, exceptions)
end

#resolved_endpoints ⇒ `Array`

Return the actual, potentially DNS-resolved endpoints that are used for requests. If the balancer was constructed with :resolve=>false, return self.endpoints.

Returns:

(Array) —

collection of endpoints



131
132
133

# File 'lib/right_support/net/request_balancer.rb', line 131

def resolved_endpoints
  (@ips.nil? || @ips.empty?) ? @endpoints : @ips 
end

Class: RightSupport::Net::RequestBalancer

Overview

Constant Summary collapse

Constants included from Log::Mixin

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Log::Mixin

Constructor Details

#initialize(endpoints, options = {}) ⇒ RequestBalancer

Parameters

Options

Instance Attribute Details

#endpoints ⇒ Object (readonly)

Class Method Details

.request(endpoints, options = {}, &block) ⇒ Object

Instance Method Details

#get_stats ⇒ Object

#lookup_hostname(endpoint) ⇒ Object

Parameters

Return

#request ⇒ Object

Block

Raise

Return

#resolved_endpoints ⇒ Array

#initialize(endpoints, options = {}) ⇒ `RequestBalancer`

#endpoints ⇒ `Object` (readonly)

.request(endpoints, options = {}, &block) ⇒ `Object`

#get_stats ⇒ `Object`

#lookup_hostname(endpoint) ⇒ `Object`

#request ⇒ `Object`

#resolved_endpoints ⇒ `Array`