Module: Resque::Plugins::ExponentialBackoff

Defined in:
lib/resque/plugins/exponential_backoff.rb

Overview

resque-exponential-backoff is a plugin to add retry/exponential backoff to your resque jobs.

Simply extend your module/class with this module:

require 'resque-exponential-backoff'

class DeliverWebHook
    extend Resque::Plugins::ExponentialBackoff

    def self.perform(url, hook_id, hmac_key)
        heavy_lifting
    end
end

Or do something more custom:

class DeliverWebHook
    extend Resque::Plugins::ExponentialBackoff

    # max number of attempts.
    @max_attempts = 4
    # retry delay in seconds.
    @backoff_strategy = [0, 60]

    # used to build redis key to store job attempts counter.
    def self.identifier(url, hook_id, hmac_key)
        "#{url}-#{hook_id}"
    end

    def self.perform(url, hook_id, hmac_key)
        heavy_lifting
    end
end

Instance Method Summary collapse

Instance Method Details

#after_perform_exponential_backoff(*args) ⇒ Object

Called after if #perform was successfully.

  • Delete attempts counter from redis.



124
125
126
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 124

def after_perform_exponential_backoff(*args)
    delete_attempts_counter(*args)
end

#attempts ⇒ Fixnum

Number of attempts so far to try and perform the job. Default value: 0

Returns:

  • (Fixnum) —

    number of attempts



77
78
79
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 77

def attempts
    @attempts ||= 0
end

#backoff_strategy ⇒ Object

Default backoff strategy.

1st retry  :  0 delay
2nd retry  :  1 minute
3rd retry  : 10 minutes
4th retry  :  1 hour
5th retry  :  3 hours
6th retry  :  6 hours

You can set your own backoff strategy in your job module/class:

Using this strategy, the first two retries will be immediate, the third and any subsequent retries will be delayed by 2 minutes.

Examples:

custom backoff strategy, in your class/module:

@backoff_strategy = [0, 0, 120]



109
110
111
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 109

def backoff_strategy
    @backoff_strategy ||= [0, 60, 600, 3600, 10_800, 21_600]
end

#before_perform_exponential_backoff(*args) ⇒ Object

Called before #perform.

  • Initialise or increment attempts counter.



116
117
118
119
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 116

def before_perform_exponential_backoff(*args)
    Resque.redis.setnx(key(*args), 0)         # default to 0 if not set.
    @attempts = Resque.redis.incr(key(*args)) # increment by 1.
end

#delete_attempts_counter(*args) ⇒ Object

Delete the attempts counter from redis, keepin it clean ;-)



142
143
144
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 142

def delete_attempts_counter(*args)
    Resque.redis.del(key(*args))
end

#identifier(*args) ⇒ String

This method is abstract.

You may override to implement a custom identifier, you should consider doing this if your job arguments are many/long or may not cleanly cleanly to strings.

Builds an identifier using the job arguments. This identifier is used as part of the redis key.

Parameters:

  • args (Array) —

    job arguments

Returns:

  • (String) —

    job identifier



50
51
52
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 50

def identifier(*args)
    args.join('-')
end

#key(*args) ⇒ String

Builds the redis key to be used for keeping state of the job attempts.

Returns:

  • (String) —

    redis key



59
60
61
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 59

def key(*args)
    ['exponential-backoff', name, identifier(*args)].compact.join(":")
end

#max_attempts ⇒ Fixnum

Maximum number of attempts we can use to successfully perform the job. Default value: 7

Returns:

  • (Fixnum) —

    number of attempts



68
69
70
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 68

def max_attempts
    @max_attempts ||= 7
end

#on_failure_exponential_backoff(exception, *args) ⇒ Object

Called if the job raises an exception.

  • Requeue the job if maximum attempts has not been reached.



131
132
133
134
135
136
137
138
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 131

def on_failure_exponential_backoff(exception, *args)
    if attempts >= max_attempts
        delete_attempts_counter(*args)
        return
    end
    
    requeue(*args)
end

#requeue(*args) ⇒ Object

Requeue the current job, immediately or delayed if #retry_delay_seconds returns grater then zero.

Parameters:

  • args (Array) —

    job arguments



151
152
153
154
155
156
157
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 151

def requeue(*args)
    if retry_delay_seconds > 0
        Resque.enqueue_in(retry_delay_seconds, self, *args)
    else
        Resque.enqueue(self, *args)
    end
end

#retry_delay_seconds ⇒ Number, #to_i

This method is abstract.

You may override to implement your own delay logic.

Returns the number of seconds to delay until the job is tried again. By default, this delay is taken from the #backoff_strategy.

Returns:

  • (Number, #to_i) —

    number of seconds to delay.



88
89
90
 
# File 'lib/resque/plugins/exponential_backoff.rb', line 88

def retry_delay_seconds
    backoff_strategy[attempts - 1] || backoff_strategy.last
end