Class: GovukMessageQueueConsumer::Consumer

Inherits:
Object
  • Object
show all
Defined in:
lib/govuk_message_queue_consumer/consumer.rb

Defined Under Namespace

Classes: NullStatsd

Constant Summary collapse

NUMBER_OF_MESSAGES_TO_PREFETCH =

Only fetch one message at a time on the channel.

By default, queues will grab messages eagerly, which reduces latency. However, that also means that if multiple workers are running one worker can starve another of work. We’re not expecting a high throughput on this queue, and a small bit of latency isn’t a problem, so we fetch one at a time to share the work evenly.

1

Instance Method Summary collapse

Constructor Details

#initialize(queue_name:, processor:, rabbitmq_connection: Consumer.default_connection_from_env, statsd_client: NullStatsd.new, logger: Logger.new(STDERR)) ⇒ Consumer

Create a new consumer

Parameters:

  • queue_name (String) —

    Your queue name. This is specific to your application, and should already exist and have a binding via puppet

  • processor (Object) —

    An object that responds to ‘process`

  • rabbitmq_connection (Object) (defaults to: Consumer.default_connection_from_env) —

    A Bunny connection object derived from ‘Bunny.new`

  • statsd_client (Statsd) (defaults to: NullStatsd.new) —

    An instance of the Statsd class

  • logger (Object) (defaults to: Logger.new(STDERR)) —

    A Logger object for emitting errors (to stderr by default)



20
21
22
23
24
25
26
 
# File 'lib/govuk_message_queue_consumer/consumer.rb', line 20

def initialize(queue_name:, processor:, rabbitmq_connection: Consumer.default_connection_from_env, statsd_client: NullStatsd.new, logger: Logger.new(STDERR))
  @queue_name = queue_name
  @processor = processor
  @rabbitmq_connection = rabbitmq_connection
  @statsd_client = statsd_client
  @logger = logger
end

Instance Method Details

#run ⇒ Object 



28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
 
# File 'lib/govuk_message_queue_consumer/consumer.rb', line 28

def run
  @rabbitmq_connection.start

  queue.subscribe(block: true, manual_ack: true) do |delivery_info, headers, payload|
    begin
      message = Message.new(payload, headers, delivery_info)
      @statsd_client.increment("#{@queue_name}.started")
      processor_chain.process(message)
      @statsd_client.increment("#{@queue_name}.#{message.status}")
    rescue Exception => e
      @statsd_client.increment("#{@queue_name}.uncaught_exception")
      GovukError.notify(e) if defined?(GovukError)
      @logger.error "Uncaught exception in processor: \n\n #{e.class}: #{e.message}\n\n#{e.backtrace.join("\n")}"
      exit(1) # Ensure rabbitmq requeues outstanding messages
    end
  end
end