Class: GovukMessageQueueConsumer::Consumer

Inherits:
Object
  • Object
show all
Defined in:
lib/govuk_message_queue_consumer/consumer.rb

Direct Known Subclasses

BatchConsumer

Defined Under Namespace

Classes: NullStatsd

Constant Summary collapse

HANDLE_BATCHES = 
false
NUMBER_OF_MESSAGES_TO_PREFETCH =

Only fetch one message at a time on the channel.

By default, queues will grab messages eagerly, which reduces latency. However, that also means that if multiple workers are running one worker can starve another of work. We’re not expecting a high throughput on this queue, and a small bit of latency isn’t a problem, so we fetch one at a time to share the work evenly.

1

Instance Method Summary collapse

Constructor Details

#initialize(queue_name:, processor:, rabbitmq_connection: Consumer.default_connection_from_env, statsd_client: NullStatsd.new, logger: Logger.new(STDERR)) ⇒ Consumer

Create a new consumer

Parameters:

  • queue_name (String) —

    Your queue name. This is specific to your application, and should already exist and have a binding via puppet

  • processor (Object) —

    An object that responds to ‘process`

  • rabbitmq_connection (Object) (defaults to: Consumer.default_connection_from_env) —

    A Bunny connection object derived from ‘Bunny.new`

  • statsd_client (Statsd) (defaults to: NullStatsd.new) —

    An instance of the Statsd class

  • logger (Object) (defaults to: Logger.new(STDERR)) —

    A Logger object for emitting errors (to stderr by default)



21
22
23
24
25
26
27
 
# File 'lib/govuk_message_queue_consumer/consumer.rb', line 21

def initialize(queue_name:, processor:, rabbitmq_connection: Consumer.default_connection_from_env, statsd_client: NullStatsd.new, logger: Logger.new(STDERR))
  @queue_name = queue_name
  @processor = processor
  @rabbitmq_connection = rabbitmq_connection
  @statsd_client = statsd_client
  @logger = logger
end

Instance Method Details

#run ⇒ Object 



29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
 
# File 'lib/govuk_message_queue_consumer/consumer.rb', line 29

def run
  @rabbitmq_connection.start

  queue.subscribe(block: true, manual_ack: true) do |delivery_info, headers, payload|
    begin
      message = Message.new(payload, headers, delivery_info)
      @statsd_client.increment("#{@queue_name}.started")
      message_consumer.process(message)
      @statsd_client.increment("#{@queue_name}.#{message.status}")
    rescue Exception => e
      @statsd_client.increment("#{@queue_name}.uncaught_exception")
      GovukError.notify(e) if defined?(GovukError)
      @logger.error "Uncaught exception in processor: \n\n #{e.class}: #{e.message}\n\n#{e.backtrace.join("\n")}"
      exit(1) # Ensure rabbitmq requeues outstanding messages
    end
  end
end