Module: MessageBus::Implementation

Included in:
MessageBus, Instance
Defined in:
lib/message_bus.rb

Overview

The main server-side interface to a message bus for the purposes of configuration, publishing and subscribing

Defined Under Namespace

Classes: Synchronizer

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#configHash<Symbol => Object> (readonly) Also known as: redis_config

Returns Configuration options hash.

Returns:

  • (Hash<Symbol => Object>)

    Configuration options hash



29
30
31
# File 'lib/message_bus.rb', line 29

def config
  @config
end

Instance Method Details

#after_forkvoid

This method returns an undefined value.

Performs routines that are necessary after a process fork, typically

triggered by a forking webserver. Performs whatever the backend requires
and ensures the server is listening for publications and running
scheduled tasks.


512
513
514
515
516
517
# File 'lib/message_bus.rb', line 512

def after_fork
  reliable_pub_sub.after_fork
  ensure_subscriber_thread
  # will ensure timer is running
  timer.queue {}
end

#allow_broadcast=(val) ⇒ void

This method returns an undefined value.

Parameters:

  • val (Boolean)

    whether or not to allow broadcasting (debugging)



252
253
254
# File 'lib/message_bus.rb', line 252

def allow_broadcast=(val)
  configure(allow_broadcast: val)
end

#allow_broadcast?Boolean

Returns whether or not broadcasting is allowed. If not explicitly set, defaults to false unless we’re in Rails test or development mode.

Returns:

  • (Boolean)

    whether or not broadcasting is allowed. If not explicitly set, defaults to false unless we’re in Rails test or development mode.



258
259
260
261
262
263
264
265
# File 'lib/message_bus.rb', line 258

def allow_broadcast?
  @config[:allow_broadcast] ||=
    if defined? ::Rails
      ::Rails.env.test? || ::Rails.env.development?
    else
      false
    end
end

#backendSymbol

Returns the name of the backend implementation configured.

Returns:

  • (Symbol)

    the name of the backend implementation configured



292
293
294
# File 'lib/message_bus.rb', line 292

def backend
  @config[:backend] || :redis
end

#backlog(channel = nil, last_id = nil, site_id = nil) ⇒ Array<MessageBus::Message>

Get messages from a channel backlog since the last ID specified, filtered by site

Parameters:

  • channel (String) (defaults to: nil)

    the name of the channel in question

  • last_id (#to_i) (defaults to: nil)

    the channel-specific ID of the last message that the caller received on the specified channel

  • site_id (String) (defaults to: nil)

    the ID of the site by which to filter

Returns:

  • (Array<MessageBus::Message>)

    all messages published to the specified channel since the specified last ID



451
452
453
454
455
456
457
458
459
460
461
462
463
# File 'lib/message_bus.rb', line 451

def backlog(channel = nil, last_id = nil, site_id = nil)
  old =
    if channel
      reliable_pub_sub.backlog(encode_channel_name(channel, site_id), last_id)
    else
      reliable_pub_sub.global_backlog(last_id)
    end

  old.each do |m|
    decode_message!(m)
  end
  old
end

#blocking_subscribe(channel = nil) {|message| ... } ⇒ void

This method returns an undefined value.

Subscribe to messages. Each message will be delivered by yielding to the passed block as soon as it is available. This will block until subscription is terminated.

Parameters:

  • channel (String, nil) (defaults to: nil)

    the name of the channel to which we should subscribe. If ‘nil`, messages on every channel will be provided.

Yields:

  • (message)

    a message-handler block

Yield Parameters:



371
372
373
374
375
376
377
# File 'lib/message_bus.rb', line 371

def blocking_subscribe(channel = nil, &blk)
  if channel
    reliable_pub_sub.subscribe(encode_channel_name(channel), &blk)
  else
    reliable_pub_sub.global_subscribe(&blk)
  end
end

#cache_assetsBoolean

Returns whether or not to cache static assets for the diagnostics pages.

Returns:

  • (Boolean)

    whether or not to cache static assets for the diagnostics pages



48
49
50
51
52
53
54
# File 'lib/message_bus.rb', line 48

def cache_assets
  if defined? @config[:cache_assets]
    @config[:cache_assets]
  else
    true
  end
end

#cache_assets=(val) ⇒ void

This method returns an undefined value.

Parameters:

  • val (Boolean)

    whether or not to cache static assets for the diagnostics pages



43
44
45
# File 'lib/message_bus.rb', line 43

def cache_assets=(val)
  configure(cache_assets: val)
end

#chunked_encoding_enabled=(val) ⇒ void

This method returns an undefined value.

Parameters:

  • val (Boolean)

    whether or not to enable chunked encoding



82
83
84
# File 'lib/message_bus.rb', line 82

def chunked_encoding_enabled=(val)
  configure(chunked_encoding_enabled: val)
end

#chunked_encoding_enabled?Boolean

Returns whether or not chunked encoding is enabled. If not explicitly set, defaults to true.

Returns:

  • (Boolean)

    whether or not chunked encoding is enabled. If not explicitly set, defaults to true.



76
77
78
# File 'lib/message_bus.rb', line 76

def chunked_encoding_enabled?
  @config[:chunked_encoding_enabled] == false ? false : true
end

#configure(config) ⇒ void

This method returns an undefined value.

Overrides existing configuration

Parameters:

  • config (Hash<Symbol => Object>)

    values to merge into existing config



172
173
174
# File 'lib/message_bus.rb', line 172

def configure(config)
  @config.merge!(config)
end

#destroyvoid

This method returns an undefined value.

Stops listening for publications and stops executing scheduled tasks. Mostly used in tests to detroy entire bus.



492
493
494
495
496
497
498
499
500
501
502
503
504
505
# File 'lib/message_bus.rb', line 492

def destroy
  return if @destroyed

  reliable_pub_sub.global_unsubscribe

  @mutex.synchronize do
    return if @destroyed

    @subscriptions ||= {}
    @destroyed = true
  end
  @subscriber_thread.join if @subscriber_thread
  timer.stop
end

#enable_diagnosticsvoid

This method returns an undefined value.

Enables diagnostics tracking



298
299
300
# File 'lib/message_bus.rb', line 298

def enable_diagnostics
  MessageBus::Diagnostics.enable(self)
end

#extra_response_headers_lookup {|env| ... } ⇒ void

This method returns an undefined value.

Yields:

  • (env)

    a routine to determine extra headers to be set on a subscriber response

Yield Parameters:

  • env (Rack::Request::Env)

    the subscriber request environment

Yield Returns:

  • (Hash<String => String>)

    the extra headers to set on the response



235
236
237
238
# File 'lib/message_bus.rb', line 235

def extra_response_headers_lookup(&blk)
  configure(extra_response_headers_lookup: blk) if blk
  @config[:extra_response_headers_lookup]
end

#global_backlog(last_id = nil) ⇒ Array<MessageBus::Message>

Get messages from the global backlog since the last ID specified

Parameters:

  • last_id (#to_i) (defaults to: nil)

    the global ID of the last message that the caller received

Returns:

  • (Array<MessageBus::Message>)

    all messages published on any channel since the specified last ID



440
441
442
# File 'lib/message_bus.rb', line 440

def global_backlog(last_id = nil)
  backlog(nil, last_id)
end

#group_ids_lookup {|env| ... } ⇒ void

This method returns an undefined value.

Yields:

  • (env)

    a routine to determine the group IDs for a subscriber

Yield Parameters:

  • env (optional, Rack::Request::Env)

    the subscriber request environment

Yield Returns:

  • (optional, Array<String,Integer>)

    the group IDs for the subscriber



207
208
209
210
# File 'lib/message_bus.rb', line 207

def group_ids_lookup(&blk)
  configure(group_ids_lookup: blk) if blk
  @config[:group_ids_lookup]
end

#initializeObject



36
37
38
39
# File 'lib/message_bus.rb', line 36

def initialize
  @config = {}
  @mutex = Synchronizer.new
end

#is_admin_lookup {|env| ... } ⇒ void

This method returns an undefined value.

Yields:

  • (env)

    a routine to determine if a request comes from an admin user

Yield Parameters:

  • env (Rack::Request::Env)

    the subscriber request environment

Yield Returns:

  • (Boolean)

    whether or not the request is from an admin user



216
217
218
219
# File 'lib/message_bus.rb', line 216

def is_admin_lookup(&blk)
  configure(is_admin_lookup: blk) if blk
  @config[:is_admin_lookup]
end

#keepalive_intervalInteger

Returns the keepalive interval in seconds. If not explicitly set, defaults to ‘60`.

Returns:

  • (Integer)

    the keepalive interval in seconds. If not explicitly set, defaults to ‘60`.



553
554
555
# File 'lib/message_bus.rb', line 553

def keepalive_interval
  @config[:keepalive_interval] || 60
end

#keepalive_interval=(interval) ⇒ Object

Parameters:

  • interval (Integer)

    the keepalive interval in seconds. Set to 0 to disable; anything higher and a keepalive will run every N seconds. If it fails, the process is killed.



547
548
549
# File 'lib/message_bus.rb', line 547

def keepalive_interval=(interval)
  configure(keepalive_interval: interval)
end

#last_id(channel, site_id = nil) ⇒ Integer

Get the ID of the last message published on a channel, filtered by site

Parameters:

  • channel (String)

    the name of the channel in question

  • site_id (String) (defaults to: nil)

    the ID of the site by which to filter

Returns:

  • (Integer)

    the channel-specific ID of the last message published to the given channel



471
472
473
# File 'lib/message_bus.rb', line 471

def last_id(channel, site_id = nil)
  reliable_pub_sub.last_id(encode_channel_name(channel, site_id))
end

#last_message(channel) ⇒ MessageBus::Message

Get the last message published on a channel

Parameters:

  • channel (String)

    the name of the channel in question

Returns:



480
481
482
483
484
485
486
487
# File 'lib/message_bus.rb', line 480

def last_message(channel)
  if last_id = last_id(channel)
    messages = backlog(channel, last_id - 1)
    if messages
      messages[0]
    end
  end
end

#listening?Boolean

Returns whether or not the server is actively listening for publications on the bus.

Returns:

  • (Boolean)

    whether or not the server is actively listening for publications on the bus



521
522
523
# File 'lib/message_bus.rb', line 521

def listening?
  @subscriber_thread && @subscriber_thread.alive?
end

#local_subscribe(channel = nil, last_id = -1,) {|message| ... } ⇒ Proc

Subscribe to messages on a particular channel, filtered by the current site (@see #site_id_lookup). Each message since the last ID specified will be delivered by yielding to the passed block as soon as it is available. This will not block, but instead the callbacks will be executed asynchronously in a dedicated subscriber thread.

Parameters:

  • channel (String) (defaults to: nil)

    the name of the channel to which we should subscribe

  • last_id (#to_i) (defaults to: -1,)

    the channel-specific ID of the last message that the caller received on the specified channel

Yields:

  • (message)

    a message-handler block

Yield Parameters:

Returns:

  • (Proc)

    the callback block that will be executed



408
409
410
411
# File 'lib/message_bus.rb', line 408

def local_subscribe(channel = nil, last_id = -1, &blk)
  site_id = site_id_lookup.call if site_id_lookup && !global?(channel)
  subscribe_impl(channel, site_id, last_id, &blk)
end

#local_unsubscribe(channel = nil, &blk) ⇒ void

This method returns an undefined value.

Removes a subscription to a particular channel, filtered by the current site (@see #site_id_lookup).

Parameters:

  • channel (String) (defaults to: nil)

    the name of the channel from which we should unsubscribe

  • blk (Proc, nil)

    the callback which should be removed. If ‘nil`, removes all.



430
431
432
433
# File 'lib/message_bus.rb', line 430

def local_unsubscribe(channel = nil, &blk)
  site_id = site_id_lookup.call if site_id_lookup
  unsubscribe_impl(channel, site_id, &blk)
end

#loggerLogger

Returns the logger used by the bus. If not explicitly set, is configured to log to STDOUT at INFO level.

Returns:

  • (Logger)

    the logger used by the bus. If not explicitly set, is configured to log to STDOUT at INFO level.



64
65
66
67
68
69
70
71
72
# File 'lib/message_bus.rb', line 64

def logger
  return @config[:logger] if @config[:logger]

  require 'logger'
  logger = Logger.new(STDOUT)
  logger.level = Logger::INFO
  configure(logger: logger)
  logger
end

#logger=(logger) ⇒ void

This method returns an undefined value.

Parameters:

  • logger (Logger)

    a logger object to be used by the bus



58
59
60
# File 'lib/message_bus.rb', line 58

def logger=(logger)
  configure(logger: logger)
end

#long_polling_enabled=(val) ⇒ void

This method returns an undefined value.

Parameters:

  • val (Boolean)

    whether or not to enable long polling



94
95
96
# File 'lib/message_bus.rb', line 94

def long_polling_enabled=(val)
  configure(long_polling_enabled: val)
end

#long_polling_enabled?Boolean

Returns whether or not long polling is enabled. If not explicitly set, defaults to true.

Returns:

  • (Boolean)

    whether or not long polling is enabled. If not explicitly set, defaults to true.



88
89
90
# File 'lib/message_bus.rb', line 88

def long_polling_enabled?
  @config[:long_polling_enabled] == false ? false : true
end

#long_polling_intervalInteger

Returns the long-polling interval in milliseconds. If not explicitly set, defaults to 25,000.

Returns:

  • (Integer)

    the long-polling interval in milliseconds. If not explicitly set, defaults to 25,000.



148
149
150
# File 'lib/message_bus.rb', line 148

def long_polling_interval
  @config[:long_polling_interval] || 25 * 1000
end

#long_polling_interval=(millisecs) ⇒ void

This method returns an undefined value.

Parameters:

  • millisecs (Integer)

    the long-polling interval in milliseconds



142
143
144
# File 'lib/message_bus.rb', line 142

def long_polling_interval=(millisecs)
  configure(long_polling_interval: millisecs)
end

#max_active_clientsInteger

Returns The number of simultanuous clients we can service; will revert to polling if we are out of slots. Defaults to 1000 if not explicitly set.

Returns:

  • (Integer)

    The number of simultanuous clients we can service; will revert to polling if we are out of slots. Defaults to 1000 if not explicitly set.



108
109
110
# File 'lib/message_bus.rb', line 108

def max_active_clients
  @config[:max_active_clients] || 1000
end

#max_active_clients=(val) ⇒ void

This method returns an undefined value.

Parameters:

  • val (Integer)

    The number of simultanuous clients we can service; will revert to polling if we are out of slots



101
102
103
# File 'lib/message_bus.rb', line 101

def max_active_clients=(val)
  configure(max_active_clients: val)
end

#offvoid

This method returns an undefined value.

Disables publication to the bus



159
160
161
# File 'lib/message_bus.rb', line 159

def off
  @off = true
end

#off?Boolean

Returns whether the bus is disabled or not.

Returns:

  • (Boolean)

    whether the bus is disabled or not



153
154
155
# File 'lib/message_bus.rb', line 153

def off?
  @off
end

#onvoid

This method returns an undefined value.

Enables publication to the bus



165
166
167
# File 'lib/message_bus.rb', line 165

def on
  @destroyed = @off = false
end

#on_connect(&blk) ⇒ Object



240
241
242
243
# File 'lib/message_bus.rb', line 240

def on_connect(&blk)
  configure(on_connect: blk) if blk
  @config[:on_connect]
end

#on_disconnect(&blk) ⇒ Object



245
246
247
248
# File 'lib/message_bus.rb', line 245

def on_disconnect(&blk)
  configure(on_disconnect: blk) if blk
  @config[:on_disconnect]
end

#on_middleware_error {|env, e| ... } ⇒ void

This method returns an undefined value.

Yields:

  • (env, e)

    a routine to handle exceptions raised when handling a subscriber request

Yield Parameters:

  • env (Rack::Request::Env)

    the subscriber request environment

  • e (Exception)

    the exception that was raised

Yield Returns:

  • (optional, Array<(Integer,Hash,Array)>)

    a Rack response to be delivered



226
227
228
229
# File 'lib/message_bus.rb', line 226

def on_middleware_error(&blk)
  configure(on_middleware_error: blk) if blk
  @config[:on_middleware_error]
end

#publish(channel, data, opts = nil) ⇒ Integer

Publishes a message to a channel

Parameters:

  • channel (String)

    the name of the channel to which the message should be published

  • data (JSON)

    some data to publish to the channel. Must be an object that can be encoded as JSON

  • opts (Hash) (defaults to: nil)

Options Hash (opts):

  • :client_ids (Array<String>) — default: `nil`

    the unique client IDs to which the message should be available. If nil, available to all.

  • :user_ids (Array<String,Integer>) — default: `nil`

    the user IDs to which the message should be available. If nil, available to all.

  • :group_ids (Array<String,Integer>) — default: `nil`

    the group IDs to which the message should be available. If nil, available to all.

  • :site_id (String) — default: `nil`

    the site ID to scope the message to; used for hosting multiple applications or instances of an application against a single message_bus

  • :max_backlog_age (nil, Integer)

    the longest amount of time a message may live in a backlog before beging removed, in seconds

  • :max_backlog_size (nil, Integer)

    the largest permitted size (number of messages) for the channel backlog; beyond this capacity, old messages will be dropped

Returns:

  • (Integer)

    the channel-specific ID the message was given

Raises:



319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
# File 'lib/message_bus.rb', line 319

def publish(channel, data, opts = nil)
  return if @off

  @mutex.synchronize do
    raise ::MessageBus::BusDestroyed if @destroyed
  end

  user_ids = nil
  group_ids = nil
  client_ids = nil

  site_id = nil
  if opts
    user_ids = opts[:user_ids]
    group_ids = opts[:group_ids]
    client_ids = opts[:client_ids]
    site_id = opts[:site_id]
  end

  raise ::MessageBus::InvalidMessage if (user_ids || group_ids) && global?(channel)

  encoded_data = JSON.dump(
    data: data,
    user_ids: user_ids,
    group_ids: group_ids,
    client_ids: client_ids
  )

  channel_opts = nil

  if opts && ((age = opts[:max_backlog_age]) || (size = opts[:max_backlog_size]))
    channel_opts = {
      max_backlog_size: size,
      max_backlog_age: age
    }
  end

  encoded_channel_name = encode_channel_name(channel, site_id)
  reliable_pub_sub.publish(encoded_channel_name, encoded_data, channel_opts)
end

#rack_hijack_enabled=(val) ⇒ void

This method returns an undefined value.

Parameters:

  • val (Boolean)

    whether or not to enable Rack Hijack



136
137
138
# File 'lib/message_bus.rb', line 136

def rack_hijack_enabled=(val)
  configure(rack_hijack_enabled: val)
end

#rack_hijack_enabled?Boolean

Returns whether or not Rack Hijack is enabled. If not explicitly set, will default to true, unless we’re on Passenger without the ability to set the advertised_concurrency_level to 0.

Returns:

  • (Boolean)

    whether or not Rack Hijack is enabled. If not explicitly set, will default to true, unless we’re on Passenger without the ability to set the advertised_concurrency_level to 0.



115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
# File 'lib/message_bus.rb', line 115

def rack_hijack_enabled?
  if @config[:rack_hijack_enabled].nil?
    enable = true

    # without this switch passenger will explode
    # it will run out of connections after about 10
    if defined? PhusionPassenger
      enable = false
      if PhusionPassenger.respond_to? :advertised_concurrency_level
        PhusionPassenger.advertised_concurrency_level = 0
        enable = true
      end
    end
    configure(rack_hijack_enabled: enable)
  end

  @config[:rack_hijack_enabled]
end

#redis_config=(config) ⇒ void

This method returns an undefined value.

Overrides existing configuration, explicitly enabling the redis backend

Parameters:

  • config (Hash<Symbol => Object>)

    values to merge into existing config



179
180
181
# File 'lib/message_bus.rb', line 179

def redis_config=(config)
  configure(config.merge(backend: :redis))
end

#reliable_pub_subMessageBus::Backend::Base

Returns the configured backend. If not explicitly set, will be loaded based on the configuration provided.

Returns:

  • (MessageBus::Backend::Base)

    the configured backend. If not explicitly set, will be loaded based on the configuration provided.



275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
# File 'lib/message_bus.rb', line 275

def reliable_pub_sub
  @mutex.synchronize do
    return nil if @destroyed

    # Make sure logger is loaded before config is
    # passed to backend.
    logger

    @config[:reliable_pub_sub] ||= begin
      @config[:backend_options] ||= {}
      require "message_bus/backends/#{backend}"
      MessageBus::BACKENDS[backend].new @config
    end
  end
end

#reliable_pub_sub=(pub_sub) ⇒ void

This method returns an undefined value.

Parameters:

  • pub_sub (MessageBus::Backend::Base)

    a configured backend



269
270
271
# File 'lib/message_bus.rb', line 269

def reliable_pub_sub=(pub_sub)
  configure(reliable_pub_sub: pub_sub)
end

#reset!Object



526
527
528
# File 'lib/message_bus.rb', line 526

def reset!
  reliable_pub_sub.reset! if reliable_pub_sub
end

#site_id_lookup {|env| ... } ⇒ void

This method returns an undefined value.

Yields:

  • (env)

    a routine to determine the site ID for a subscriber

Yield Parameters:

  • env (optional, Rack::Request::Env)

    the subscriber request environment

Yield Returns:

  • (optional, String)

    the site ID for the subscriber



189
190
191
192
# File 'lib/message_bus.rb', line 189

def site_id_lookup(&blk)
  configure(site_id_lookup: blk) if blk
  @config[:site_id_lookup]
end

#subscribe(channel = nil, last_id = -1,) {|message| ... } ⇒ Proc

Subscribe to messages on a particular channel. Each message since the last ID specified will be delivered by yielding to the passed block as soon as it is available. This will not block, but instead the callbacks will be executed asynchronously in a dedicated subscriber thread.

Parameters:

  • channel (String) (defaults to: nil)

    the name of the channel to which we should subscribe

  • last_id (#to_i) (defaults to: -1,)

    the channel-specific ID of the last message that the caller received on the specified channel

Yields:

  • (message)

    a message-handler block

Yield Parameters:

Returns:

  • (Proc)

    the callback block that will be executed



391
392
393
# File 'lib/message_bus.rb', line 391

def subscribe(channel = nil, last_id = -1, &blk)
  subscribe_impl(channel, nil, last_id, &blk)
end

#timerMessageBus::TimerThread

Returns the timer thread used for triggering scheduled routines at specific times/intervals.

Returns:

  • (MessageBus::TimerThread)

    the timer thread used for triggering scheduled routines at specific times/intervals.



532
533
534
535
536
537
538
539
540
541
542
# File 'lib/message_bus.rb', line 532

def timer
  return @timer_thread if @timer_thread

  @timer_thread ||= begin
    t = MessageBus::TimerThread.new
    t.on_error do |e|
      logger.warn "Failed to process job: #{e} #{e.backtrace}"
    end
    t
  end
end

#unsubscribe(channel = nil, &blk) ⇒ void

This method returns an undefined value.

Removes a subscription to a particular channel.

Parameters:

  • channel (String) (defaults to: nil)

    the name of the channel from which we should unsubscribe

  • blk (Proc, nil)

    the callback which should be removed. If ‘nil`, removes all.



419
420
421
# File 'lib/message_bus.rb', line 419

def unsubscribe(channel = nil, &blk)
  unsubscribe_impl(channel, nil, &blk)
end

#user_id_lookup {|env| ... } ⇒ void

This method returns an undefined value.

Yields:

  • (env)

    a routine to determine the user ID for a subscriber (authenticate)

Yield Parameters:

  • env (optional, Rack::Request::Env)

    the subscriber request environment

Yield Returns:

  • (optional, String, Integer)

    the user ID for the subscriber



198
199
200
201
# File 'lib/message_bus.rb', line 198

def user_id_lookup(&blk)
  configure(user_id_lookup: blk) if blk
  @config[:user_id_lookup]
end