Class: MobSpawner

Inherits:
Object
  • Object
show all
Defined in:
lib/mob_spawner.rb

Overview

MobSpawner manages worker threads that can run arbitrary commands and report results. Unlike distributed queues, MobSpawner is self-contained and perfect for small batch scripts that need to run multiple independent jobs.

Defined Under Namespace

Classes: Command

Constant Summary collapse

VERSION = 
'1.0.0'

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(opts = {}) ⇒ MobSpawner #initialize(*commands) ⇒ MobSpawner

Creates a new spawner, use #run to run it.

Overloads:

  • #initialize(opts = {}) ⇒ MobSpawner

    Parameters:

    • opts (Hash{Symbol=>Object}) (defaults to: {}) —

      option data to be passed during initialization. Keys can be any attribute defined on this class, such as #num_workers, #commands, etc.

  • #initialize(*commands) ⇒ MobSpawner

    Parameters:

    • commands (Array<String>) —

      a list of commands to be run using default settings.



76
77
78
79
80
81
82
83
84
85
86
87
 
# File 'lib/mob_spawner.rb', line 76

def initialize(*commands)
  super()
  self.num_workers = 3
  self.commands = []
  self.before_callbacks = []
  self.after_callbacks = []
  if commands.size == 1 && commands.first.is_a?(Hash)
    setup_options(commands.first)
  else
    self.commands = commands.flatten
  end
end

Instance Attribute Details

#after_callbacks ⇒ Array<Proc>

Returns a list of callbacks to be called after each worker. Use #after_worker instead of setting the callbacks list directly.

Returns:

  • (Array<Proc>) —

    a list of callbacks to be called after each worker. Use #after_worker instead of setting the callbacks list directly.

See Also:



65
66
67
 
# File 'lib/mob_spawner.rb', line 65

def after_callbacks
  @after_callbacks
end

#before_callbacks ⇒ Array<Proc>

Returns a list of callbacks to be called before each worker. Use #before_worker instead of setting the callbacks list directly.

Returns:

  • (Array<Proc>) —

    a list of callbacks to be called before each worker. Use #before_worker instead of setting the callbacks list directly.

See Also:



60
61
62
 
# File 'lib/mob_spawner.rb', line 60

def before_callbacks
  @before_callbacks
end

#commands ⇒ Array<Command,String>

Returns a list of commands to be executed. Note that if a command is a String, it will eventually be converted into a Command object.

Returns:

  • (Array<Command,String>) —

    a list of commands to be executed. Note that if a command is a String, it will eventually be converted into a Command object.



55
56
57
 
# File 'lib/mob_spawner.rb', line 55

def commands
  @commands
end

#num_workers ⇒ Fixnum

Returns the number of workers to run, defaults to 3.

Returns:

  • (Fixnum) —

    the number of workers to run, defaults to 3.



50
51
52
 
# File 'lib/mob_spawner.rb', line 50

def num_workers
  @num_workers
end

Instance Method Details

#after_worker {|data| ... } ⇒ Object

Creates a callback that is executed after each worker is run.

Yields:

  • (data) —

    worker information

Yield Parameters:

  • data (Hash{Symbol=>Object}) —

    information about the worker thread. Valid keys are:

    • :worker - the worker number (starting from 1)

    • :command - the Command object about to be run

    • :output - all stdout and stderr output from the command

    • :status - the status code from the exited command

    • :exception - if a Ruby exception occurred during execution



143
144
145
 
# File 'lib/mob_spawner.rb', line 143

def after_worker(&block)
  after_callbacks << block
end

#before_worker {|data| ... } ⇒ Object

Creates a callback that is executed before each worker is run.

Yields:

  • (data) —

    worker information

Yield Parameters:

  • data (Hash{Symbol=>Object}) —

    information about the worker thread. Valid keys are:

    • :worker - the worker number (starting from 1)

    • :command - the Command object about to be run



128
129
130
 
# File 'lib/mob_spawner.rb', line 128

def before_worker(&block)
  before_callbacks << block
end

#run ⇒ Object

Runs the spawner, initializing all workers and running the commands.



90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
 
# File 'lib/mob_spawner.rb', line 90

def run
  self.commands = commands.map {|c| c.is_a?(Command) ? c : Command.new(c) }
  workers = []
  num_workers.times { workers << [] }
  divide_to_workers(workers, commands)
  threads = []
  workers.each_with_index do |worker, i|
    next if worker.size == 0
    threads << Thread.new do
      worker.each do |cmd|
        data = {:worker => i+1, :command => cmd}
        before_callbacks.each {|cb| cb.call(data) }
        begin
          output, status = Open3.capture2e(cmd.env, cmd.command)
          data.update(:output => output, :status => status)
        rescue => exc
          data.update(:exception => exc, :status => 256)
        end
        after_callbacks.each {|cb| cb.call(data) }
      end
    end
  end
  while threads.size > 0
    threads.dup.each do |thr|
      thr.join(0.1)
      threads.delete(thr) unless thr.alive?
    end
  end
end