Method: Wukong::Script#initialize

Defined in:
lib/wukong/script.rb

#initialize(mapper, reducer = nil, extra_options = {}) ⇒ Script

Instantiate the Script with the Mapper and the Reducer class (each a Wukong::Streamer) it should call back.

Identity or External program as map or reduce

To use the identity reducer (‘cat’), instantiate your Script class with nil as the reducer class. (And similarly to use an identity mapper, supply nil for the mapper class.)

To use an external program as your reducer (mapper), subclass the reduce_command (map_command) method to return the full command line expression to call.

class MyMapper < Wukong::Streamer::Base
  # ... awesome stuff ...
end

class MyScript < Wukong::Script
  # prefix each unique line with the count of its occurrences.
  def reduce_command
    '/usr/bin/uniq -c'
  end
end
MyScript.new(MyMapper, nil).run



128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/wukong/script.rb', line 128

def initialize mapper, reducer=nil, extra_options={}
  Settings.resolve!
  @options = Settings
  options.merge! extra_options
  @mapper  = (case mapper  when Class then mapper.new  when nil then nil else mapper  ; end)
  @reducer = (case reducer when Class then reducer.new when nil then nil else reducer ; end)
  @output_path = options.rest.pop
  @input_paths = options.rest.reject(&:blank?)
  if (input_paths.blank? || output_path.blank?) && (not options[:dry_run]) && (not ['map', 'reduce'].include?(run_mode))
    raise "You need to specify a parsed input directory and a directory for output. Got #{ARGV.inspect}"
  end
end