Module: Wukong::Load

Includes:
Plugin
Defined in:
lib/wukong-load.rb,
lib/wukong-load/loader.rb,
lib/wukong-load/version.rb,
lib/wukong-load/load_runner.rb,
lib/wukong-load/loaders/sql.rb,
lib/wukong-load/loaders/kafka.rb,
lib/wukong-load/source_driver.rb,
lib/wukong-load/source_runner.rb,
lib/wukong-load/loaders/mongodb.rb,
lib/wukong-load/loaders/elasticsearch.rb

Overview

Loads data from the command-line into data stores.

Defined Under Namespace

Classes: ElasticsearchLoader, KafkaLoader, LoadRunner, Loader, MongoDBLoader, SQLLoader, SourceDriver, SourceRunner

Constant Summary collapse

VERSION = 
'0.1.1'

Class Method Summary collapse

Class Method Details

.boot(settings, dir) ⇒ Object

Boot Wukong-Load from the resolved settings in the given dir.

Parameters:

  • settings (Configliere::Param) —

    the resolved settings

  • dir (String) —

    the directory to boot in



30
31
 
# File 'lib/wukong-load.rb', line 30

def self.boot settings, dir
end

.configure(settings, program) ⇒ Object

Configure settings for Wukong-Load.

Will ensure that wu-load has the same settings as wu-local.

Parameters:

  • settings (Configliere::Param) —

    the settings to configure

  • program (String) —

    the currently executing program name



14
15
16
17
18
19
20
21
22
23
 
# File 'lib/wukong-load.rb', line 14

def self.configure settings, program
  case program
  when 'wu-load'
    settings.define :tcp_port, description: "Consume TCP requests on the given port instead of lines over STDIN", type: Integer, flag: 't'
  when 'wu-source'
    settings.define :per_sec,    description: "Number of events produced per second", type: Float
    settings.define :period,     description: "Number of seconds between events (overrides --per_sec)", type: Float
    settings.define :batch_size, description: "Trigger a finalize across the dataflow each time this many records are processed", type: Integer
  end
end