Class: DataMiner::Step::Import

Inherits:
DataMiner::Step show all
Defined in:
lib/data_miner/step/import.rb

Overview

A step that imports data from a remote source.

Create these by calling import inside a data_miner block.

See Also:

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods inherited from DataMiner::Step

#notify, #pos, #target?

Instance Attribute Details

#attributes ⇒ Array<DataMiner::Attribute> (readonly)

The mappings of local columns to remote data source fields.

Returns:



17
18
19
 
# File 'lib/data_miner/step/import.rb', line 17

def attributes
  @attributes
end

#description ⇒ String (readonly)

Description of what this step does.

Returns:

  • (String) 


21
22
23
 
# File 'lib/data_miner/step/import.rb', line 21

def description
  @description
end

#limit ⇒ Numeric (readonly)

Max number of rows to import.

Returns:

  • (Numeric) 


25
26
27
 
# File 'lib/data_miner/step/import.rb', line 25

def limit
  @limit
end

#random_skip ⇒ Numeric (readonly)

Number from zero to one representing what percentage of rows to skip. Defaults to 0, of course :)

Returns:

  • (Numeric) 


29
30
31
 
# File 'lib/data_miner/step/import.rb', line 29

def random_skip
  @random_skip
end

Instance Method Details

#key(attr_name, attr_options = {}) ⇒ nil

Store data into a model column AND use it as the key.

Enables idempotency. In other words, you can run the data miner script multiple times, get updated data, and not get duplicate rows.

Parameters:

  • attr_name (String) —

    The name of the local model column.

  • attr_options (optional, Hash) (defaults to: {}) —

    Options that will be passed to DataMiner::Attribute.new

Options Hash (attr_options):

  • anything (*) —

    Any option for DataMiner::Attribute.

Returns:

  • (nil)

See Also:



89
90
91
92
93
94
95
96
 
# File 'lib/data_miner/step/import.rb', line 89

def key(attr_name, attr_options = {})
  attr_name = attr_name.to_s
  if attributes.has_key? attr_name
    raise "You should only call store or key once for #{model.name}##{attr_name}"
  end
  @key = attr_name
  store attr_name, attr_options
end

#register(step) ⇒ Object 



111
112
113
114
115
 
# File 'lib/data_miner/step/import.rb', line 111

def register(step)
  if step.target?(self)
    listeners << step
  end
end

#store(attr_name, attr_options = {}, &blk) ⇒ nil

Store data into a model column.

Parameters:

  • attr_name (String) —

    The name of the local model column.

  • attr_options (optional, Hash) (defaults to: {}) —

    Options that will be passed to DataMiner::Attribute.new

Options Hash (attr_options):

  • anything (*) —

    Any option for DataMiner::Attribute.

Returns:

  • (nil)

See Also:



70
71
72
73
74
75
76
 
# File 'lib/data_miner/step/import.rb', line 70

def store(attr_name, attr_options = {}, &blk)
  attr_name = attr_name.to_s
  if attributes.has_key? attr_name
    raise "You should only call store or key once for #{model.name}##{attr_name}"
  end
  attributes[attr_name] = DataMiner::Attribute.new self, attr_name, attr_options, &blk
end