Module: DataMiner::ActiveRecordClassMethods

Defined in:
lib/data_miner/active_record_class_methods.rb

Overview

Class methods that are mixed into models (i.e. ActiveRecord::Base)

Instance Method Summary collapse

Instance Method Details

#data_miner(options = {}) { ... } ⇒ nil

Define a data miner script.

Examples:

Creating steps

class MyModel < ActiveRecord::Base
  data_miner do
    process [...]
    import [...]
    import [...yes, it's ok to have more than one import step...]
    sql [...]
    process [...]
    [...etc...]
  end
end

From the README

class Country < ActiveRecord::Base
  self.primary_key = 'iso_3166_code'
  data_miner do
    import("OpenGeoCode.org's Country Codes to Country Names list",
           :url => 'http://opengeocode.org/download/countrynames.txt',
           :format => :delimited,
           :delimiter => '; ',
           :headers => false,
           :skip => 22) do
      key   :iso_3166_code, :field_number => 0
      store :iso_3166_alpha_3_code, :field_number => 1
      store :iso_3166_numeric_code, :field_number => 2
      store :name, :field_number => 5
    end
  end
end

Parameters:

  • options (optional, Hash) (defaults to: {})

Options Hash (options):

  • :append (TrueClass, FalseClass) — default: false —

    Add steps to existing data miner script instead of starting from scratch.

Yields:

  • The block defining the steps.

Returns:

  • (nil)

See Also:



93
94
95
96
97
98
99
100
 
# File 'lib/data_miner/active_record_class_methods.rb', line 93

def data_miner(options = {}, &blk)
  options = options.stringify_keys
  unless options['append']
    @data_miner_script = nil
  end
  data_miner_script.append_block blk
  nil
end

#data_miner_script ⇒ DataMiner::Script

Access this model’s script.

Returns:



9
10
11
12
13
 
# File 'lib/data_miner/active_record_class_methods.rb', line 9

def data_miner_script
  @data_miner_script || ::Thread.exclusive do
    @data_miner_script ||= DataMiner::Script.new(self)
  end
end

#run_data_miner! ⇒ Object

Run this model’s script.

Returns:

  • nil



18
19
20
 
# File 'lib/data_miner/active_record_class_methods.rb', line 18

def run_data_miner!
  data_miner_script.start
end

#run_data_miner_on_parent_associations! ⇒ Object

Note:

Used extensively in github.com/brighterplanet/earth

Run the data miner scripts of parent associations. Useful for dependencies. Safe to call using process.

Examples:

Since Provinces depend on Countries, make sure Countries are data mined first

class Country < ActiveRecord::Base
  [...some data miner script...]
end
class Province < ActiveRecord::Base
  belongs_to :country
  data_miner do
    [...]
    process "make sure my dependencies have been loaded" do
      run_data_miner_on_parent_associations!
    end
    [...]
  end
end

Returns:

  • nil



42
43
44
45
46
47
48
49
 
# File 'lib/data_miner/active_record_class_methods.rb', line 42

def run_data_miner_on_parent_associations!
  reflect_on_all_associations(:belongs_to).reject do |assoc|
    assoc.options['polymorphic']
  end.map do |non_polymorphic_belongs_to_assoc|
    non_polymorphic_belongs_to_assoc.klass.run_data_miner!
  end
  nil
end