Module: DataMiner::ActiveRecordClassMethods

Defined in:
lib/data_miner/active_record_class_methods.rb

Overview

Class methods that are mixed into models (i.e. ActiveRecord::Base)

Instance Method Summary collapse

Instance Method Details

#data_miner(options = {}) { ... } ⇒ nil

Define a data miner script.

Examples:

Creating steps

class MyModel < ActiveRecord::Base
  data_miner do
    process [...]
    import [...]
    import [...yes, it's ok to have more than one import step...]
    sql [...]
    process [...]
    [...etc...]
  end
end

From the README

class Country < ActiveRecord::Base
  self.primary_key = 'iso_3166_code'
  data_miner do
    import("OpenGeoCode.org's Country Codes to Country Names list",
           :url => 'http://opengeocode.org/download/countrynames.txt',
           :format => :delimited,
           :delimiter => '; ',
           :headers => false,
           :skip => 22) do
      key   :iso_3166_code, :field_number => 0
      store :iso_3166_alpha_3_code, :field_number => 1
      store :iso_3166_numeric_code, :field_number => 2
      store :name, :field_number => 5
    end
  end
end

Parameters:

  • options (optional, Hash) (defaults to: {})

Options Hash (options):

  • :append (TrueClass, FalseClass) — default: false —

    Add steps to existing data miner script instead of starting from scratch.

Yields:

  • The block defining the steps.

Returns:

  • (nil)

See Also:



99
100
101
102
103
104
105
 
# File 'lib/data_miner/active_record_class_methods.rb', line 99

def data_miner(options = {}, &blk)
  unless options[:append]
    @data_miner_script = nil
  end
  data_miner_script.append_block blk
  nil
end

#data_miner_runs ⇒ ActiveRecord::Relation

Access to recordkeeping.

Returns:

  • (ActiveRecord::Relation) —

    Records of running the data miner script.



18
19
20
 
# File 'lib/data_miner/active_record_class_methods.rb', line 18

def data_miner_runs
  DataMiner::Run.scoped :conditions => { :model_name => name }
end

#data_miner_script ⇒ DataMiner::Script

Access this model’s script.

Returns:



9
10
11
12
13
 
# File 'lib/data_miner/active_record_class_methods.rb', line 9

def data_miner_script
  @data_miner_script || ::Thread.exclusive do
    @data_miner_script ||= DataMiner::Script.new(self)
  end
end

#run_data_miner! ⇒ DataMiner::Run

Run this model’s script.

Returns:



25
26
27
 
# File 'lib/data_miner/active_record_class_methods.rb', line 25

def run_data_miner!
  data_miner_script.start
end

#run_data_miner_on_parent_associations! ⇒ Array<DataMiner::Run>

Note:

Used extensively in github.com/brighterplanet/earth

Run the data miner scripts of parent associations. Useful for dependencies. Safe to call using process.

Examples:

Since Provinces depend on Countries, make sure Countries are data mined first

class Country < ActiveRecord::Base
  [...some data miner script...]
end
class Province < ActiveRecord::Base
  belongs_to :country
  data_miner do
    [...]
    process "make sure my dependencies have been loaded" do
      run_data_miner_on_parent_associations!
    end
    [...]
  end
end

Returns:



49
50
51
52
53
54
55
 
# File 'lib/data_miner/active_record_class_methods.rb', line 49

def run_data_miner_on_parent_associations!
  reflect_on_all_associations(:belongs_to).reject do |assoc|
    assoc.options[:polymorphic]
  end.map do |non_polymorphic_belongs_to_assoc|
    non_polymorphic_belongs_to_assoc.klass.run_data_miner!
  end
end