Class: Wukong::Load::MongoDBLoader

Inherits:
Loader
  • Object
show all
Defined in:
lib/wukong-load/loaders/mongodb.rb

Overview

Loads data into MongoDB.

Uses the 'mongo' gem to connect and write data.

Allows loading records into a given database and collection. Records can have fields _database and _collection which override the given database and collection on a per-record basis.

Records can have an _id field which indicates an update, not an insert.

The names of these fields within each record (_database, _collection, and _id) can be customized.

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods inherited from Loader

#process

Instance Attribute Details

#client ⇒ Object

The Mongo::MongoClient we'll use for talking to MongoDB.



55
56
57
 
# File 'lib/wukong-load/loaders/mongodb.rb', line 55

def client
  @client
end

Instance Method Details

#collection_for(record) ⇒ Object

:nodoc:



100
101
102
 
# File 'lib/wukong-load/loaders/mongodb.rb', line 100

def collection_for record
  database_for(record)[collection_name_for(record)]
end

#collection_name_for(record) ⇒ Object

:nodoc:



110
111
112
 
# File 'lib/wukong-load/loaders/mongodb.rb', line 110

def collection_name_for record
  record[collection_field] || self.collection
end

#database_for(record) ⇒ Object

:nodoc:



95
96
97
 
# File 'lib/wukong-load/loaders/mongodb.rb', line 95

def database_for record
  client[database_name_for(record)]
end

#database_name_for(record) ⇒ Object

:nodoc:



105
106
107
 
# File 'lib/wukong-load/loaders/mongodb.rb', line 105

def database_name_for record
  record[database_field] || self.database
end

#id_for(record) ⇒ Object

:nodoc:



115
116
117
 
# File 'lib/wukong-load/loaders/mongodb.rb', line 115

def id_for record
  record[id_field]
end

#load(record) ⇒ Object

Load a single record into MongoDB.

If the record has an ID, we'll issue an update, otherwise an insert.

Parameters:

  • Hash (record) 


79
80
81
82
83
84
85
86
87
88
89
90
91
92
 
# File 'lib/wukong-load/loaders/mongodb.rb', line 79

def load record
  id = id_for(record)
  if id
    res = collection_for(record).update({:_id => id}, record, :upsert => true)
    if res['updatedExisting']
      log.info("Updated #{id}")
    else
      log.info("Inserted #{id}")
    end
  else
    res = collection_for(record).insert(record)
    log.info("Inserted #{res}")
  end
end

#setup ⇒ Object

Creates the client connection.



58
59
60
61
62
63
64
65
66
67
68
69
70
71
 
# File 'lib/wukong-load/loaders/mongodb.rb', line 58

def setup
  begin
    require 'mongo'
  rescue LoadError => e
    raise Error.new("Please ensure that the 'mongo' gem is installed and available (in your Gemfile)")
  end
  h = host.gsub(%r{^http://},'')
  log.debug("Connecting to MongoDB server at #{h}:#{port}...")
  begin
    self.client = Mongo::MongoClient.new(h, port)
  rescue => e
    raise Error.new(e.message)
  end
end