Importer is a tool to help you with importing objects to your database from various sources.
can import from XML and CSV formats, but it's possible to add custom parsers
reports how many new objects got imported, how many objects was modified and how many objects were invalid
includes ActiveRecord, DataMapper and MongoMapper adapters
Install the gem
gem install importer
To use with Ruby < 1.9
gem install importer -v "~> 0.4.0"
Add to your model
# ActiveRecord class Product < ActiveRecord::Base include end # MongoMapper class Product include MongoMapper::Document include end # DataMapper class Product include DataMapper::Resource include end
And start importing
This will parse the file and import all products there are defined in it. This will return import summary which will tell you how many products were created, modified or invalid. Exact information about each product (detected attributes and errors) will be available in summary too.
You can create your own parser to import from sources other than XML or CSV files. Check the implementation of one of existing parsers to find out how to write your own. Then just pass parser class to import method:
Product.import(path_to_file, :parser => CustomParserClass)
You can also create your custom versions of Import and ImportedObject classes. A possible alternative version could be ActiveRecord Import and ImportedObject models that would save import summary to database for later inspection. Check the rdocs for these classes for more information. You force the importer to use a custom Import class with:
Product.import(path_to_file, :import => CustomImportClass)
Importer is smart enough to figure out whether it has to create a new
object or just update existing one during import. By default it's tries
to find existing object by detected id attribute. If the object is found
Importer updates it, otherwise a new object is created. You can change the
way how existing objects are searched for by overriding your model's
find_on_import class method.
class Product < ActiveRecord::Base include def find_on_import(import, attributes) find_by_name(attributes["custom_attribute"]) end end
The default way to build new objects or update existing ones is to merge
their attributes with detected ones. To provide custom building logic
override your model's
def merge_attributes_on_import(import, attributes) self.attributes = attributes self.imported_at = Time.now end
Note on Patches/Pull Requests
Fork the project.
Make your feature addition or bug fix.
Add tests for it. This is important so I don't break it in a future version unintentionally.
Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
Send me a pull request. Bonus points for topic branches.
Importer gem/plugin was extracted from an actual Ruby on Rails application. It probably lacks some features or needs some polishing. Feel free to contribute.
The gem is still under development, backward compatibility can not be guaranteed (at least until it reaches 1.0 stable version).
Copyright © 2010 Michal Szajbe. See LICENSE for details.