README

CSV Importable

Intelligently parse CSVs and display errors to your users.

Installation

Add this line to your application’s Gemfile:

gem 'csv_importable'

And then execute:

$ bundle

Usage

High level steps:

  1. Create an Import model: this model stores the file, status, and results of the import for the user to see.
  2. Create a RowImporter class: this class handles the logic surrounding how one row in the CSV should be imported.
  3. Create route(s), controller, and view(s) to allow your users to upload a CSV and return comprehensive, easy to understand error messages for your users to correct the CSV and re-upload.

Please note, it is also possible to implement an Importer class, which handles the logic surrounding how the entire file is imported. This is not usually needed though as our default Importer will take care of the heavy lifting.

If any errors happen during the upload process, they are recorded, stored, and ultimately displayed to the user for an opportunity to correct the CSV file.

Create Import Model

This model handles and stores the file, status, and results of the import for the user to see. By storing the file and results, we can process the import in the background when the file is too large to process real-time, and then email the user when the import is finished.

Note: if you’re not using Paperclip, don’t worry about implementing the file field. The important thing is that you implement a read_file method on your Import class so we know how to get the StringIO data for your CSV file.

$ rails g model Import status:string results:text type:string file:attachment should_replace:boolean
$ bundle exec rake db:migrate

Change the Import class to look something like below:

class Import < ActiveRecord::Base
  include CSVImportable::Importable

  def read_file
    # needs to return StringIO of file
    # for paperclip, use:
    # Paperclip.io_adapters.for(file).read
  end

  def after_async_complete
    # this is an optional hook for when an async import finishes
    # e.g. SiteMailer.import_completed(import).deliver_later
  end

  def save_to_db
    save
  end

  def big_file_threshold
    # max number of rows before processing with a background job
    # super returns the default of 10
    super
  end
end

And then create a subclass that should correspond to the specific importing task you are implementing. For example, if you are trying to import users from a CSV, you might implement a UserImport class which inherits from Import:

class UserImport < Import
  def row_importer_class
    UserRowImporter
  end
end

The only method that you need to define here is the row_importer_class, which tells csv_importable how to import each row in the CSV. Let’s take a look.

Create RowImporter Class

The RowImporter class handles the logic surrounding how one row in the CSV should be imported and added to the database. You need only (1) inherit from CSVImportable::CSVImporter and (2) implement the import_row method.

class UserRowImporter < CSVImportable::CSVImporter
  def import_row
    user = User.new
    user.email = pull_string('email', required: true)
    user.first_name = pull_string('first_name', required: true)
    user.last_name = pull_string('last_name', required: true)
    user.birthdate = pull_date('birthdate') # format: YYYYMMDD    
    user.salary = pull_float('salary')
  end
end

See that pull_string method? Let’s talk about that for a second…

Parsers

To assist you in getting data out of your CSV, we’ve implemented some basic parsers. These parsers will grab the raw data for the particular row/column and attempt to coerce it into the correct type (e.g. take string from CSV and convert to float).

If the parser fails to coerce the data properly, it will add an error message to the array of errors that your user receives after the import runs. These errors help the user fix the import file in order to try again.

  • pull_string
  • pull_boolean
  • pull_date
  • pull_float
  • pull_integer
  • pull_select

Basic syntax: pull_string(column_key, args) where column_key is the CSV header string for the column and args is a hash with the following defaults: { required: false }

Custom Parsers

To implement a custom

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/launchpadlab/csv_importable