mongar
Replicate data from MySQL, PostgreSQL, SQLite, etc to MongoDB!
Mongar will replicate data from ActiveRecord (or any data represented by a Ruby class) to MongoDB. For example, if you have MySQL data represented by ActiveRecord, you can easily replicate it to MongoDB.
Adding to a rails project
For now, you’ll have to manually add Mongar to your Rails project. We’ll work on a Railtie for Rails 3 soon.
Put your Mongar config into config/mongar.rb (See Example Config below)
Put the following code into your Rakefile
require 'mongar'
namespace :mongar do
desc "Run mongar replication"
task :run => :environment do
mongar_config = File.join(Rails.root, 'config', 'mongar.rb')
mongar = eval(File.read(mongar_config))
mongar.run
end
end
Run it with
rake mongar:run
Assumptions
The default configuration assumes that you are using ActiveRecord and that are you using a plugin similar to acts_as_paranoid that never deletes records from the database but only marks them as deleted with a deleted_at column.
You can customize the finder methods that Mongar uses to accommodate any data model or your own custom scopes.
Minimal example config
Given an ActiveRecord model call Domain with two attributes name and client_id:
Mongar.configure do
mongo :default do
database 'mydb'
end
replicate Domain do
column :name do
primary_index
end
column :client_id
end
end
Full example Config
Mongar.configure do
# Currently we only log to STDOUT, we have plans to use a
# plans to use a configurable logger.
# Valid log levels are :info and :debug
log_level :debug
mongo :default do
database 'mydb'
end
mongo :otherdb do
database 'myotherdb'
user 'mongouser'
password 'password'
host '127.0.0.1'
port 27017
# By default Mongar uses the 'statuses' collection to
# keep track of the last replicated times, you can specify
# a custom collection name
status_collection :stati
end
# Minimal replica config to replicate the Domain model
# to the 'domains' mongo collection on the mongo db defined
# in the :default config above
replicate Domain do
column :name do
primary_index
end
column :client_id
end
# Customized replica config to replicate the Client
# model to the 'customers' mongo collection on the db
# defined in the :otherdb config above as well as the
# default database
replicate Client => [{ :otherdb => 'clients' }, 'clients'] do
# By default, Mongar will try to determine the time on the
# backend database. The supported ActiveRecord database adapters
# are MysqlAdapter, Mysql2Adapter, and SQLServerAdapter.
# You can use a custom function to determine the time on Client's
# backend database
db_time_selector do
# this will run Client#get_db_time
get_db_time
end
# Items are never deleted
no_deleted_finder
# Customize your updated record finder. The block is
# run in the source class context
set_updated_finder do |last_replicated_date|
find(:all, :conditions => ['something > ?', last_replicated_date])
end
# Custom created item finder
set_created_finder do |last_replicated_date|
created_scope(last_replicated_date)
end
# Run a full refresh of all items in the database based
# a condition returned by the Proc
full_refresh :if => Proc.new do |last_replicated_date|
# class eval'ed code
any_changes_since?(last_replicated_date)
end
# You can also refresh every N seconds
# full_refresh :every => 3600
# Define your columns
column :id do
# The 'primary index' column is used to find items in the mongo collection
primary_index
end
column :tag do
# Run the procedure :downcase on the value of Client#tag
transform :downcase
end
column :employee_count do
# Run a block transformation on the value of Client#employee_count
transform do |value|
value.nil? ? 0 : value
end
end
end
end
Contributing to mongar
Find this project on Pivotal Tracker here: www.pivotaltracker.com/projects/434475
-
Check out the latest master to make sure the feature hasn’t been implemented or the bug hasn’t been fixed yet
-
Check out the issue tracker to make sure someone already hasn’t requested it and/or contributed it
-
Fork the project
-
Start a feature/bugfix branch
-
Commit and push until you are happy with your contribution
-
Make sure to add tests for it. This is important so I don’t break it in a future version unintentionally.
-
Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
Copyright
Copyright © 2011 Greenview Data, Inc. See LICENSE.txt for further details.