Delayed Deltas for Thinking Sphinx (with Resque)
This code is HEAVILY borrowed from ts-delayed-delta.
This gem depends on the following gems: thinking-sphinx, resque, and resque-lock-timeout.
gem install ts-resque-delta
Add ts-resque-delta to your Gemfile file, with the rest of your gem dependencies:
gem 'ts-resque-delta', '1.1.1'
If you’re using Rails 3, the rake tasks will automatically be loaded by Rails. If you’re using Rails 2, add the following line to your Rakefile:
Add the delta property to each
define_index do # ... set_property :delta => :::: end
If you’ve never used delta indexes before, you’ll want to add the boolean column named
:delta to each model’s table (note, you must set the
:default value to
def self.up add_column :foos, :delta, :boolean, :default => true, :null => false end
Also, I highly recommend adding a MySQL index to the table of any model using delta indexes. The Sphinx indexer uses
WHERE table.delta = 1 whenever the delta indexer runs and
... = 0 whenever the normal indexer runs. Having the MySQL index on the delta column will generally be a win:
def self.up # ... add_index :foos, :delta end
Once you’ve got it all set up, all you need to do is make sure that the Resque worker is running. You can do this by specifying the
:ts_delta queue when running Resque:
QUEUE=ts_delta,other_queues rake resque:work
Additionally, ts-resque-delta will wrap thinking-sphinx’s
thinking_sphinx:reindex tasks with
thinking_sphinx:unlock_deltas. This will prevent the delta indexer from running at the same time as the main indexer.
Finally, ts-resque-delta also provides a rake task called
ts:si for short). This task, instead of locking all the delta indexes at once while the main indexer runs, will lock each delta index independently and sequentially. Thay way, your delta indexer can run while the main indexer is processing large core indexes.
Contributors (for ts-delayed-delta)
Original Contributors (for ts-delayed-delta)
- Pat Allan
- Ryan Schlesinger (Allowing installs as a plugin)
- Maximilian Schulz (Ensuring compatibility with Bundler)
- Edgars Beigarts (Adding intelligent description for tasks)
- Alexander Simonov (Explicit table definition)
Copyright (c) 2011 Aaron Gibralter, and released under an MIT Licence.