Hoardable gem version

Hoardable is an ActiveRecord extension for Ruby 3+, Rails 7+, and PostgreSQL that allows for versioning and soft-deletion of records through the use of uni-temporal inherited tables.

Temporal tables are a database design pattern where each row of a table contains data along with one or more time ranges. In the case of this gem, each database row has a time range that represents the row’s valid time range - hence "uni-temporal".

Table inheritance is a feature of PostgreSQL that allows a table to inherit all columns of a parent table. The descendant table’s schema will stay in sync with its parent. If a new column is added to or removed from the parent, the schema change is reflected on its descendants.

With these concepts combined, hoardable offers a model versioning and soft deletion system for Rails. Versions of records are stored in separate, inherited tables along with their valid time ranges and contextual data. Compared to other Rails-oriented versioning systems, this gem strives to be more explicit and obvious on the lower database level, while still familiar and convenient to use within Ruby on Rails.

👉 Documentation

Installation

Add this line to your application's Gemfile:

gem 'hoardable'

Run bundle install, and then run:

bin/rails g hoardable:install
bin/rails db:migrate

Model Installation

You must include Hoardable::Model into an ActiveRecord model that you would like to hoard versions of:

class Post < ActiveRecord::Base
  include Hoardable::Model
  ...
end

Then, run the generator command to create a database migration and migrate it:

bin/rails g hoardable:migration Post
bin/rails db:migrate

By default, it will guess the foreign key type for the _versions table based on the primary key of the model specified in the migration generator above. If you want/need to specify this explicitly, you can do so:

bin/rails g hoardable:migration Post --foreign-key-type uuid

Note: Creating an inherited table does not inherit the indexes from the parent table. If you need to query versions often, you should add appropriate indexes to the _versions tables. See here for more info.

Usage

Overview

Once you include Hoardable::Model into a model, it will dynamically generate a "Version" subclass of that model. As we continue our example from above:

Post #=> Post(id: integer, ..., hoardable_id: integer)
PostVersion #=> PostVersion(id: integer, ..., hoardable_id: integer, _data: jsonb, _during: tsrange, _event_uuid: uuid, _operation: enum)
Post.version_class #=> same as `PostVersion`

A Post now has_many :versions. With the default configuration, whenever an update or deletion of a post occurs, a version is created:

post = Post.create!(title: "Title")
post.versions.size # => 0
post.update!(title: "Revised Title")
post.reload.versions.size # => 1
post.versions.first.title # => "Title"
post.destroy!
post.trashed? # true
post.versions.size # => 2
Post.find(post.id) # raises ActiveRecord::RecordNotFound

Each PostVersion has access to the same attributes, relationships, and other model behavior that Post has, but as a read-only record:

post.versions.last.update!(title: "Rewrite history") #=> raises ActiveRecord::ReadOnlyRecord

If you ever need to revert to a specific version, you can call version.revert! on it.

post = Post.create!(title: "Title")
post.update!(title: "Whoops")
version = post.reload.versions.last
version.title # -> "Title"
version.revert!
post.title # => "Title"

If you would like to untrash a specific version of a record you deleted, you can call version.untrash! on it. This will re-insert the model in the parent class’s table with the original primary key.

post = Post.create!(title: "Title")
post.id # => 1
post.destroy!
post.versions.size # => 1
Post.find(post.id) # raises ActiveRecord::RecordNotFound
trashed_post = post.versions.trashed.last
trashed_post.id # => 2
trashed_post.untrash!
Post.find(post.id) # #<Post>

Note: You will notice above that both posts and post_versions pull from the same ID sequence. This allows for uniquely identifying source records and versions when results are mixed together. Both a source record and versions have an automatically managed hoardable_id that always represents the primary key value of the original source record.

Querying and Temporal Lookup

Including Hoardable::Model into your source model modifies its default scope to make sure you only query the parent table:

Post.where(state: :draft).to_sql # => SELECT posts.* FROM ONLY posts WHERE posts.status = 'draft'

Note: If you are executing raw SQL, you will need to include the ONLY keyword you see above to the select statement if you do not wish to return versions in the results. Learn more about table inheritance in the PostgreSQL documentation.

Since a PostVersion is an ActiveRecord class, you can query them like another model resource:

post.versions.where(state: :draft)

If you want to look-up the version of a record at a specific time, you can use the .at method:

post.at(1.day.ago) # => #<PostVersion>
# or you can use the scope on the version model class
post.versions.at(1.day.ago) # => #<PostVersion>
PostVersion.at(1.day.ago).find_by(hoardable_id: post.id) # => same as above

The source model class also has an .at method:

Post.at(1.day.ago) # => [#<Post>, #<Post>]

This will return an ActiveRecord scoped query of all Post and PostVersion records that were valid at that time, all cast as instances of Post.

There is also an at method on Hoardable itself for more complex and experimental temporal resource querying. See Relationships for more.

By default, hoardable will keep copies of records you have destroyed. You can query them specifically with:

PostVersion.trashed.where(user_id: user.id)
Post.version_class.trashed.where(user_id: user.id) # <- same as above

Note: A Version is not created upon initial source model creation. To accurately track the beginning of the first temporal period, you will need to ensure the source model table has a created_at timestamp column. If this is missing, an error will be raised.

Tracking Contextual Data

You’ll often want to track contextual data about the creation of a version. There are 2 options that can be provided for tracking this:

  • :whodunit - an identifier for who/what is responsible for creating the version
  • :meta - any other contextual information you’d like to store along with the version

This information is stored in a jsonb column. Each key’s value can be in the format of your choosing.

One convenient way to assign contextual data to these is by defining a proc in an initializer, i.e.:

# config/initializers/hoardable.rb
Hoardable.whodunit = -> { Current.user&.id }

# somewhere in your app code
Current.set(user: User.find(123)) do
  post.update!(status: :live)
  post.reload.versions.last.hoardable_whodunit # => 123
end

You can also set these context values manually as well:

Hoardable.meta = {note: "reverting due to accidental deletion"}
post.update!(title: "We’re back!")
Hoardable.meta = nil
post.reload.versions.last.hoardable_meta['note'] # => "reverting due to accidental deletion"

A more useful pattern would be to use Hoardable.with to set the context around a block. For example, you could have the following in your ApplicationController:

class ApplicationController < ActionController::Base
  around_action :use_hoardable_context

  private

  def use_hoardable_context
    Hoardable.with(whodunit: current_user.id, meta: {request_uuid: request.uuid}) do
      yield
    end
    # `Hoardable.whodunit` and `Hoardable.meta` are back to nil or their previously set values
  end
end

hoardable will also automatically capture the ActiveRecord changes hash, the operation that cause the version (update or delete), and it will also tag all versions created in the same database transaction with a shared and unique event_uuid for that transaction. These values are available as:

version.changes
version.hoardable_operation
version.hoardable_event_uuid

Model Callbacks

Sometimes you might want to do something with a version after it gets inserted to the database. You can access it in after_versioned callbacks on the source record as hoardable_version. These happen within ActiveRecord’s .save, which is enclosed in an ActiveRecord transaction.

There are also after_reverted and after_untrashed callbacks available as well, which are called on the source record after a version is reverted or untrashed.

class User
  include Hoardable::Model
  after_versioned :track_versioned_event
  after_reverted :track_reverted_event
  after_untrashed :track_untrashed_event

  private

  def track_versioned_event
    track_event(:user_versioned, hoardable_version)
  end

  def track_reverted_event
    track_event(:user_reverted, self)
  end

  def track_untrashed_event
    track_event(:user_untrashed, self)
  end
end

Configuration

The configurable options are:

Hoardable.enabled # => default true
Hoardable.version_updates # => default true
Hoardable.save_trash # => default true

Hoardable.enabled globally controls whether versions will be ever be created.

Hoardable.version_updates globally controls whether versions get created on record updates.

Hoardable.save_trash globally controls whether to create versions upon source record deletion. When this is set to false, all versions of a source record will be deleted when the record is destroyed.

If you would like to temporarily set a config setting, you can use Hoardable.with:

Hoardable.with(enabled: false) do
  post.update!(title: "replace title without creating a version")
end

You can also configure these settings per ActiveRecord class using hoardable_config:

class Comment < ActiveRecord::Base
  include Hoardable::Model
  hoardable_config version_updates: false
end

If you want to temporarily set the hoardable_config for a specific model, you can use with_hoardable_config:

Comment.with_hoardable_config(version_updates: true) do
  comment.update!(text: "Edited")
end

If a model-level option exists, it will use that. Otherwise, it will fall back to the global Hoardable config.

Relationships

Belongs To Trashable

Sometimes you’ll have a record that belongs to a parent record that you’ll trash. Now the child record’s foreign key will point to the non-existent trashed version of the parent. If you would like to have belongs_to resolve to the trashed parent model in this case, you can give it the option of trashable: true:

class Comment
  include Hoardable::Associations # <- This includes is not required if this model already includes `Hoardable::Model`
  belongs_to :post, trashable: true
end

Hoardable Has Many & Has One

Sometimes you'll have a Hoardable record that has_one or has_many other Hoardable records and you will want to know the state of both the parent record and the children at a certain point in time. You accomplish this by adding hoardable: true to the has_many relationship and using the Hoardable.at method:

class Post
  include Hoardable::Model
  has_many :comments, hoardable: true
end

class Comment
  include Hoardable::Model
end

post = Post.create!(title: "Title")
comment1 = post.comments.create!(body: "Comment")
comment2 = post.comments.create!(body: "Comment")
datetime = DateTime.current
comment2.destroy!
post.update!(title: "New Title")
post_id = post.id # 1

Hoardable.at(datetime) do
  post = Post.find(post_id)
  post.title # => "Title"
  post.comments.size # => 2
  post.id # => 2
  post.version? # => true
  post.hoardable_id # => 1
end

You’ll notice above that the post within the #at block is actually a temporal post_version, since it has been subsequently updated and has a different id - it is reified as a post for the purposes of your business logic (serialization, rendering views, exporting, etc). Don’t fret - you will not be able to commit any updates to the version, even though it is masquerading as a Post because a database trigger won’t allow it.

If you are ever unsure if a Hoardable record is a source record or a version, you can be sure by calling version? on it. If you want to get the true original source record ID, you can call hoardable_id.

Note: Hoardable.at is still very experimental and is potentially not performant for querying large data sets.

Cascading Untrashing

Sometimes you’ll trash something that has_many :children, dependent: :destroy and if you untrash the parent record, you’ll want to also untrash the children. Whenever a hoardable version is created in a database transaction, it will create or re-use a unique event UUID for the current database transaction and tag all versions created with it. That way, when you untrash! a record, you could find and untrash! records that were trashed with it:

class Post < ActiveRecord::Base
  include Hoardable::Model
  has_many :comments, hoardable: true, dependent: :destroy # `Comment` also includes `Hoardable::Model`

  after_untrashed do
    Comment
      .version_class
      .trashed
      .with_hoardable_event_uuid(hoardable_event_uuid)
      .find_each(&:untrash!)
  end
end

Action Text

Hoardable provides support for ActiveRecord models with has_rich_text. First, you must create a temporal table for ActionText::RichText:

bin/rails g hoardable:migration ActionText::RichText
bin/rails db:migrate

Then in your model include Hoardable::Model and provide the hoardable: true keyword to has_rich_text:

class Post < ActiveRecord::Base
  include Hoardable::Model # or `Hoardable::Associations` if you don't need `PostVersion`
  has_rich_text :content, hoardable: true
  # alternately, this could be `has_hoardable_rich_text :content`
end

Now the rich_text_content relationship will be managed as a Hoardable has_one relationship:

post = Post.create!(content: '<div>Hello World</div>')
datetime = DateTime.current
post.update!(content: '<div>Goodbye Cruel World</div>')
post.content.versions.size # => 1
post.content.to_plain_text # => 'Goodbye Cruel World'
Hoardable.at(datetime) do
  post.content.to_plain_text # => 'Hello World'
end

Known Gotchas

Rails Fixtures

Rails uses a method called disable_referential_integrity when inserting fixtures into the database. This disables PostgreSQL triggers, which Hoardable relies on for assigning hoardable_id from the primary key’s value. If you would still like to use fixtures, you must specify the primary key’s value and hoardable_id to the same identifier value in the fixture. This is not an issue with fixture replacement libraries like factory_bot or world_factory however.

Gem Comparison

paper_trail

paper_trail is maybe the most popular and fully featured gem in this space. It works for other database types than PostgeSQL. Bby default it stores all versions of all versioned models in a single versions table. It stores changes in a text, json, or jsonb column. In order to efficiently query the versions table, a jsonb column should be used, which can take up a lot of space to index. Unless you customize your configuration, all versions for all models types are in the same table which is inefficient if you are only interested in querying versions of a single model. By contrast, hoardable stores versions in smaller, isolated, inherited tables with the same database columns as their parents, which are more efficient for querying as well as auditing for truncating and dropping. The concept of a temporal timeframe does not exist for a single version since there is only a created_at timestamp.

audited

audited works in a similar manner as paper_trail. It stores all versions for all model types in a single table, you must opt into using jsonb as the column type to store "changes", in case you want to query them, and there is no concept of a temporal timeframe for a single version. It makes opinionated decisions about contextual data requirements and stores them as top level data types on the audited table.

discard

discard only covers soft-deletion. The act of "soft deleting" a record is only captured through the time-stamping of a discarded_at column on the records table. There is no other capturing of the event that caused the soft deletion unless you implement it yourself. Once the "discarded" record is restored, the previous "discarded" awareness is lost. Since "discarded" records exist in the same table as "undiscarded" records, you must explicitly omit the discarded records from queries across your app to keep them from leaking in.

paranoia

paranoia also only covers soft-deletion. In their README, they recommend using discard instead of paranoia because of the fact they override ActiveRecord’s delete and destroy methods. hoardable employs callbacks to create trashed versions instead of overriding methods. Otherwise, paranoia works similarly to discard in that it keeps deleted records in the same table and tags them with a deleted_at timestamp. No other information about the soft-deletion event is stored.

logidze

logidze is an interesting versioning alternative that leverages the power of PostgreSQL triggers. Instead of storing the previous versions or changes in a separate table, it stores them in a proprietary JSON format directly on the database row of the record itself. If does not support soft deletion.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/waymondo/hoardable.

License

The gem is available as open source under the terms of the MIT License.