Module: Hoodoo::ActiveRecord::ManuallyDated
- Defined in:
- lib/hoodoo/active/active_record/manually_dated.rb
Overview
Support mixin for models subclassed from ActiveRecord::Base providing as-per-API-standard dating support with services needing to know that dating is enabled and cooperate with this mixin’s API, rather than working automatically via database triggers as per Hoodoo::ActiveRecord::Dated. The latter is close to transparent for ActiveRecord-based code, but it involves very complex database queries that can have high cost and is tied into PostgreSQL.
Depends upon and auto-includes Hoodoo::ActiveRecord::Finder.
Overview
This mixin lets you record and retrieve the historical state of any given ActiveRecord model. This is achieved by adding two date/time columns to the model and using these to track the start (inclusive) and end (exclusive and always set to precisely DATE_MAXIMUM for “this is the ‘contemporary’ record) date/times for which a particular row is valid.
The majority of the functionality is implemented within class methods defined in module Hoodoo::ActiveRecord::ManuallyDated::ClassMethods.
Prerequisites
A table in the database needs to have various changes and additions to support manual dating. For these to be possible:
-
Your database table may not already have columns called
uuid
,effective_start
oreffective_end
. If it does, you’ll need to first migrate this to change the names and update any references in code. -
Your database table must have a column called
created_at
with the creation timestamp of a record which will become the time from which it is “visible” in historically-dated read queries. There can be noNULL
values in this column. -
Your database table must have a column called
updated_at
with a nonNULL
value. If this isn’t already present, migrate your data to add it, setting the initial value to the same ascreated_at
.
For data safety it is very strongly recommended that you add in database level non-null constraints on created_at
and updated_at
if you don’t have them already. The ActiveRecord change_column_null
method can be used in migrations to do this in a database-engine-neutral fashion.
Vital caveats
Since both the ‘contemporary’ and historic states of the model are all recorded in one table, anyone using this mechanism must ensure that (unless they specifically want to run a query across all of the representations) the mixin’s scoping methods are always used to target either current, or historic, or specifically-dated rows only.
With this mechanism in place, the id
attribute of the model is still a unique primary key AND THIS IS NO LONGER THE RESOURCE UUID. The UUID moves to a non-unique uuid
column. When rendering resources, YOU MUST USE THE uuid
COLUMN for the resource ID. This is a potentially serious gotcha and strong test coverage is advised! If you send back the wrong field value, it’ll look like a reasonable UUID but will not match any records at all through API-based interfaces, assuming Hoodoo::ActiveRecord::Finder is in use for read-based queries. The UUID will appear to refer to a non-existant resource.
-
The
id
column becomes a unique database primary key and of little to no interest whatsoever to a service or API callers. -
The
uuid
column becomes the non-unique resource UUID which is of great interest to a service and API callers. -
The
uuid
column is also the target for foreign keys with relationships between records, NOTid
. The relationships can only be used when scoped by date.
Accuracy
Time accuracy is intentionally limited, to aid database indices and help avoid clock accuracy differences across operating systems or datbase engines. Hoodoo::ActiveRecord::ManuallyDated::SECONDS_DECIMAL_PLACES describes the accuracy applicable.
If a record is, say, both created and then deleted within the accuracy window, then a dated query attempting to read the resource state from that (within-accuracy) identical time will return an undefined result. It might find the resource before it were deleted, or might not find the resource because it considers it to be no longer current. Of course, any dated query from outside the accuracy window will work as you would expect; only rapid changes in state within the accuracy window result in ambiguity.
Typical workflow
Having included the mixin, run any required migrations (see below) and declared manual dating as active inside your ActiveRecord::Base
subclass by calling Hoodoo::ActiveRecord::ManuallyDated::ClassMethods#manual_dating_enabled, you MUST include the ActiveRecord::Relation instances (scopes) inside any query chain used to read or write data.
Show and List
You might use Hoodoo::ActiveRecord::Finder::ClassMethods#list_in or Hoodoo::ActiveRecord::Finder::ClassMethods#acquire_in for list
or show
actions; such code changes from e.g.:
SomeModel.list_in( context )
…to:
SomeModel.manually_dated( context ).list_in( context )
Create
As with automatic dating - see Hoodoo::ActiveRecord::Dated - you should use method Hoodoo::ActiveRecord::Creator::ClassMethods.new_in to create new resource instances, to help ensure correct initial date setup and to help isolate your code from future functionality extensions/changes. An ActiveRecord before_create
filter deals with some of the “behind the scenes” maintenance but the initial acquisition of dating information from the prevailing request context only happens for you if you use Hoodoo::ActiveRecord::Creator::ClassMethods::new_in.
Update and Delete
You MUST NOT update or delete records using conventional ActiveRecord methods if you want to use manual dating to record state changes. Instead, use Hoodoo::ActiveRecord::ManuallyDated::ClassMethods#manually_dated_update_in or Hoodoo::ActiveRecord::ManuallyDated::ClassMethods#manually_dated_destruction_in. For example to update a model based on the context.request.body
data without changes to the item in context.request.ident
, handling “not found” or valiation error cases with the assumption that the Hoodoo::ActiveRecord::ErrorMapping mixin is in use, do this:
result = SomeModel.manually_dated_destruction_in( context )
if result.nil?
context.response.not_found( context.request.ident )
elsif result.adds_errors_to?( context.response.errors ) == false
rendered_data = render_model( result )
context.response.set_data( rendered_data )
end
See the documentation for the update/destroy methods mentioned above for information on overriding the identifier used to find the target record and the attribute data used for updates.
Rendering
When rendering, you MUST remember to set the resource’s id
field from the model’s uuid
field:
SomePresenter.render_in(
context,
model.attributes,
{
:uuid => model.uuid, # <-- ".uuid" - IMPORTANT!
:created_at => model.created_at
}
)
Associations
Generally, use of ActiveRecord associations is minimal in most services because there is an implied database-level coupling of resources and a temptation to use cross-table ActiveRecord mechanisms for things like relational UUID integrity checks, rather than inter-resource calls. Doing so couples resources together at the database rather than keeping them isolated purely by API, which is often a really bad idea. It is, however, sometimes necessary for best possible performance, or sometimes one complex resource may be represented by several models with relationships between them.
In such cases, remember to set foreign keys for relational declarations to a manually dated table via the uuid
column - e.g. go from this:
member.account_id = account.id
…to this:
member.account_id = account.uuid
…with the relational declarations in Member changing from:
belongs_to :account
…to:
belongs_to :account, :primary_key => :uuid
Required migrations
You must write an ActiveRecord migration for any table that wishes to use manual dating. The template below can handle multiple tables in one pass and can be rolled back safely IF no historic records have been added. Rollback becomes impossible once historic entries appear.
require 'hoodoo/active'
class ConvertToManualDating < ActiveRecord::Migration
# This example migration can handle multiple tables at once - e.g. pass an
# array of ":accounts, :members" if you were adding manual dating support to
# tables supporting an Account and Member ActiveRecord model.
#
TABLES_TO_CONVERT = [ :table_name, :another_table_name, ... ]
# This will come in handy later.
#
SQL_DATE_MAXIMUM = ActiveRecord::Base.connection.quoted_date( Hoodoo::ActiveRecord::ManuallyDated::DATE_MAXIMUM )
def up
# If you have any uniqueness constraints on this table, you'll need to
# remove them and re-add them with date-based scope. The main table will
# contain duplicated entries once historical versions of a row appear.
#
# remove_index :table_name, <index fields(s) or name: 'index name'>
#
# For example, suppose you had declared this index somewhere:
#
# add_index :accounts, :account_number, :unique => true
#
# Remove it with:
#
# remove_index :accounts, :account_number
TABLES_TO_CONVERT.each do | table |
add_column table, :effective_start, :datetime, :null => true # (initially, but see below)
add_column table, :effective_end, :datetime, :null => true # (initially, but see below)
add_column table, :uuid, :string, :limit => 32
add_index table, [ :effective_start, :effective_end ], :name => "index_#{ table }_start_end"
add_index table, [ :uuid, :effective_start, :effective_end ], :name => "index_#{ table }_uuid_start_end"
# We can't allow duplicate UUIDs. Here's how to correctly scope based on
# any 'contemporary' record, given its known fixed 'effective_end'.
#
ActiveRecord::Migration.add_index table,
:uuid,
:unique => true,
:name => "index_#{ table }_uuid_end_unique",
:where => "(effective_end = '#{ SQL_DATE_MAXIMUM }')"
# If there's any data in the table already, it can't have any historic
# entries. So, we want to set the UUID to the 'id' field's old value,
# but we can also leave the 'id' field as-is. New rows for historical
# entries will acquire a new value of 'id' via Hoodoo.
#
execute "UPDATE #{ table } SET uuid = id"
# This won't follow the date/time rounding described by manual dating
# but it's good enough for an initial migration.
#
execute "UPDATE #{ table } SET effective_start = created_at"
# Mark these records as contemporary/current.
#
execute "UPDATE #{ table } SET effective_end = '#{ ActiveRecord::Base.connection.quoted_date( Hoodoo::ActiveRecord::ManuallyDated::DATE_MAXIMUM ) }'"
# We couldn't add the UUID column with a not-null constraint until the
# above SQL had run to update any existing records with a value. Now we
# should put this back in, for rigour. Likewise for the start/end times.
#
change_column_null table, :uuid, false
change_column_null table, :effective_start, false
change_column_null table, :effective_end, false
end
# Now add back any indices dropped earlier, but add them back as a
# conditional index as shown earlier for the "uuid" column. For example,
# suppose you had declared this index somewhere:
#
# add_index :accounts, :account_number, :unique => true
#
# You need to have done "remove_index :accounts, :account_number" earlier;
# then now add the new equivalent. You may well find you have to give it a
# custom name to avoid hitting index name length limits in your database:
#
# ActiveRecord::Migration.add_index :accounts,
# :account_number,
# :unique => true,
# :name => "index_#{ table }_account_number_end_unique",
# :where => "(effective_end = '#{ SQL_DATE_MAXIMUM }')"
#
# You might want to perform more detailed analysis on your index
# requirements once manual dating is enabled, but the above is a good rule
# of thumb.
end
# This would fail if any historic entries now existed in the database,
# because primary key 'id' values would get set to non-unique 'uuid'
# values. This is intentional and required to avoid corruption; you
# cannot roll back once history entries accumulate.
#
def down
# Remove any indices added manually at the end of "up", for example:
#
# remove_index :accounts, :name => 'index_accounts_an_es_ee'
# remove_index :accounts, :name => 'index_accounts_an_ee'
TABLES_TO_CONVERT.each do | table |
remove_index table, :name => "index_#{ table }_id_end"
remove_index table, :name => "index_#{ table }_id_start_end"
remove_index table, :name => "index_#{ table }_start_end"
execute "UPDATE #{ table } SET id = uuid"
remove_column table, :uuid
remove_column table, :effective_end
remove_column table, :effective_start
end
# Add back any indexes you removed at the very start of "up", e.g.:
#
# add_index :accounts, :account_number, :unique => true
end
end
Defined Under Namespace
Modules: ClassMethods
Constant Summary collapse
- SECONDS_DECIMAL_PLACES =
Rounding resolution, in terms of number of decimal places to which seconds are rounded. Excessive accuracy makes for difficult, large indices in the database and may fall foul of system / database clock accuracy mismatches.
2
- DATE_MAXIMUM =
In order for indices to work properly on
effective_end
dates,NULL
values cannot be permitted as SQLNULL
is magic and means “has no value”, so such a value in a column prohibits indexing.We might have used a
NULL
value in the ‘end’ date to mean “this is the contemporary/current record”, but since we can’t do that, we need the rather nasty alternative of an agreed constant that defines a “large date” which represents “maximum possible end-of-time”.SQL does not define a maximum date, but most implementations do. PostgreSQL has a very high maximum year, while SQLite, MS SQL Server and MySQL (following a cursory Google search for documentation) say that the end of year 9999 is as high as it goes.
To use this
DATE_MAXIMUM
constant in raw SQL, be sure to format the Time instance through your ActiveRecord database adapter thus:ActiveRecord::Base.connection.quoted_date( Hoodoo::ActiveRecord::ManuallyDated::DATE_MAXIMUM ) # => returns "9999-12-31 23:59:59.000000" for PostgreSQL 9.4.
Time.parse( '9999-12-31T23:59:59.0Z' ).round( SECONDS_DECIMAL_PLACES )
Class Method Summary collapse
-
.included(model) ⇒ Object
Instantiates this module when it is included.
-
.instantiate(model) ⇒ Object
When instantiated in an ActiveRecord::Base subclass, all of the Hoodoo::ActiveRecord::ManullyDated::ClassMethods methods are defined as class methods on the including class.
Class Method Details
.included(model) ⇒ Object
Instantiates this module when it is included.
Example:
class SomeModel < ActiveRecord::Base
include Hoodoo::ActiveRecord::ManuallyDated
# ...
end
Depends upon and auto-includes Hoodoo::ActiveRecord::UUID and Hoodoo::ActiveRecord::Finder.
model
-
The ActiveRecord::Base descendant that is including this module.
386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 |
# File 'lib/hoodoo/active/active_record/manually_dated.rb', line 386 def self.included( model ) model.class_attribute( :nz_co_loyalty_hoodoo_manually_dated, :instance_predicate => false, :instance_accessor => false ) unless model == Hoodoo::ActiveRecord::Base model.send( :include, Hoodoo::ActiveRecord::UUID ) model.send( :include, Hoodoo::ActiveRecord::Finder ) instantiate( model ) end super( model ) end |
.instantiate(model) ⇒ Object
When instantiated in an ActiveRecord::Base subclass, all of the Hoodoo::ActiveRecord::ManullyDated::ClassMethods methods are defined as class methods on the including class.
model
-
The ActiveRecord::Base descendant that is including this module.
410 411 412 |
# File 'lib/hoodoo/active/active_record/manually_dated.rb', line 410 def self.instantiate( model ) model.extend( ClassMethods ) end |