Class: Partitioned::PartitionedBase

Inherits:
ActiveRecord::Base
  • Object
show all
Extended by:
BulkMethodsMixin
Includes:
ActiveRecordOverrides
Defined in:
lib/partitioned/partitioned_base.rb,
lib/partitioned/partitioned_base/sql_adapter.rb,
lib/partitioned/partitioned_base/configurator.rb,
lib/partitioned/partitioned_base/configurator/dsl.rb,
lib/partitioned/partitioned_base/configurator/data.rb,
lib/partitioned/partitioned_base/partition_manager.rb,
lib/partitioned/partitioned_base/configurator/reader.rb,
lib/partitioned/partitioned_base/redshift_sql_adapter.rb

Overview

PartitionedBase an ActiveRecord::Base class that can be partitioned.

Uses a domain specific language to configure, see Partitioned::PartitionedBase::Configurator for more information.

Extends BulkMethodsMixin to provide create_many and update_many.

Uses PartitionManager to manage creation of child tables.

Monkey patches some ActiveRecord routines to call back to this class when INSERT and UPDATE statements are built (to determine the table_name with respect to values being inserted or updated)

Direct Known Subclasses

ByIntegerField, ByTimeField, MultiLevel

Defined Under Namespace

Modules: Configurator Classes: PartitionManager, RedshiftSqlAdapter, SqlAdapter

Class Method Summary collapse

Instance Method Summary collapse

Methods included from ActiveRecordOverrides

#arel_attributes_values, #delete

Class Method Details

.add_parent_table_rules(*partition_key_values) ⇒ Object

:singleton-method: add_parent_table_rules delegated to Partitioned::PartitionedBase::PartitionManager#add_parent_table_rules



369
370
371
 
# File 'lib/partitioned/partitioned_base.rb', line 369

def self.add_parent_table_rules(*partition_key_values)
  partition_manager.add_parent_table_rules(*partition_key_values)
end

.add_partition_table_index(*partition_key_values) ⇒ Object

:singleton-method: add_partition_table_index delegated to Partitioned::PartitionedBase::PartitionManager#add_partition_table_index



348
349
350
 
# File 'lib/partitioned/partitioned_base.rb', line 348

def self.add_partition_table_index(*partition_key_values)
  partition_manager.add_partition_table_index(*partition_key_values)
end

.add_references_to_partition_table(*partition_key_values) ⇒ Object

:singleton-method: add_references_to_partition_table delegated to Partitioned::PartitionedBase::PartitionManager#add_references_to_partition_table



355
356
357
 
# File 'lib/partitioned/partitioned_base.rb', line 355

def self.add_references_to_partition_table(*partition_key_values)
  partition_manager.add_references_to_partition_table(*partition_key_values)
end

.archive_old_partition(*partition_key_values) ⇒ Object

:method: archive_old_partition delegated to Partitioned::PartitionedBase::PartitionManager#archive_old_partition



397
398
399
 
# File 'lib/partitioned/partitioned_base.rb', line 397

def self.archive_old_partition(*partition_key_values)
  partition_manager.archive_old_partition(*partition_key_values)
end

.archive_old_partitions ⇒ Object

:method: archive_old_partitions delegated to Partitioned::PartitionedBase::PartitionManager#archive_old_partitions



376
377
378
 
# File 'lib/partitioned/partitioned_base.rb', line 376

def self.archive_old_partitions
  partition_manager.archive_old_partitions
end

.arel_table_from_key_values(partition_key_values, as = nil) ⇒ Object 



110
111
112
113
114
115
116
117
118
119
120
121
 
# File 'lib/partitioned/partitioned_base.rb', line 110

def self.arel_table_from_key_values(partition_key_values, as = nil)
  @arel_tables ||= {}
  new_arel_table = @arel_tables[[partition_key_values, as]]
  
  unless new_arel_table
    arel_engine_hash = {:engine => self.arel_engine, :as => as}
    new_arel_table = Arel::Table.new(self.partition_table_name(*partition_key_values), arel_engine_hash)
    @arel_tables[[partition_key_values, as]] = new_arel_table
  end

  return new_arel_table
end

.configurator ⇒ {Configurator::Reader}

Return a object used to read configurator information.

Returns:



194
195
196
197
198
199
 
# File 'lib/partitioned/partitioned_base.rb', line 194

def self.configurator
  unless @configurator
    @configurator = self::Configurator::Reader.new(self)
  end
  return @configurator
end

.configurator_dsl ⇒ {Configurator::Dsl}

Returns the configurator DSL object.

Returns:



222
223
224
 
# File 'lib/partitioned/partitioned_base.rb', line 222

def self.configurator_dsl
  return @configurator_dsl
end

.create_infrastructure ⇒ Object

:method: create_infrastructure delegated to Partitioned::PartitionedBase::PartitionManager#create_infrastructure



425
426
427
 
# File 'lib/partitioned/partitioned_base.rb', line 425

def self.create_infrastructure
  partition_manager.create_infrastructure
end

.create_new_partition(*partition_key_values) ⇒ Object

:method: create_new_partition delegated to Partitioned::PartitionedBase::PartitionManager#create_new_partition



411
412
413
 
# File 'lib/partitioned/partitioned_base.rb', line 411

def self.create_new_partition(*partition_key_values)
  partition_manager.create_new_partition(*partition_key_values)
end

.create_new_partition_tables(enumerable) ⇒ Object

:method: create_new_partition_tables delegated to Partitioned::PartitionedBase::PartitionManager#create_new_partition_tables



418
419
420
 
# File 'lib/partitioned/partitioned_base.rb', line 418

def self.create_new_partition_tables(enumerable)
  partition_manager.create_new_partition_tables(enumerable)
end

.create_new_partitions ⇒ Object

:method: create_new_partitions delegated to Partitioned::PartitionedBase::PartitionManager#create_new_partitions



390
391
392
 
# File 'lib/partitioned/partitioned_base.rb', line 390

def self.create_new_partitions
  partition_manager.create_new_partitions
end

.create_partition_schema(*partition_key_values) ⇒ Object

:method: create_partition_schema delegated to Partitioned::PartitionedBase::PartitionManager#create_partition_schema



362
363
364
 
# File 'lib/partitioned/partitioned_base.rb', line 362

def self.create_partition_schema(*partition_key_values)
  partition_manager.create_partition_schema(*partition_key_values)
end

.create_partition_table(*partition_key_values) ⇒ Object

:singleton-method: create_partition_table delegated to Partitioned::PartitionedBase::PartitionManager#create_partition_table



341
342
343
 
# File 'lib/partitioned/partitioned_base.rb', line 341

def self.create_partition_table(*partition_key_values)
  partition_manager.create_partition_table(*partition_key_values)
end

.delete_infrastructure ⇒ Object

:method: delete_infrastructure delegated to Partitioned::PartitionedBase::PartitionManager#delete_infrastructure



432
433
434
 
# File 'lib/partitioned/partitioned_base.rb', line 432

def self.delete_infrastructure
  partition_manager.delete_infrastructure
end

.drop_old_partition(*partition_key_values) ⇒ Object

:method: drop_old_partition delegated to Partitioned::PartitionedBase::PartitionManager#drop_old_partition



404
405
406
 
# File 'lib/partitioned/partitioned_base.rb', line 404

def self.drop_old_partition(*partition_key_values)
  partition_manager.drop_old_partition(*partition_key_values)
end

.drop_old_partitions ⇒ Object

:method: drop_old_partitions delegated to Partitioned::PartitionedBase::PartitionManager#drop_old_partitions



383
384
385
 
# File 'lib/partitioned/partitioned_base.rb', line 383

def self.drop_old_partitions
  partition_manager.drop_old_partitions
end

.drop_partition_table(*partition_key_values) ⇒ Object

:singleton-method: drop_partition_table delegated to Partitioned::PartitionedBase::PartitionManager#drop_partition_table



334
335
336
 
# File 'lib/partitioned/partitioned_base.rb', line 334

def self.drop_partition_table(*partition_key_values)
  partition_manager.drop_partition_table(*partition_key_values)
end

.dynamic_arel_table(values, as = nil) ⇒ Arel::Table

In activerecord 3.0 we need to supply an Arel::Table for the key value(s) used to determine the specific child table to access.

Parameters:

  • values (Hash) —

    key/value pairs for all attributes

  • as (String) (defaults to: nil) —

    (nil) the name of the table associated with this Arel::Table

Returns:

  • (Arel::Table) —

    the generated Arel::Table



130
131
132
133
 
# File 'lib/partitioned/partitioned_base.rb', line 130

def self.dynamic_arel_table(values, as = nil)
  key_values = self.partition_key_values(values)
  return arel_table_from_key_values(key_values, as)
end

.from_partition(*partition_key_values) ⇒ Hash

This scoping is used to target the active record find() to a specific child table and alias it to the name of the parent table (so activerecord can generally work with it)

Use as:

Foo.from_partition(KEY).find(:first)

where KEY is the key value(s) used as the check constraint on Foo’s table.

Parameters:

  • partition_field (*Array<Object>) —

    the field values to partition on

Returns:

  • (Hash) —

    the scoping



159
160
161
162
 
# File 'lib/partitioned/partitioned_base.rb', line 159

def self.from_partition(*partition_key_values)
  table_alias_name = partition_table_alias_name(*partition_key_values)
  return ActiveRecord::Relation.new(self, self.arel_table_from_key_values(partition_key_values, table_alias_name))
end

.from_partition_without_alias(*partition_key_values) ⇒ Hash

This scope is used to target the active record find() to a specific child table. Is probably best used in advanced activerecord queries when a number of tables are involved in the query.

Use as:

Foo.from_partition_without_alias(KEY).find(:all, :select => "*")

where KEY is the key value(s) used as the check constraint on Foo’s table.

it’s not obvious why :select => “*” is supplied. note activerecord wants to use the name of parent table for access to any attributes, so without the :select argument the sql result would be something like:

SELECT foos.* FROM foos_partitions.pXXX

which fails because table foos is not referenced. using the form #from_partition is almost always the correct thing when using activerecord.

Parameters:

  • partition_field (*Array<Object>) —

    the field values to partition on

Returns:

  • (Hash) —

    the scoping



186
187
188
 
# File 'lib/partitioned/partitioned_base.rb', line 186

def self.from_partition_without_alias(*partition_key_values)
  return ActiveRecord::Relation.new(self, self.arel_table_from_key_values(partition_key_values, nil))
end

.partition_generate_range(start_value, end_value, step = 1) ⇒ Enumerable

Range generation provided for methods like created_infrastructure that need a set of partition key values to operate on.

Parameters:

  • start_value (Object) —

    the first value to generate the range from

  • end_value (Object) —

    the last value to generate the range from

  • step (Object) (defaults to: 1) —

    (1) number of values to advance.

Returns:

  • (Enumerable) —

    the range generated



87
88
89
 
# File 'lib/partitioned/partitioned_base.rb', line 87

def self.partition_generate_range(start_value, end_value, step = 1)
  return Range.new(start_value, end_value).step(step)
end

.partition_key_values(values) ⇒ Object+

The specific values for a partition of this active record’s type which are defined by Partitioned::PartitionedBase#self#self.partition_keys

Parameters:

  • values (Hash) —

    key/value pairs to extract values from

Returns:

  • (Object) —

    value of partition key

  • (Array<Object>) —

    values of partition keys



53
54
55
56
 
# File 'lib/partitioned/partitioned_base.rb', line 53

def self.partition_key_values(values)
  symbolized_values = values.symbolize_keys
  return self.partition_keys.map{|key| symbolized_values[key.to_sym]}
end

.partition_keys ⇒ String+

Returns an array of attribute names (strings) used to fetch the key value(s) the determine this specific partition table.

Returns:

  • (String) —

    the column name used to partition this table

  • (Array<String>) —

    the column names used to partition this table



42
43
44
 
# File 'lib/partitioned/partitioned_base.rb', line 42

def self.partition_keys
  return configurator.on_fields
end

.partition_manager ⇒ {PartitionManager}

Return an instance of this partition table’s table manager.

Returns:



95
96
97
98
 
# File 'lib/partitioned/partitioned_base.rb', line 95

def self.partition_manager
  @partition_manager = self::PartitionManager.new(self) unless @partition_manager.present?
  return @partition_manager
end

.partition_name(*partition_key_values) ⇒ Object

:method: partition_name delegated to Partitioned::PartitionedBase::PartitionManager#partition_table_name



446
447
448
 
# File 'lib/partitioned/partitioned_base.rb', line 446

def self.partition_name(*partition_key_values)
  return partition_manager.partition_table_name(*partition_key_values)
end

.partition_normalize_key_value(value) ⇒ Object

Normalize the value to be used for partitioning. This allows, for instance, a class that partitions on a time field to group the times by month. An integer field might be grouped by every 10mil values, A string field might be grouped by its first character.

Parameters:

  • value (Object) —

    the partition key value

Returns:

  • (Object) —

    the normalized value for the key value passed in



75
76
77
 
# File 'lib/partitioned/partitioned_base.rb', line 75

def self.partition_normalize_key_value(value)
  return value
end

.partition_table_alias_name(*partition_key_values) ⇒ Object

:method: partition_table_alias_name delegated to Partitioned::PartitionedBase::PartitionManager#partition_table_alias_name



453
454
455
 
# File 'lib/partitioned/partitioned_base.rb', line 453

def self.partition_table_alias_name(*partition_key_values)
  return partition_manager.partition_table_alias_name(*partition_key_values).gsub(/[^a-zA-Z0-9]/, '_')
end

.partition_table_name(*partition_key_values) ⇒ Object

:method: partition_table_name delegated to Partitioned::PartitionedBase::PartitionManager#partition_table_name



439
440
441
 
# File 'lib/partitioned/partitioned_base.rb', line 439

def self.partition_table_name(*partition_key_values)
  return partition_manager.partition_table_name(*partition_key_values)
end

.partitioned {|@configurator_dsl| ... } ⇒ {Configurator::Dsl}

Yields an object used to configure the ActiveRecord class for partitioning using the Configurator Domain Specific Language.

usage:

partitioned do |partition|
  partition.on    :company_id
  partition.index :id, :unique => true
  partition.foreign_key :company_id
end

Yields:

Returns:



213
214
215
216
 
# File 'lib/partitioned/partitioned_base.rb', line 213

def self.partitioned
  @configurator_dsl ||= self::Configurator::Dsl.new(self)
  yield @configurator_dsl
end

.sql_adapter ⇒ {SqlAdapter}

Return an instance of this partition table’s sql_adapter (used by the partition manage to create SQL statements)

Returns:

  • ({SqlAdapter}) —

    the object used to create sql statements for this partitioned model



105
106
107
108
 
# File 'lib/partitioned/partitioned_base.rb', line 105

def self.sql_adapter
  @sql_adapter ||= connection.partitioned_sql_adapter(self)
  return @sql_adapter
end

Instance Method Details

#dynamic_arel_table(as = nil) ⇒ Arel::Table

Used by our active record hacks to supply an Arel::Table given this active record’s current attributes.

Parameters:

  • as (String) (defaults to: nil) —

    (nil) the name of the table associated with the Arel::Table

Returns:

  • (Arel::Table) —

    the generated Arel::Table



141
142
143
144
 
# File 'lib/partitioned/partitioned_base.rb', line 141

def dynamic_arel_table(as = nil)
  key_values = self.class.partition_key_values(attributes)
  return self.class.arel_table_from_key_values(key_values, as)
end

#partition_table_name ⇒ String

The name of the current partition table determined by this active records attributes that define the key value(s) for the constraint check.

Returns:

  • (String) —

    the fully qualified name of the database table, ie: foos_partitions.p17



63
64
65
66
 
# File 'lib/partitioned/partitioned_base.rb', line 63

def partition_table_name
  symbolized_attributes = attributes.symbolize_keys
  return self.class.partition_table_name(*self.class.partition_keys.map{|attribute_name| symbolized_attributes[attribute_name]})
end