Module: Gitlab::Database::PartitioningMigrationHelpers::IndexHelpers

Includes:
MigrationHelpers, SchemaHelpers
Included in:
Gitlab::Database::PartitioningMigrationHelpers
Defined in:
lib/gitlab/database/partitioning_migration_helpers/index_helpers.rb

Constant Summary collapse

ERROR_SCOPE = 
'index'

Constants included from MigrationHelpers

MigrationHelpers::DEFAULT_TIMESTAMP_COLUMNS, MigrationHelpers::MAX_IDENTIFIER_NAME_LENGTH

Constants included from DynamicModelHelpers

DynamicModelHelpers::BATCH_SIZE

Constants included from Migrations::BatchedBackgroundMigrationHelpers

Migrations::BatchedBackgroundMigrationHelpers::BATCH_CLASS_NAME, Migrations::BatchedBackgroundMigrationHelpers::BATCH_MIN_DELAY, Migrations::BatchedBackgroundMigrationHelpers::BATCH_MIN_VALUE, Migrations::BatchedBackgroundMigrationHelpers::BATCH_SIZE, Migrations::BatchedBackgroundMigrationHelpers::SUB_BATCH_SIZE

Constants included from Migrations::BackgroundMigrationHelpers

Migrations::BackgroundMigrationHelpers::BATCH_SIZE, Migrations::BackgroundMigrationHelpers::JOB_BUFFER_SIZE

Instance Method Summary collapse

Methods included from SchemaHelpers

#assert_not_in_transaction_block, #create_comment, #create_trigger, #create_trigger_function, #drop_function, #drop_trigger, #function_exists?, #object_name, #tmp_table_name, #trigger_exists?, #with_lock_retries

Methods included from MigrationHelpers

#add_check_constraint, #add_column_with_default, #add_concurrent_foreign_key, #add_concurrent_index, #add_not_null_constraint, #add_text_limit, #add_timestamps_with_timezone, #backfill_conversion_of_integer_to_bigint, #backfill_iids, #change_column_type_concurrently, #check_constraint_exists?, #check_constraint_name, #check_not_null_constraint_exists?, #check_text_limit_exists?, #check_trigger_permissions!, #cleanup_concurrent_column_rename, #cleanup_concurrent_column_type_change, #column_for, #concurrent_foreign_key_name, #convert_to_bigint_column, #copy_check_constraints, #copy_foreign_keys, #copy_indexes, #create_extension, #create_or_update_plan_limit, #create_table_with_constraints, #define_batchable_model, #disable_statement_timeout, #drop_extension, #each_batch, #each_batch_range, #ensure_batched_background_migration_is_finished, #false_value, #foreign_key_exists?, #foreign_keys_for, #index_exists_by_name?, #index_invalid?, #indexes_for, #initialize_conversion_of_integer_to_bigint, #install_rename_triggers, #postgres_exists_by_name?, #remove_check_constraint, #remove_concurrent_index, #remove_concurrent_index_by_name, #remove_foreign_key_if_exists, #remove_foreign_key_without_error, #remove_not_null_constraint, #remove_rename_triggers, #remove_text_limit, #remove_timestamps, #rename_column_concurrently, #rename_constraint, #rename_trigger_name, #replace_sql, #restore_conversion_of_integer_to_bigint, #revert_backfill_conversion_of_integer_to_bigint, #revert_initialize_conversion_of_integer_to_bigint, #sidekiq_queue_length, #sidekiq_queue_migrate, #true_value, #undo_change_column_type_concurrently, #undo_cleanup_concurrent_column_rename, #undo_cleanup_concurrent_column_type_change, #undo_rename_column_concurrently, #update_column_in_batches, #validate_check_constraint, #validate_foreign_key, #validate_not_null_constraint, #validate_text_limit, #with_lock_retries

Methods included from AsyncIndexes::MigrationHelpers

#async_index_creation_available?, #prepare_async_index, #unprepare_async_index, #unprepare_async_index_by_name

Methods included from RenameTableHelpers

#finalize_table_rename, #rename_table_safely, #undo_finalize_table_rename, #undo_rename_table_safely

Methods included from DynamicModelHelpers

#define_batchable_model, #each_batch, #each_batch_range

Methods included from Migrations::BatchedBackgroundMigrationHelpers

#queue_batched_background_migration

Methods included from Migrations::BackgroundMigrationHelpers

#delete_job_tracking, #delete_queued_jobs, #finalize_background_migration, #migrate_in, #queue_background_migration_jobs_by_range_at_intervals, #requeue_background_migration_jobs_by_range_at_intervals

Instance Method Details

#add_concurrent_partitioned_index(table_name, column_names, options = {}) ⇒ Object

Concurrently creates a new index on a partitioned table. In concept this works similarly to `add_concurrent_index`, and won't block reads or writes on the table while the index is being built.

A special helper is required for partitioning because Postgres does not support concurrently building indexes on partitioned tables. This helper concurrently adds the same index to each partition, and creates the final index on the parent table once all of the partitions are indexed. This is the recommended safe way to add indexes to partitioned tables.

Example:

add_concurrent_partitioned_index :users, :some_column

See Rails' `add_index` for more info on the available arguments.

Raises:

  • (ArgumentError) 

25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
 
# File 'lib/gitlab/database/partitioning_migration_helpers/index_helpers.rb', line 25

def add_concurrent_partitioned_index(table_name, column_names, options = {})
  assert_not_in_transaction_block(scope: ERROR_SCOPE)

  raise ArgumentError, 'A name is required for indexes added to partitioned tables' unless options[:name]

  partitioned_table = find_partitioned_table(table_name)

  if index_name_exists?(table_name, options[:name])
    Gitlab::AppLogger.warn "Index not created because it already exists (this may be due to an aborted" \
      " migration or similar): table_name: #{table_name}, index_name: #{options[:name]}"

    return
  end

  partitioned_table.postgres_partitions.order(:name).each do |partition|
    partition_index_name = generated_index_name(partition.identifier, options[:name])
    partition_options = options.merge(name: partition_index_name)

    add_concurrent_index(partition.identifier, column_names, partition_options)
  end

  with_lock_retries do
    add_index(table_name, column_names, **options)
  end
end

#remove_concurrent_partitioned_index_by_name(table_name, index_name) ⇒ Object

Safely removes an existing index from a partitioned table. The method name is a bit inaccurate as it does not drop the index concurrently, but it's named as such to maintain consistency with other similar helpers, and indicate that this should be safe to use in a production environment.

In current versions of Postgres it's impossible to drop an index concurrently, or drop an index from an individual partition that exists across the entire partitioned table. As a result this helper drops the index from the parent table, which automatically cascades to all partitions. While this does require an exclusive lock, dropping an index is a fast operation that won't block the table for a significant period of time.

Example:

remove_concurrent_partitioned_index_by_name :users, 'index_name_goes_here'


63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/gitlab/database/partitioning_migration_helpers/index_helpers.rb', line 63

def remove_concurrent_partitioned_index_by_name(table_name, index_name)
  assert_not_in_transaction_block(scope: ERROR_SCOPE)

  find_partitioned_table(table_name)

  unless index_name_exists?(table_name, index_name)
    Gitlab::AppLogger.warn "Index not removed because it does not exist (this may be due to an aborted " \
      "migration or similar): table_name: #{table_name}, index_name: #{index_name}"

    return
  end

  with_lock_retries do
    remove_index(table_name, name: index_name)
  end
end