Class: Gitlab::BackgroundMigration::MigrateMergeRequestDiffCommitUsers::MergeRequestDiffCommit

Inherits:
ActiveRecord::Base
  • Object
show all
Extended by:
SuppressCompositePrimaryKeyWarning
Includes:
FromUnion
Defined in:
lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb

Overview

rubocop: disable Style/Documentation

Class Method Summary collapse

Class Method Details

.each_row_to_migrate(start_id, stop_id, &block) ⇒ Object

Yields each row to migrate in the given range.

This method uses keyset pagination to ensure we don't retrieve potentially tens of thousands (or even hundreds of thousands) of rows in a single query. Such queries could time out, or increase the amount of memory needed to process the data.

We can't use `EachBatch` and similar approaches, as merge_request_diff_commits doesn't have a single monotonically increasing primary key.


47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
# File 'lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb', line 47

def self.each_row_to_migrate(start_id, stop_id, &block)
  order = Pagination::Keyset::Order.build(
    %w[merge_request_diff_id relative_order].map do |col|
      Pagination::Keyset::ColumnOrderDefinition.new(
        attribute_name: col,
        order_expression: self.arel_table[col.to_sym].asc,
        nullable: :not_nullable,
        distinct: false
      )
    end
  )

  scope = MergeRequestDiffCommit
    .where(merge_request_diff_id: start_id...stop_id)
    .order(order)

  Pagination::Keyset::Iterator
    .new(scope: scope, use_union_optimization: true)
    .each_batch(of: COMMIT_ROWS_PER_QUERY) { |rows| rows.each(&block) }
end