Class: BackgroundMigrationWorker

Inherits:
Object
  • Object
show all
Includes:
ApplicationWorker
Defined in:
app/workers/background_migration_worker.rb

Overview

rubocop:disable Scalability/IdempotentWorker

Constant Summary

Constants included from ApplicationWorker

ApplicationWorker::LOGGING_EXTRA_KEY

Constants included from WorkerAttributes

WorkerAttributes::NAMESPACE_WEIGHTS, WorkerAttributes::VALID_RESOURCE_BOUNDARIES, WorkerAttributes::VALID_URGENCIES

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Gitlab::SidekiqVersioning::Worker

#job_version

Methods included from WorkerContext

#with_context

Class Method Details

.minimum_intervalObject

The minimum amount of time between processing two jobs of the same migration class.

This interval is set to 2 or 5 minutes so autovacuuming and other maintenance related tasks have plenty of time to clean up after a migration has been performed.


16
17
18
# File 'app/workers/background_migration_worker.rb', line 16

def self.minimum_interval
  2.minutes.to_i
end

Instance Method Details

#always_perform?Boolean

Returns:

  • (Boolean)

75
76
77
# File 'app/workers/background_migration_worker.rb', line 75

def always_perform?
  Rails.env.test?
end

#database_unhealthy_counterObject


88
89
90
91
92
93
# File 'app/workers/background_migration_worker.rb', line 88

def database_unhealthy_counter
  Gitlab::Metrics.counter(
    :background_migration_database_health_reschedules,
    'The number of times a background migration is rescheduled because the database is unhealthy.'
  )
end

#healthy_database?Boolean

Returns true if the database is healthy enough to allow the migration to be performed.

class_name - The name of the background migration that we might want to

run.

Returns:

  • (Boolean)

84
85
86
# File 'app/workers/background_migration_worker.rb', line 84

def healthy_database?
  !Postgresql::ReplicationSlot.lag_too_great?
end

#lease_for(class_name) ⇒ Object


66
67
68
69
# File 'app/workers/background_migration_worker.rb', line 66

def lease_for(class_name)
  Gitlab::ExclusiveLease
    .new(lease_key_for(class_name), timeout: self.class.minimum_interval)
end

#lease_key_for(class_name) ⇒ Object


71
72
73
# File 'app/workers/background_migration_worker.rb', line 71

def lease_key_for(class_name)
  "#{self.class.name}:#{class_name}"
end

#perform(class_name, arguments = []) ⇒ Object

Performs the background migration.

See Gitlab::BackgroundMigration.perform for more information.

class_name - The class name of the background migration to run. arguments - The arguments to pass to the migration class.


26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# File 'app/workers/background_migration_worker.rb', line 26

def perform(class_name, arguments = [])
  with_context(caller_id: class_name.to_s) do
    should_perform, ttl = perform_and_ttl(class_name)

    if should_perform
      Gitlab::BackgroundMigration.perform(class_name, arguments)
    else
      # If the lease could not be obtained this means either another process is
      # running a migration of this class or we ran one recently. In this case
      # we'll reschedule the job in such a way that it is picked up again around
      # the time the lease expires.
      self.class
        .perform_in(ttl || self.class.minimum_interval, class_name, arguments)
    end
  end
end

#perform_and_ttl(class_name) ⇒ Object


43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# File 'app/workers/background_migration_worker.rb', line 43

def perform_and_ttl(class_name)
  if always_perform?
    # In test environments `perform_in` will run right away. This can then
    # lead to stack level errors in the above `#perform`. To work around this
    # we'll just perform the migration right away in the test environment.
    [true, nil]
  else
    lease = lease_for(class_name)
    perform = !!lease.try_obtain

    # If we managed to acquire the lease but the DB is not healthy, then we
    # want to simply reschedule our job and try again _after_ the lease
    # expires.
    if perform && !healthy_database?
      database_unhealthy_counter.increment

      perform = false
    end

    [perform, lease.ttl]
  end
end