Class: Projects::BuildArtifactsSizeRefresh

Inherits:
ApplicationRecord show all
Includes:
AfterCommitQueue, BulkInsertSafe
Defined in:
app/models/projects/build_artifacts_size_refresh.rb

Constant Summary collapse

STALE_WINDOW =
2.hours
FINALIZE_DELAY =

This delay is set to 10 minutes to accommodate any ongoing deletion that might have happened. The delete on the database may have been committed before the refresh completed its batching. If the resulting decrement is pushed into Redis after the refresh has ended, it would result in net negative value. The delay is needed to ensure this negative value is ignored.

10.minutes
COUNTER_ATTRIBUTE_NAME =
:build_artifacts_size
STATES =

The refresh of the project statistics counter is performed in 4 stages:

  1. created - The refresh is on the queue to be processed by Projects::RefreshBuildArtifactsSizeStatisticsWorker

  2. running - The refresh is ongoing. The project statistics counter switches to the temporary refresh counter key. Counter increments are deduplicated.

  3. pending - The refresh is pending to be picked up by Projects::RefreshBuildArtifactsSizeStatisticsWorker again.

  4. finalizing - The refresh has finished summing existing job artifact size into the refresh counter key. The sum will need to be moved into the counter key.

{
  created: 1,
  running: 2,
  pending: 3,
  finalizing: 4
}.freeze

Constants included from BulkInsertSafe

BulkInsertSafe::ALLOWED_CALLBACKS, BulkInsertSafe::DEFAULT_BATCH_SIZE, BulkInsertSafe::MethodNotAllowedError, BulkInsertSafe::PrimaryKeySetError

Constants inherited from ApplicationRecord

ApplicationRecord::MAX_PLUCK

Constants included from ResetOnUnionError

ResetOnUnionError::MAX_RESET_PERIOD

Class Method Summary collapse

Instance Method Summary collapse

Methods included from AfterCommitQueue

#run_after_commit, #run_after_commit_or_now

Methods inherited from ApplicationRecord

cached_column_list, #create_or_load_association, declarative_enum, default_select_columns, id_in, id_not_in, iid_in, pluck_primary_key, primary_key_in, #readable_by?, safe_ensure_unique, safe_find_or_create_by, safe_find_or_create_by!, #to_ability_name, underscore, where_exists, where_not_exists, with_fast_read_statement_timeout, without_order

Methods included from SensitiveSerializableHash

#serializable_hash

Class Method Details

.enqueue_refresh(projects) ⇒ Object



84
85
86
87
88
89
90
91
92
# File 'app/models/projects/build_artifacts_size_refresh.rb', line 84

def self.enqueue_refresh(projects)
  now = Time.zone.now

  records = Array(projects).map do |project|
    new(project: project, state: STATES[:created], created_at: now, updated_at: now)
  end

  bulk_insert!(records, skip_duplicates: true)
end

.process_next_refresh!Object



94
95
96
97
98
99
100
101
102
103
104
105
106
# File 'app/models/projects/build_artifacts_size_refresh.rb', line 94

def self.process_next_refresh!
  next_refresh = nil

  transaction do
    next_refresh = processing_queue
      .lock('FOR UPDATE SKIP LOCKED')
      .take

    next_refresh&.process!
  end

  next_refresh
end

Instance Method Details

#finalize!Object



124
125
126
127
128
# File 'app/models/projects/build_artifacts_size_refresh.rb', line 124

def finalize!
  project.statistics.finalize_refresh(COUNTER_ATTRIBUTE_NAME)

  destroy!
end

#next_batch(limit:) ⇒ Object



112
113
114
115
116
117
118
# File 'app/models/projects/build_artifacts_size_refresh.rb', line 112

def next_batch(limit:)
  project.job_artifacts.select(:id, :size)
    .id_before(last_job_artifact_id_on_refresh_start)
    .id_after(last_job_artifact_id.to_i)
    .ordered_by_id
    .limit(limit)
end

#reset_project_statistics!Object



108
109
110
# File 'app/models/projects/build_artifacts_size_refresh.rb', line 108

def reset_project_statistics!
  project.statistics.initiate_refresh!(COUNTER_ATTRIBUTE_NAME)
end

#schedule_finalize_workerObject



130
131
132
133
134
# File 'app/models/projects/build_artifacts_size_refresh.rb', line 130

def schedule_finalize_worker
  run_after_commit do
    Projects::FinalizeProjectStatisticsRefreshWorker.perform_in(FINALIZE_DELAY, self.class.to_s, id)
  end
end

#started?Boolean

Returns:

  • (Boolean)


120
121
122
# File 'app/models/projects/build_artifacts_size_refresh.rb', line 120

def started?
  !created?
end