Module: CounterAttribute

Extended by:
ActiveSupport::Concern, AfterCommitQueue
Includes:
Gitlab::ExclusiveLeaseHelpers, Gitlab::Utils::StrongMemoize
Included in:
ProjectStatistics, Projects::DataTransfer
Defined in:
app/models/concerns/counter_attribute.rb

Overview

Add capabilities to increment a numeric model attribute efficiently by using Redis and flushing the increments asynchronously to the database after a period of time (10 minutes). When an attribute is incremented by a value, the increment is added to a Redis key. Then, FlushCounterIncrementsWorker will execute ‘commit_increment!` which removes increments from Redis for a given model attribute and updates the values in the database.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :commit_count
  counter_attribute :storage_size
end

It’s possible to define a conditional counter attribute. You need to pass a proc that must accept a single argument, the object instance on which this concern is included.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :conditional_one, if: -> { |object| object.use_counter_attribute? }
end

The ‘counter_attribute` by default will return last persisted value. It’s possible to always return accurate (real) value instead by using ‘returns_current: true`. While doing this the `counter_attribute` will overwrite attribute accessor to fetch the buffered information added to the last persisted value. This will incur cost a Redis call per attribute fetched.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :commit_count, returns_current: true
end

in that case

model.commit_count => persisted value + buffered amount to be added

To increment the counter we can use the method:

increment_amount(:commit_count, 3)

This method would determine whether it would increment the counter using Redis, or fallback to legacy increment on ActiveRecord counters.

It is possible to register callbacks to be executed after increments have been flushed to the database. Callbacks are not executed if there are no increments to flush.

counter_attribute_after_commit do |statistic|
  Namespaces::ScheduleAggregationWorker.perform_async(statistic.namespace_id)
end

Constant Summary

Constants included from Gitlab::ExclusiveLeaseHelpers

Gitlab::ExclusiveLeaseHelpers::FailedToObtainLockError

Instance Method Summary collapse

Methods included from AfterCommitQueue

run_after_commit, run_after_commit_or_now

Methods included from Gitlab::ExclusiveLeaseHelpers

#in_lock

Instance Method Details

#bulk_increment_counter(attribute, increments) ⇒ Object



136
137
138
139
140
141
142
# File 'app/models/concerns/counter_attribute.rb', line 136

def bulk_increment_counter(attribute, increments)
  run_after_commit_or_now do
    new_value = counter(attribute).bulk_increment(increments)

    log_bulk_increment_counter(attribute, increments, new_value)
  end
end

#counter(attribute) ⇒ Object



109
110
111
112
113
114
115
# File 'app/models/concerns/counter_attribute.rb', line 109

def counter(attribute)
  strong_memoize_with(:counter, attribute) do
    # This needs #to_sym because attribute could come from a Sidekiq param,
    # which would be a string.
    build_counter_for(attribute.to_sym)
  end
end

#counter_attribute_enabled?(attribute) ⇒ Boolean

Returns:

  • (Boolean)


101
102
103
104
105
106
107
# File 'app/models/concerns/counter_attribute.rb', line 101

def counter_attribute_enabled?(attribute)
  counter_attribute = self.class.counter_attributes.find { |registered| registered[:attribute] == attribute }
  return false unless counter_attribute
  return true unless counter_attribute[:if_proc]

  counter_attribute[:if_proc].call(self)
end

#current_counter(attribute) ⇒ Object



122
123
124
# File 'app/models/concerns/counter_attribute.rb', line 122

def current_counter(attribute)
  read_attribute(attribute) + counter(attribute).get
end

#execute_after_commit_callbacksObject



166
167
168
169
170
# File 'app/models/concerns/counter_attribute.rb', line 166

def execute_after_commit_callbacks
  self.class.after_commit_callbacks.each do |callback|
    callback.call(self.reset)
  end
end

#finalize_refresh(attribute) ⇒ Object

Raises:

  • (ArgumentError)


160
161
162
163
164
# File 'app/models/concerns/counter_attribute.rb', line 160

def finalize_refresh(attribute)
  raise ArgumentError, %(attribute "#{attribute}" cannot be refreshed) unless counter_attribute_enabled?(attribute)

  counter(attribute).finalize_refresh
end

#increment_amount(attribute, amount) ⇒ Object



117
118
119
120
# File 'app/models/concerns/counter_attribute.rb', line 117

def increment_amount(attribute, amount)
  counter = Gitlab::Counters::Increment.new(amount: amount)
  increment_counter(attribute, counter)
end

#increment_counter(attribute, increment) ⇒ Object



126
127
128
129
130
131
132
133
134
# File 'app/models/concerns/counter_attribute.rb', line 126

def increment_counter(attribute, increment)
  return if increment.amount == 0

  run_after_commit_or_now do
    new_value = counter(attribute).increment(increment)

    log_increment_counter(attribute, increment, new_value)
  end
end

#initiate_refresh!(attribute) ⇒ Object

Raises:

  • (ArgumentError)


150
151
152
153
154
155
156
157
158
# File 'app/models/concerns/counter_attribute.rb', line 150

def initiate_refresh!(attribute)
  raise ArgumentError, %(attribute "#{attribute}" cannot be refreshed) unless counter_attribute_enabled?(attribute)

  detect_race_on_record(log_fields: { caller: __method__, attributes: attribute }) do
    counter(attribute).initiate_refresh!
  end

  log_clear_counter(attribute)
end

#update_counters_with_lease(increments) ⇒ Object



144
145
146
147
148
# File 'app/models/concerns/counter_attribute.rb', line 144

def update_counters_with_lease(increments)
  detect_race_on_record(log_fields: { caller: __method__, attributes: increments.keys }) do
    self.class.update_counters(id, increments)
  end
end