Module: CounterAttribute

Extended by:
ActiveSupport::Concern
Includes:
AfterCommitQueue, Gitlab::ExclusiveLeaseHelpers, Gitlab::Utils::StrongMemoize
Included in:
ProjectDailyStatistic, ProjectStatistics, Projects::DataTransfer
Defined in:
app/models/concerns/counter_attribute.rb

Overview

Add capabilities to increment a numeric model attribute efficiently by using Redis and flushing the increments asynchronously to the database after a period of time (10 minutes). When an attribute is incremented by a value, the increment is added to a Redis key. Then, FlushCounterIncrementsWorker will execute ‘commit_increment!` which removes increments from Redis for a given model attribute and updates the values in the database.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :commit_count
  counter_attribute :storage_size
end

It’s possible to define a conditional counter attribute. You need to pass a proc that must accept a single argument, the object instance on which this concern is included.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :conditional_one, if: -> { |object| object.use_counter_attribute? }
end

The ‘counter_attribute` by default will return last persisted value. It’s possible to always return accurate (real) value instead by using ‘returns_current: true`. While doing this the `counter_attribute` will overwrite attribute accessor to fetch the buffered information added to the last persisted value. This will incur cost a Redis call per attribute fetched.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :commit_count, returns_current: true
end

in that case

model.commit_count => persisted value + buffered amount to be added

To increment the counter we can use the method:

increment_amount(:commit_count, 3)

This method would determine whether it would increment the counter using Redis, or fallback to legacy increment on ActiveRecord counters.

It is possible to register callbacks to be executed after increments have been flushed to the database. Callbacks are not executed if there are no increments to flush.

counter_attribute_after_commit do |statistic|
  Namespaces::ScheduleAggregationWorker.perform_async(statistic.namespace_id)
end

Constant Summary

Constants included from Gitlab::ExclusiveLeaseHelpers

Gitlab::ExclusiveLeaseHelpers::FailedToObtainLockError

Instance Method Summary collapse

Methods included from Gitlab::ExclusiveLeaseHelpers

#in_lock

Methods included from AfterCommitQueue

#run_after_commit, #run_after_commit_or_now

Instance Method Details

#bulk_increment_counter(attribute, increments) ⇒ Object 



136
137
138
139
140
141
142
 
# File 'app/models/concerns/counter_attribute.rb', line 136

def bulk_increment_counter(attribute, increments)
  run_after_commit_or_now do
    new_value = counter(attribute).bulk_increment(increments)

    log_bulk_increment_counter(attribute, increments, new_value)
  end
end

#counter(attribute) ⇒ Object 



109
110
111
112
113
114
115
 
# File 'app/models/concerns/counter_attribute.rb', line 109

def counter(attribute)
  strong_memoize_with(:counter, attribute) do
    # This needs #to_sym because attribute could come from a Sidekiq param,
    # which would be a string.
    build_counter_for(attribute.to_sym)
  end
end

#counter_attribute_enabled?(attribute) ⇒ Boolean

Returns:

  • (Boolean) 


101
102
103
104
105
106
107
 
# File 'app/models/concerns/counter_attribute.rb', line 101

def counter_attribute_enabled?(attribute)
  counter_attribute = self.class.counter_attributes.find { |registered| registered[:attribute] == attribute }
  return false unless counter_attribute
  return true unless counter_attribute[:if_proc]

  counter_attribute[:if_proc].call(self)
end

#current_counter(attribute) ⇒ Object 



122
123
124
 
# File 'app/models/concerns/counter_attribute.rb', line 122

def current_counter(attribute)
  read_attribute(attribute) + counter(attribute).get
end

#execute_after_commit_callbacks ⇒ Object 



167
168
169
170
171
 
# File 'app/models/concerns/counter_attribute.rb', line 167

def execute_after_commit_callbacks
  self.class.after_commit_callbacks.each do |callback|
    callback.call(self.reset)
  end
end

#finalize_refresh(attribute) ⇒ Object

Raises:

  • (ArgumentError) 


161
162
163
164
165
 
# File 'app/models/concerns/counter_attribute.rb', line 161

def finalize_refresh(attribute)
  raise ArgumentError, %(attribute "#{attribute}" cannot be refreshed) unless counter_attribute_enabled?(attribute)

  counter(attribute).finalize_refresh
end

#increment_amount(attribute, amount) ⇒ Object 



117
118
119
120
 
# File 'app/models/concerns/counter_attribute.rb', line 117

def increment_amount(attribute, amount)
  counter = Gitlab::Counters::Increment.new(amount: amount)
  increment_counter(attribute, counter)
end

#increment_counter(attribute, increment) ⇒ Object 



126
127
128
129
130
131
132
133
134
 
# File 'app/models/concerns/counter_attribute.rb', line 126

def increment_counter(attribute, increment)
  return if increment.amount == 0

  run_after_commit_or_now do
    new_value = counter(attribute).increment(increment)

    log_increment_counter(attribute, increment, new_value)
  end
end

#initiate_refresh!(attribute) ⇒ Object

Raises:

  • (ArgumentError) 


154
155
156
157
158
159
 
# File 'app/models/concerns/counter_attribute.rb', line 154

def initiate_refresh!(attribute)
  raise ArgumentError, %(attribute "#{attribute}" cannot be refreshed) unless counter_attribute_enabled?(attribute)

  counter(attribute).initiate_refresh!
  log_clear_counter(attribute)
end

#update_counters(increments) ⇒ Object 



144
145
146
 
# File 'app/models/concerns/counter_attribute.rb', line 144

def update_counters(increments)
  self.class.update_counters(id, increments)
end

#update_counters_with_lease(increments) ⇒ Object 



148
149
150
151
152
 
# File 'app/models/concerns/counter_attribute.rb', line 148

def update_counters_with_lease(increments)
  detect_race_on_record(log_fields: { caller: __method__, attributes: increments.keys }) do
    update_counters(increments)
  end
end