Class: Gitlab::Redis::MultiStore

Inherits:
Object
  • Object
show all
Includes:
Utils::StrongMemoize
Defined in:
lib/gitlab/redis/multi_store.rb

Defined Under Namespace

Classes: MethodMissingError, NestedReadonlyPipelineError, PipelinedDiffError

Constant Summary collapse

FAILED_TO_READ_ERROR_MESSAGE = 
'Failed to read from the redis default_store.'
FAILED_TO_WRITE_ERROR_MESSAGE = 
'Failed to write to the redis non_default_store.'
FAILED_TO_RUN_PIPELINE = 
'Failed to execute pipeline on the redis non_default_store.'
SKIP_LOG_METHOD_MISSING_FOR_COMMANDS = 
%i[info].freeze
REDIS_CLIENT_COMMANDS =

_client and without_reconnect are Redis::Client methods which may be called through multistore

%i[
  _client
  without_reconnect
].freeze
PUBSUB_SUBSCRIBE_COMMANDS = 
%i[
  subscribe
  unsubscribe
].freeze
READ_COMMANDS = 
%i[
  exists
  exists?
  get
  hexists
  hget
  hgetall
  hlen
  hmget
  hscan_each
  mapped_hmget
  mget
  scan
  scan_each
  scard
  sismember
  smembers
  sscan
  sscan_each
  ttl
  zscan_each
].freeze
WRITE_COMMANDS = 
%i[
  del
  eval
  expire
  flushdb
  hdel
  hset
  incr
  incrby
  mapped_hmset
  publish
  rpush
  sadd
  sadd?
  set
  setex
  setnx
  srem
  unlink

  memory
].freeze
PIPELINED_COMMANDS = 
%i[
  pipelined
  multi
].freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(primary_store, secondary_store, instance_name) ⇒ MultiStore

To transition between two Redis store, ‘primary_store` should be the target store, and `secondary_store` should be the current store. Transition is controlled with feature flags:

  • At the default state, all read and write operations are executed in the secondary instance.

  • Turning use_primary_and_secondary_stores_for_<instance_name> on: The store writes to both instances. The read commands are executed in primary, but fallback to secondary. Other commands are executed in the the default instance (Secondary).

  • Turning use_primary_store_as_default_for_<instance_name> on: The behavior is the same as above, but other commands are executed in the primary now.

  • Turning use_primary_and_secondary_stores_for_<instance_name> off: commands are executed in the primary store.



117
118
119
120
121
122
123
 
# File 'lib/gitlab/redis/multi_store.rb', line 117

def initialize(primary_store, secondary_store, instance_name)
  @primary_store = primary_store
  @secondary_store = secondary_store
  @instance_name = instance_name

  validate_stores!
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing ⇒ Object 



175
176
177
178
179
180
181
 
# File 'lib/gitlab/redis/multi_store.rb', line 175

def method_missing(...)
  return @instance.send(...) if @instance

  log_method_missing(...)

  default_store.send(...)
end

Instance Attribute Details

#instance_name ⇒ Object (readonly)

Returns the value of attribute instance_name.



37
38
39
 
# File 'lib/gitlab/redis/multi_store.rb', line 37

def instance_name
  @instance_name
end

#primary_store ⇒ Object (readonly)

Returns the value of attribute primary_store.



37
38
39
 
# File 'lib/gitlab/redis/multi_store.rb', line 37

def primary_store
  @primary_store
end

#secondary_store ⇒ Object (readonly)

Returns the value of attribute secondary_store.



37
38
39
 
# File 'lib/gitlab/redis/multi_store.rb', line 37

def secondary_store
  @secondary_store
end

Instance Method Details

#default_store ⇒ Object 



232
233
234
 
# File 'lib/gitlab/redis/multi_store.rb', line 232

def default_store
  use_primary_store_as_default? ? primary_store : secondary_store
end

#increment_method_missing_count(command_name) ⇒ Object 



220
221
222
223
224
 
# File 'lib/gitlab/redis/multi_store.rb', line 220

def increment_method_missing_count(command_name)
  @method_missing_counter ||= Gitlab::Metrics.counter(:gitlab_redis_multi_store_method_missing_total,
                                                      'Client side Redis MultiStore method missing')
  @method_missing_counter.increment(command: command_name, instance_name: instance_name)
end

#increment_pipelined_command_error_count(command_name) ⇒ Object 



214
215
216
217
218
 
# File 'lib/gitlab/redis/multi_store.rb', line 214

def increment_pipelined_command_error_count(command_name)
  @pipelined_command_error ||= Gitlab::Metrics.counter(:gitlab_redis_multi_store_pipelined_diff_error_total,
                                                       'Redis MultiStore pipelined command diff between stores')
  @pipelined_command_error.increment(command: command_name, instance_name: instance_name)
end

#is_a?(klass) ⇒ Boolean Also known as: kind_of?

This is needed because of Redis::Rack::Connection is requiring Redis::Store github.com/redis-store/redis-rack/blob/a833086ba494083b6a384a1a4e58b36573a9165d/lib/redis/rack/connection.rb#L15 Done similarly in github.com/lsegal/yard/blob/main/lib/yard/templates/template.rb#L122

Returns:



191
192
193
194
195
 
# File 'lib/gitlab/redis/multi_store.rb', line 191

def is_a?(klass)
  return true if klass == default_store.class

  super(klass)
end

#log_error(exception, command_name, extra = {}) ⇒ Object 



226
227
228
229
230
 
# File 'lib/gitlab/redis/multi_store.rb', line 226

def log_error(exception, command_name, extra = {})
  Gitlab::ErrorTracking.log_exception(
    exception,
    extra.merge(command_name: command_name, instance_name: instance_name))
end

#non_default_store ⇒ Object 



236
237
238
 
# File 'lib/gitlab/redis/multi_store.rb', line 236

def non_default_store
  use_primary_store_as_default? ? secondary_store : primary_store
end

#ping(message = nil) ⇒ Object 



240
241
242
243
244
245
246
247
248
249
 
# File 'lib/gitlab/redis/multi_store.rb', line 240

def ping(message = nil)
  if use_primary_and_secondary_stores?
    # Both stores have to response success for the ping to be considered success.
    # We assume both stores cannot return different responses (only both "PONG" or both echo the message).
    # If either store is not reachable, an Error will be raised anyway thus taking any response works.
    [primary_store, secondary_store].map { |store| store.ping(message) }.first
  else
    default_store.ping(message)
  end
end

#readonly_pipeline? ⇒ Boolean

Returns:



140
141
142
 
# File 'lib/gitlab/redis/multi_store.rb', line 140

def readonly_pipeline?
  Thread.current[:readonly_pipeline].present?
end

#respond_to_missing?(command_name, include_private = false) ⇒ Boolean

rubocop:enable GitlabSecurity/PublicSend

Returns:



184
185
186
 
# File 'lib/gitlab/redis/multi_store.rb', line 184

def respond_to_missing?(command_name, include_private = false)
  true
end

#to_s ⇒ Object 



198
199
200
 
# File 'lib/gitlab/redis/multi_store.rb', line 198

def to_s
  use_primary_and_secondary_stores? ? primary_store.to_s : default_store.to_s
end

#use_primary_and_secondary_stores? ⇒ Boolean

Returns:



202
203
204
205
206
 
# File 'lib/gitlab/redis/multi_store.rb', line 202

def use_primary_and_secondary_stores?
  feature_table_exists? &&
    Feature.enabled?("use_primary_and_secondary_stores_for_#{instance_name.underscore}") && # rubocop:disable Cop/FeatureFlagUsage
    !same_redis_store?
end

#use_primary_store_as_default? ⇒ Boolean

Returns:



208
209
210
211
212
 
# File 'lib/gitlab/redis/multi_store.rb', line 208

def use_primary_store_as_default?
  feature_table_exists? &&
    Feature.enabled?("use_primary_store_as_default_for_#{instance_name.underscore}") && # rubocop:disable Cop/FeatureFlagUsage
    !same_redis_store?
end

#with_readonly_pipeline ⇒ Object

Pipelines are sent to both instances by default since they could execute both read and write commands.

But for pipelines that only consists of read commands, this method can be used to scope the pipeline and send it only to the default store.



130
131
132
133
134
135
136
137
138
 
# File 'lib/gitlab/redis/multi_store.rb', line 130

def with_readonly_pipeline
  raise NestedReadonlyPipelineError if readonly_pipeline?

  Thread.current[:readonly_pipeline] = true

  yield
ensure
  Thread.current[:readonly_pipeline] = false
end