Module: Dalli::Protocol::Meta::RequestFormatter

Extended by:
RequestFormatter
Included in:
RequestFormatter
Defined in:
lib/dalli/protocol/request_formatter.rb

Overview

Class that encapsulates logic for formatting meta protocol requests to memcached.

Constant Summary collapse

META_NOOP =

rubocop:enable Metrics/CyclomaticComplexity rubocop:enable Metrics/ParameterLists rubocop:enable Metrics/PerceivedComplexity

"mn#{TERMINATOR}".freeze
ALLOWED_STATS_ARGS = 
[nil, '', 'items', 'slabs', 'settings', 'reset'].freeze

Instance Method Summary collapse

Instance Method Details

#encoded_key(key) ⇒ Object 



243
244
245
246
247
248
249
 
# File 'lib/dalli/protocol/request_formatter.rb', line 243

def encoded_key(key)
  if KeyRegularizer.required?(key)
    KeyRegularizer.encode(key) << ' b'
  else
    key
  end
end

#flush(delay: nil, quiet: false) ⇒ Object 



226
227
228
229
230
231
 
# File 'lib/dalli/protocol/request_formatter.rb', line 226

def flush(delay: nil, quiet: false)
  cmd = +'flush_all'
  cmd << " #{parse_to_64_bit_int(delay, 0)}" if delay
  cmd << ' noreply' if quiet
  cmd << TERMINATOR
end

#meta_arithmetic(key:, delta:, initial:, incr: true, cas: nil, ttl: nil, quiet: false, p_token: nil, l_token: nil) ⇒ Object 



170
171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/dalli/protocol/request_formatter.rb', line 170

def meta_arithmetic(key:, delta:, initial:, incr: true, cas: nil, ttl: nil, quiet: false,
                    p_token: nil, l_token: nil)
  cmd = "ma #{encoded_key(key)} v"
  cmd << " D#{delta}" if delta
  cmd << " J#{initial}" if initial
  # Always set a TTL if an initial value is specified
  cmd << " N#{ttl || 0}" if ttl || initial
  cmd << cas_string(cas)
  cmd << ' q' if quiet
  cmd << " M#{incr ? 'I' : 'D'}"
  cmd << routing_tokens(p_token: p_token, l_token: l_token)
  cmd << TERMINATOR
end

#meta_delete(key:, cas: nil, ttl: nil, quiet: false, stale: false, drop_value: false, p_token: nil, l_token: nil) ⇒ Object

Thundering herd protection flag:

  • stale (I flag): Instead of deleting the item, mark it as stale. Other clients using N/R flags will see the X flag and know the item is being regenerated. Tombstone flags:
  • stale (I flag): mark the item stale instead of removing it. Readers using N/R flags, or get_with_metadata, see the X flag and know the item is being regenerated.
  • ttl (T flag): how long the stale marker lives. memcached only honors T on a delete when it is paired with I, so this raises rather than emitting a request the server would apply differently than intended.
  • drop_value (x flag): remove the item's value but leave the item, so a tombstone can be left without retaining the old payload.

Raises:

  • (ArgumentError) 


132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
 
# File 'lib/dalli/protocol/request_formatter.rb', line 132

def meta_delete(key:, cas: nil, ttl: nil, quiet: false, stale: false, drop_value: false,
                p_token: nil, l_token: nil)
  # Message uses this method's own parameter names (ttl/stale), not the
  # client-facing tombstone_ttl/invalidate names Dalli::Client validates
  # against -- this guard is also reachable by internal callers (tests,
  # direct RequestFormatter use) that never go through the client.
  raise ArgumentError, 'ttl requires stale: true' if ttl && !stale

  cmd = "md #{encoded_key(key)}"
  cmd << cas_string(cas)
  cmd << ' I' if stale # Mark stale instead of deleting
  cmd << " T#{Integer(ttl)}" if ttl
  cmd << ' x' if drop_value # Drop the value but keep the item
  cmd << ' q' if quiet
  cmd << routing_tokens(p_token: p_token, l_token: l_token)
  cmd << TERMINATOR
end

#meta_get(key:, value: true, return_cas: false, ttl: nil, quiet: false, vivify_ttl: nil, recache_ttl: nil, return_hit_status: false, return_last_access: false, return_ttl_remaining: false, skip_lru_bump: false, skip_flags: false, p_token: nil, l_token: nil) ⇒ Object

Since these are string construction methods, we're going to disable these Rubocop directives. We really can't make this construction much simpler, and introducing an intermediate object seems like overkill.

Meta get flags:

Thundering herd protection:

  • vivify_ttl (N flag): On miss, create a stub item and return W flag. The TTL specifies how long the stub lives. Other clients see X (stale) and Z (lost race).
  • recache_ttl (R flag): If item's remaining TTL is below this threshold, return W flag to indicate this client should recache. Other clients get Z (lost race).

Metadata flags:

  • return_hit_status (h flag): Return whether item has been hit before (0 or 1)
  • return_last_access (l flag): Return seconds since item was last accessed
  • skip_lru_bump (u flag): Don't bump item in LRU, don't update hit status or last access

Response flags (parsed by response processor):

  • W: Client won the right to recache this item
  • X: Item is stale (another client is regenerating)
  • Z: Client lost the recache race (another client is already regenerating)
  • h0/h1: Hit status (0 = first access, 1 = previously accessed)
  • l: Seconds since last access


37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
 
# File 'lib/dalli/protocol/request_formatter.rb', line 37

def meta_get(key:, value: true, return_cas: false, ttl: nil, quiet: false,
             vivify_ttl: nil, recache_ttl: nil,
             return_hit_status: false, return_last_access: false, return_ttl_remaining: false,
             skip_lru_bump: false, skip_flags: false, p_token: nil, l_token: nil)
  cmd = "mg #{encoded_key(key)}"
  # In raw mode (skip_flags: true), we don't request bitflags since they're not used.
  # This saves 2 bytes per request and skips parsing on response.
  cmd << (skip_flags ? ' v' : ' v f') if value
  cmd << ' c' if return_cas
  cmd << " T#{ttl}" if ttl
  cmd << routing_tokens(p_token: p_token, l_token: l_token)
  cmd << ' k q s' if quiet # Return the key in the response if quiet
  cmd << " N#{vivify_ttl}" if vivify_ttl # Thundering herd: vivify on miss
  cmd << " R#{recache_ttl}" if recache_ttl # Thundering herd: win recache if TTL below threshold
  cmd << ' h' if return_hit_status # Return hit status (0 or 1)
  cmd << ' l' if return_last_access # Return seconds since last access
  cmd << ' t' if return_ttl_remaining # Return seconds of TTL remaining (-1 = no TTL)
  cmd << ' u' if skip_lru_bump # Don't bump LRU or update access stats
  cmd << TERMINATOR
end

#meta_noop ⇒ Object 



218
219
220
 
# File 'lib/dalli/protocol/request_formatter.rb', line 218

def meta_noop
  META_NOOP
end

#meta_set(key:, value:, bitflags: nil, cas: nil, ttl: nil, mode: :set, quiet: false, p_token: nil, l_token: nil) ⇒ Object 



80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
 
# File 'lib/dalli/protocol/request_formatter.rb', line 80

def meta_set(key:, value:, bitflags: nil, cas: nil, ttl: nil, mode: :set, quiet: false,
             p_token: nil, l_token: nil)
  base64 = KeyRegularizer.required?(key)
  key = KeyRegularizer.encode(key) if base64
  cmd = "ms #{key} #{value.bytesize}"
  # Skip the cas-return flag in quiet mode: the response is suppressed,
  # so requesting it only adds bytes to the request.
  cmd << ' c' if !quiet && !i[append prepend].include?(mode)
  cmd << ' b' if base64
  cmd << " F#{bitflags}" if bitflags
  cmd << cas_string(cas)
  cmd << " T#{ttl}" if ttl
  cmd << " M#{mode_to_token(mode)}"
  cmd << ' q' if quiet
  cmd << routing_tokens(p_token: p_token, l_token: l_token)
  cmd << TERMINATOR
end

#multi_meta_delete(keys, stale: false, ttl: nil, drop_value: false, p_token: nil, l_token: nil) ⇒ Object

Tombstone and routing-token flags apply to every key in the batch; see meta_delete.

Raises:

  • (ArgumentError) 


152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
 
# File 'lib/dalli/protocol/request_formatter.rb', line 152

def multi_meta_delete(keys, stale: false, ttl: nil, drop_value: false, p_token: nil, l_token: nil)
  raise ArgumentError, 'tombstone_ttl requires invalidate: true' if ttl && !stale

  suffix = +''
  suffix << ' I' if stale
  suffix << " T#{Integer(ttl)}" if ttl
  suffix << ' x' if drop_value
  suffix << ' q'
  suffix << routing_tokens(p_token: p_token, l_token: l_token)
  suffix << TERMINATOR

  buffer = ''.b
  keys.each do |key|
    buffer << 'md ' << encoded_key(key) << suffix
  end
  buffer << META_NOOP
end

#multi_meta_get(keys, skip_flags: false, return_cas: false, p_token: nil, l_token: nil) ⇒ Object 



58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/dalli/protocol/request_formatter.rb', line 58

def multi_meta_get(keys, skip_flags: false, return_cas: false, p_token: nil, l_token: nil)
  # In raw mode: "mg <key> v k q s\r\n" (no f flag, key at index 2)
  # Normal mode: "mg <key> v f k q s\r\n" (key at index 3)
  # With return_cas a "c" flag follows, which shifts those indexes --
  # callers of that variant locate tokens by flag rather than position.
  # Routing tokens apply to every key in the batch, so the suffix is
  # built once rather than per key.
  post_get = if return_cas
               skip_flags ? ' v c' : ' v f c'
             else
               skip_flags ? ' v' : ' v f'
             end
  post_get += routing_tokens(p_token: p_token, l_token: l_token)
  post_get += " k q s#{TERMINATOR}"

  buffer = ''.b
  keys.each do |key|
    buffer << 'mg ' << encoded_key(key) << post_get
  end
  buffer << 'mn' << TERMINATOR
end

#multi_meta_set(entries, ttl: nil, p_token: nil, l_token: nil) ⇒ Object 



98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
 
# File 'lib/dalli/protocol/request_formatter.rb', line 98

def multi_meta_set(entries, ttl: nil, p_token: nil, l_token: nil)
  # Routing tokens apply to every entry in the batch, so the suffix is
  # built once rather than per entry.
  token_suffix = routing_tokens(p_token: p_token, l_token: l_token)

  buffer = ''.b
  entries.each do |key, pair|
    value, bitflags = pair

    base64 = KeyRegularizer.required?(key)
    key = KeyRegularizer.encode(key) if base64

    # Inline format: "ms <key> <size> c [b] F<flags> T<ttl> MS q [P/L]\r\n"
    buffer << "ms #{key} #{value.bytesize} c"
    buffer << ' b' if base64
    buffer << " F#{bitflags}" if bitflags
    buffer << " T#{ttl}" if ttl
    buffer << ' MS q' << token_suffix << TERMINATOR << value << TERMINATOR
  end
  buffer << META_NOOP
end

#routing_tokens(p_token: nil, l_token: nil) ⇒ Object

Builds the wire-format suffix for opaque routing tokens (P and L). memcached itself ignores these; they exist as hints for a proxy or router sitting between the client and memcached. See protocol.txt: "All commands accept tokens 'P' and 'L' which are completely ignored. The arguments to 'P' and 'L' can be used as hints or path specifications to a proxy or router inbetween a client and a memcached daemon."

Empty / nil tokens are treated as no-ops. CRLF and null bytes are rejected with ArgumentError to prevent the token from being used as a wire-protocol injection vector (e.g. "foo\r\nflush_all\r\n" would otherwise be parsed as a second command by memcached or any intermediate proxy/LB).



197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
 
# File 'lib/dalli/protocol/request_formatter.rb', line 197

def routing_tokens(p_token: nil, l_token: nil)
  # Only an empty *String* is a no-op. Checking respond_to?(:empty?)
  # instead would also swallow p_token: [] / {} before the type check
  # below ever runs, silently dropping caller mistakes that should
  # raise "must be a String".
  p_token = nil if p_token.is_a?(String) && p_token.empty?
  l_token = nil if l_token.is_a?(String) && l_token.empty?
  validate_routing_token!('p_token', p_token)
  validate_routing_token!('l_token', l_token)
  return '' unless p_token || l_token

  s = +''
  s << " P#{p_token}" if p_token
  s << " L#{l_token}" if l_token
  s
end

#stats(arg = nil) ⇒ Object

Raises:

  • (ArgumentError) 


235
236
237
238
239
240
241
 
# File 'lib/dalli/protocol/request_formatter.rb', line 235

def stats(arg = nil)
  raise ArgumentError, "Invalid stats argument: #{arg.inspect}" unless ALLOWED_STATS_ARGS.include?(arg)

  cmd = +'stats'
  cmd << " #{arg}" if arg && !arg.empty?
  cmd << TERMINATOR
end

#version ⇒ Object 



222
223
224
 
# File 'lib/dalli/protocol/request_formatter.rb', line 222

def version
  "version#{TERMINATOR}"
end