Class: Memcached
- Inherits:
-
Object
- Object
- Memcached
- Defined in:
- lib/memcached/memcached.rb,
lib/memcached.rb,
lib/memcached/rails.rb,
lib/memcached/behaviors.rb,
lib/memcached/exceptions.rb
Overview
The Memcached client class.
Direct Known Subclasses
Defined Under Namespace
Constant Summary collapse
- Lib =
Rlibmemcached
- VERSION =
File.read("#{File.dirname(__FILE__)}/../CHANGELOG")[/v([\d\.]+)\./, 1]
- BEHAVIORS =
load_constants("MEMCACHED_BEHAVIOR_")
- BEHAVIOR_VALUES =
{ false => 0, true => 1 }
- HASH_VALUES =
{}
- DISTRIBUTION_VALUES =
{}
- DIRECT_VALUE_BEHAVIORS =
[:retry_timeout, :connect_timeout, :rcv_timeout, :socket_recv_size, :poll_timeout, :socket_send_size, :server_failure_limit]
- CONVERSION_FACTORS =
{ :rcv_timeout => 1_000_000, :poll_timeout => 1_000, :connect_timeout => 1_000 }
- FLAGS =
0x0
- DEFAULTS =
{ :hash => :fnv1_32, :no_block => false, :distribution => :consistent_ketama, :ketama_weighted => true, :buffer_requests => false, :cache_lookups => true, :support_cas => false, :tcp_nodelay => false, :show_backtraces => false, :retry_timeout => 30, :timeout => 0.25, :rcv_timeout => nil, :poll_timeout => nil, :connect_timeout => 2, :prefix_key => nil, :hash_with_prefix_key => true, :default_ttl => 604800, :default_weight => 8, :sort_hosts => false, :auto_eject_hosts => true, :server_failure_limit => 2, :verify_key => true, :use_udp => false, :binary_protocol => false, :chunk_split_size => 1048300 }
- IGNORED =
:stopdoc:
0
- ERRNO_HASH =
- EXCEPTIONS =
[]
- EMPTY_STRUCT =
Rlibmemcached::MemcachedSt.new
Instance Attribute Summary collapse
-
#options ⇒ Object
readonly
:startdoc:.
Class Method Summary collapse
-
.load_constants(prefix, hash = {}) ⇒ Object
:stopdoc:.
Instance Method Summary collapse
-
#add(key, value, ttl = @default_ttl, marshal = true, flags = FLAGS) ⇒ Object
Add a key/value pair.
-
#append(key, value) ⇒ Object
Appends a string to a key’s value.
-
#big_get(key, marshal = true) ⇒ Object
EXPERIMENTAL.
-
#big_set(key, value, timeout = 0, marshal = true) ⇒ Object
EXPERIMENTAL.
-
#cas(key, ttl = @default_ttl, marshal = true, flags = FLAGS) ⇒ Object
(also: #compare_and_swap)
Reads a key’s value from the server and yields it to a block.
-
#clone ⇒ Object
(also: #dup)
Safely copy this instance.
-
#decrement(key, offset = 1) ⇒ Object
(also: #decr)
Decrement a key’s value.
-
#delete(key) ⇒ Object
Deletes a key/value pair from the server.
-
#flush ⇒ Object
Flushes all key/value pairs from all the servers.
-
#get(keys, marshal = true) ⇒ Object
Gets a key’s value from the server.
-
#increment(key, offset = 1) ⇒ Object
(also: #incr)
Increment a key’s value.
-
#initialize(servers = "localhost:11211", opts = {}) ⇒ Memcached
constructor
Create a new Memcached instance.
-
#prepend(key, value) ⇒ Object
Prepends a string to a key’s value.
-
#quit ⇒ Object
Disconnect from all currently connected servers.
-
#replace(key, value, ttl = @default_ttl, marshal = true, flags = FLAGS) ⇒ Object
Replace a key/value pair.
-
#reset(current_servers = nil) ⇒ Object
Reset the state of the libmemcached struct.
-
#server_by_key(key) ⇒ Object
Return the server used by a particular key.
-
#servers ⇒ Object
Return the array of server strings used to configure this instance.
-
#set(key, value, ttl = @default_ttl, marshal = true, flags = FLAGS) ⇒ Object
Set a key/value pair.
-
#set_servers(servers) ⇒ Object
Set the server list.
-
#stats ⇒ Object
Return a Hash of statistics responses from the set of servers.
Constructor Details
#initialize(servers = "localhost:11211", opts = {}) ⇒ Memcached
Create a new Memcached instance. Accepts string or array of server strings, as well an an optional configuration hash.
Memcached.new('localhost', ...) # A single server
Memcached.new(['web001:11212', 'web002:11212'], ...) # Two servers with custom ports
Memcached.new(['web001:11211:2', 'web002:11211:8'], ...) # Two servers with default ports and explicit weights
Weights only affect Ketama hashing. If you use Ketama hashing and don’t specify a weight, the client will poll each server’s stats and use its size as the weight.
Valid option parameters are:
:prefix_key
-
A string to prepend to every key, for namespacing. Max length is 127.
:hash
-
The name of a hash function to use. Possible values are:
:crc
,:default
,:fnv1_32
,:fnv1_64
,:fnv1a_32
,:fnv1a_64
,:hsieh
,:md5
, and:murmur
.:fnv1_32
is fast and well known, and is the default. Use:md5
for compatibility with other ketama clients. :distribution
-
Either
:modula
,:consistent_ketama
,:consistent_wheel
, or:ketama
. Defaults to:ketama
. :server_failure_limit
-
How many consecutive failures to allow before marking a host as dead. Has no effect unless
:retry_timeout
is also set. :retry_timeout
-
How long to wait until retrying a dead server. Has no effect unless
:server_failure_limit
is non-zero. Defaults to30
. :auto_eject_hosts
-
Whether to temporarily eject dead hosts from the pool. Defaults to
true
. Note that in the event of an ejection,:auto_eject_hosts
will remap the entire pool unless:distribution
is set to:consistent
. :cache_lookups
-
Whether to cache hostname lookups for the life of the instance. Defaults to
true
. :support_cas
-
Flag CAS support in the client. Accepts
true
orfalse
. Defaults tofalse
because it imposes a slight performance penalty. Note that your server must also support CAS or you will trigger Memcached::ProtocolError exceptions. :tcp_nodelay
-
Turns on the no-delay feature for connecting sockets. Accepts
true
orfalse
. Performance may or may not change, depending on your system. :no_block
-
Whether to use pipelining for writes. Accepts
true
orfalse
. :buffer_requests
-
Whether to use an internal write buffer. Accepts
true
orfalse
. Callingget
or closing the connection will force the buffer to flush. Note that:buffer_requests
might not work well without:no_block
also enabled. :show_backtraces
-
Whether Memcached::NotFound and Memcached::NotStored exceptions should include backtraces. Generating backtraces is slow, so this is off by default. Turn it on to ease debugging.
:connect_timeout
-
How long to wait for a connection to a server. Defaults to 2 seconds. Set to
0
if you want to wait forever. :timeout
-
How long to wait for a response from the server. Defaults to 0.25 seconds. Set to
0
if you want to wait forever. :default_ttl
-
The
ttl
to use on set if nottl
is specified, in seconds. Defaults to one week. Set to0
if you want things to never expire. :default_weight
-
The weight to use if
:ketama_weighted
istrue
, but no weight is specified for a server. :hash_with_prefix_key
-
Whether to include the prefix when calculating which server a key falls on. Defaults to
true
. :use_udp
-
Use the UDP protocol to reduce connection overhead. Defaults to false.
:binary_protocol
-
Use the binary protocol to reduce query processing overhead. Defaults to false.
:sort_hosts
-
Whether to force the server list to stay sorted. This defeats consistent hashing and is rarely useful.
:verify_key
-
Validate keys before accepting them. Never disable this.
:chunk_split_size
-
Number of bytes after which items that are bigger than the 1MB memcached bucket max. size are split.
Please note that when pipelining is enabled, setter and deleter methods do not raise on errors. For example, if you try to set an invalid key with :no_block => true
, it will appear to succeed. The actual setting of the key occurs after libmemcached has returned control to your program, so there is no way to backtrack and raise the exception.
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 |
# File 'lib/memcached/memcached.rb', line 84 def initialize(servers = "localhost:11211", opts = {}) @struct = Lib::MemcachedSt.new Lib.memcached_create(@struct) # Merge option defaults and discard meaningless keys @options = DEFAULTS.merge(opts) @options.delete_if { |k,v| not DEFAULTS.keys.include? k } @default_ttl = [:default_ttl] # Force :buffer_requests to use :no_block # XXX Deleting the :no_block key should also work, but libmemcached doesn't seem to set it # consistently [:no_block] = true if [:buffer_requests] # Disallow weights without ketama .delete(:ketama_weighted) if [:distribution] != :consistent_ketama # Legacy accessor [:prefix_key] = .delete(:namespace) if [:namespace] # Disallow :sort_hosts with consistent hashing if [:sort_hosts] and [:distribution] == :consistent raise ArgumentError, ":sort_hosts defeats :consistent hashing" end # Read timeouts [:rcv_timeout] ||= [:timeout] || 0 [:poll_timeout] ||= [:timeout] || 0 # Set the behaviors on the struct set_behaviors set_callbacks # Freeze the hash .freeze # Set the servers on the struct set_servers(servers) # Not found exceptions unless [:show_backtraces] @not_found = NotFound.new @not_found.no_backtrace = true @not_stored = NotStored.new @not_stored.no_backtrace = true end end |
Instance Attribute Details
#options ⇒ Object (readonly)
:startdoc:
42 43 44 |
# File 'lib/memcached/memcached.rb', line 42 def @options end |
Class Method Details
.load_constants(prefix, hash = {}) ⇒ Object
:stopdoc:
6 7 8 9 10 11 |
# File 'lib/memcached/behaviors.rb', line 6 def self.load_constants(prefix, hash = {}) Lib.constants.grep(/^#{prefix}/).each do |const_name| hash[const_name[prefix.length..-1].downcase.to_sym] = Lib.const_get(const_name) end hash end |
Instance Method Details
#add(key, value, ttl = @default_ttl, marshal = true, flags = FLAGS) ⇒ Object
Add a key/value pair. Raises Memcached::NotStored if the key already exists on the server. The parameters are the same as set
.
267 268 269 270 271 272 273 |
# File 'lib/memcached/memcached.rb', line 267 def add(key, value, ttl=@default_ttl, marshal=true, flags=FLAGS) value = marshal ? Marshal.dump(value) : value.to_s check_return_code( Lib.memcached_add(@struct, key, value, ttl, flags), key ) end |
#append(key, value) ⇒ Object
Appends a string to a key’s value. Accepts a String key
and a String value
. Raises Memcached::NotFound if the key does not exist on the server.
Note that the key must be initialized to an unmarshalled string first, via set
, add
, or replace
with marshal
set to false
.
310 311 312 313 314 315 316 |
# File 'lib/memcached/memcached.rb', line 310 def append(key, value) # Requires memcached 1.2.4 check_return_code( Lib.memcached_append(@struct, key, value.to_s, IGNORED, IGNORED), key ) end |
#big_get(key, marshal = true) ⇒ Object
EXPERIMENTAL
Gets a key’s value from the server. Accepts a single String key.
Accepts an optional marshal
argument, which defaults to true
, like get
.
406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 |
# File 'lib/memcached/memcached.rb', line 406 def big_get(key, marshal=true) chunk_header = get(key, true) # A valid chunk header should respond to #chunks. return marshal ? chunk_header : get(key, marshal) unless chunk_header.respond_to?(:chunks) chunks = chunk_header.chunks chunk_keys = (0...chunks).map { |i| "#{key}_#{i}" } # Use multiget #get - this returns a hash of key/value pairs. chunky_hash_browns = get(chunk_keys, false) # If any of the chunks are missing, the entire item is considered missing. raise Memcached::NotFound if chunky_hash_browns.size != chunks # Concat all the chunked values. value = chunky_hash_browns.sort.map { |k, v| v }.join value = Marshal.load(value) if marshal value end |
#big_set(key, value, timeout = 0, marshal = true) ⇒ Object
EXPERIMENTAL
Sets a key/value pair that is (potentially) bigger than 1MB (i.e. the memcached limit for bucket sizes).
Accepts optional timeout
and marshal
arguments like set
.
The value is split into chunks (each smaller than or equal in size to the chunk_split_size
option) and inserted into separate buckets. The keys are of the form: #key_0, #key_1, #key_2. The bucket referred to by the given key
contains a chunk “header” with a #chunks property that equals the number of chunks.
Note that values that fit within a single chunk are still “split” - the chunk header (and the single chunk) is still set.
WARNING: This method is non-atomic by nature, since we are really performing multiple set
s serially.
252 253 254 255 256 257 258 259 260 261 262 263 264 |
# File 'lib/memcached/memcached.rb', line 252 def big_set(key, value, timeout=0, marshal=true) value = marshal ? Marshal.dump(value) : value.to_s chunk_size = [:chunk_split_size] chunks = (value.size/chunk_size.to_f).ceil # Set the number of chunks (in a faux "header") in the bucket for the actual key. set(key, OpenStruct.new(:chunks => chunks), timeout, true) chunks.times do |chunk_num| set("#{key}_#{chunk_num}", value[chunk_num * chunk_size, chunk_size], timeout, false) end end |
#cas(key, ttl = @default_ttl, marshal = true, flags = FLAGS) ⇒ Object Also known as: compare_and_swap
Reads a key’s value from the server and yields it to a block. Replaces the key’s value with the result of the block as long as the key hasn’t been updated in the meantime, otherwise raises Memcached::NotStored. Accepts a String key
and a block.
Also accepts an optional ttl
value.
CAS stands for “compare and swap”, and avoids the need for manual key mutexing. CAS support must be enabled in Memcached.new or a Memcached::ClientError will be raised. Note that CAS may be buggy in memcached itself.
333 334 335 336 337 338 339 340 341 342 343 344 345 |
# File 'lib/memcached/memcached.rb', line 333 def cas(key, ttl=@default_ttl, marshal=true, flags=FLAGS) raise ClientError, "CAS not enabled for this Memcached instance" unless [:support_cas] value, flags, ret = Lib.memcached_get_rvalue(@struct, key) check_return_code(ret, key) cas = @struct.result.cas value = Marshal.load(value) if marshal value = yield value value = Marshal.dump(value) if marshal check_return_code(Lib.memcached_cas(@struct, key, value, ttl, flags, cas), key) end |
#clone ⇒ Object Also known as: dup
Safely copy this instance. Returns a Memcached instance.
clone
is useful for threading, since each thread must have its own unshared Memcached object.
169 170 171 172 173 174 175 176 |
# File 'lib/memcached/memcached.rb', line 169 def clone # FIXME Memory leak # memcached = super # struct = Lib.memcached_clone(nil, @struct) # memcached.instance_variable_set('@struct', struct) # memcached self.class.new(servers, ) end |
#decrement(key, offset = 1) ⇒ Object Also known as: decr
Decrement a key’s value. The parameters and exception behavior are the same as increment
.
287 288 289 290 291 |
# File 'lib/memcached/memcached.rb', line 287 def decrement(key, offset=1) ret, value = Lib.memcached_decrement(@struct, key, offset) check_return_code(ret, key) value end |
#delete(key) ⇒ Object
Deletes a key/value pair from the server. Accepts a String key
. Raises Memcached::NotFound if the key does not exist.
352 353 354 355 356 357 |
# File 'lib/memcached/memcached.rb', line 352 def delete(key) check_return_code( Lib.memcached_delete(@struct, key, IGNORED), key ) end |
#flush ⇒ Object
Flushes all key/value pairs from all the servers.
360 361 362 363 364 |
# File 'lib/memcached/memcached.rb', line 360 def flush check_return_code( Lib.memcached_flush(@struct, IGNORED) ) end |
#get(keys, marshal = true) ⇒ Object
Gets a key’s value from the server. Accepts a String key
or array of String keys
.
Also accepts a marshal
value, which defaults to true
. Set marshal
to false
if you want the value
to be returned directly as a String. Otherwise it will be assumed to be a marshalled Ruby object and unmarshalled.
If you pass a String key, and the key does not exist on the server, Memcached::NotFound will be raised. If you pass an array of keys, memcached’s multiget
mode will be used, and a hash of key/value pairs will be returned. The hash will contain only the keys that were found.
The multiget behavior is subject to change in the future; however, for multiple lookups, it is much faster than normal mode.
Note that when you rescue Memcached::NotFound exceptions, you should use a the block rescue syntax instead of the inline syntax. Block rescues are very fast, but inline rescues are very slow.
378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 |
# File 'lib/memcached/memcached.rb', line 378 def get(keys, marshal=true) if keys.is_a? Array # Multi get ret = Lib.memcached_mget(@struct, keys); check_return_code(ret, keys) hash = {} keys.each do value, key, flags, ret = Lib.memcached_fetch_rvalue(@struct) break if ret == Lib::MEMCACHED_END check_return_code(ret, key) # Assign the value hash[key] = (marshal ? Marshal.load(value) : value) end hash else # Single get value, flags, ret = Lib.memcached_get_rvalue(@struct, keys) check_return_code(ret, keys) marshal ? Marshal.load(value) : value end end |
#increment(key, offset = 1) ⇒ Object Also known as: incr
Increment a key’s value. Accepts a String key
. Raises Memcached::NotFound if the key does not exist.
Also accepts an optional offset
paramater, which defaults to 1. offset
must be an integer.
Note that the key must be initialized to an unmarshalled integer first, via set
, add
, or replace
with marshal
set to false
.
280 281 282 283 284 |
# File 'lib/memcached/memcached.rb', line 280 def increment(key, offset=1) ret, value = Lib.memcached_increment(@struct, key, offset) check_return_code(ret, key) value end |
#prepend(key, value) ⇒ Object
Prepends a string to a key’s value. The parameters and exception behavior are the same as append
.
319 320 321 322 323 324 325 |
# File 'lib/memcached/memcached.rb', line 319 def prepend(key, value) # Requires memcached 1.2.4 check_return_code( Lib.memcached_prepend(@struct, key, value.to_s, IGNORED, IGNORED), key ) end |
#quit ⇒ Object
Disconnect from all currently connected servers
189 190 191 192 |
# File 'lib/memcached/memcached.rb', line 189 def quit Lib.memcached_quit(@struct) self end |
#replace(key, value, ttl = @default_ttl, marshal = true, flags = FLAGS) ⇒ Object
Replace a key/value pair. Raises Memcached::NotFound if the key does not exist on the server. The parameters are the same as set
.
299 300 301 302 303 304 305 |
# File 'lib/memcached/memcached.rb', line 299 def replace(key, value, ttl=@default_ttl, marshal=true, flags=FLAGS) value = marshal ? Marshal.dump(value) : value.to_s check_return_code( Lib.memcached_replace(@struct, key, value, ttl, flags), key ) end |
#reset(current_servers = nil) ⇒ Object
Reset the state of the libmemcached struct. This is useful for changing the server list at runtime.
179 180 181 182 183 184 185 186 |
# File 'lib/memcached/memcached.rb', line 179 def reset(current_servers = nil) current_servers ||= servers @struct = Lib::MemcachedSt.new Lib.memcached_create(@struct) set_behaviors set_callbacks set_servers(current_servers) end |
#server_by_key(key) ⇒ Object
Return the server used by a particular key.
430 431 432 433 434 435 436 437 |
# File 'lib/memcached/memcached.rb', line 430 def server_by_key(key) ret = Lib.memcached_server_by_key(@struct, key) if ret.is_a?(Array) string = inspect_server(ret.first) Rlibmemcached.memcached_server_free(ret.first) string end end |
#servers ⇒ Object
Return the array of server strings used to configure this instance.
158 159 160 161 162 |
# File 'lib/memcached/memcached.rb', line 158 def servers server_structs.map do |server| inspect_server(server) end end |
#set(key, value, ttl = @default_ttl, marshal = true, flags = FLAGS) ⇒ Object
Set a key/value pair. Accepts a String key
and an arbitrary Ruby object. Overwrites any existing value on the server.
Accepts an optional ttl
value to specify the maximum lifetime of the key on the server. ttl
can be either an integer number of seconds, or a Time elapsed time object. 0
means no ttl. Note that there is no guarantee that the key will persist as long as the ttl
, but it will not persist longer.
Also accepts a marshal
value, which defaults to true
. Set marshal
to false
if you want the value
to be set directly.
225 226 227 228 229 230 231 232 233 234 235 |
# File 'lib/memcached/memcached.rb', line 225 def set(key, value, ttl=@default_ttl, marshal=true, flags=FLAGS) value = marshal ? Marshal.dump(value) : value.to_s check_return_code( Lib.memcached_set(@struct, key, value, ttl, flags), key ) rescue ClientError # FIXME Memcached 1.2.8 occasionally rejects valid sets tried = 1 and retry unless defined?(tried) raise end |
#set_servers(servers) ⇒ Object
Set the server list. FIXME Does not necessarily free any existing server structs.
134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 |
# File 'lib/memcached/memcached.rb', line 134 def set_servers(servers) Array(servers).each_with_index do |server, index| # Socket if server.is_a?(String) and File.socket?(server) args = [@struct, server, [:default_weight].to_i] check_return_code(Lib.memcached_server_add_unix_socket_with_weight(*args)) # Network elsif server.is_a?(String) and server =~ /^[\w\d\.-]+(:\d{1,5}){0,2}$/ host, port, weight = server.split(":") args = [@struct, host, port.to_i, (weight || [:default_weight]).to_i] if [:use_udp] # check_return_code(Lib.memcached_server_add_udp_with_weight(*args)) else check_return_code(Lib.memcached_server_add_with_weight(*args)) end else raise ArgumentError, "Servers must be either in the format 'host:port[:weight]' (e.g., 'localhost:11211' or 'localhost:11211:10') for a network server, or a valid path to a Unix domain socket (e.g., /var/run/memcached)." end end # For inspect @servers = send(:servers) end |
#stats ⇒ Object
Return a Hash of statistics responses from the set of servers. Each value is an array with one entry for each server, in the same order the servers were defined.
440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 |
# File 'lib/memcached/memcached.rb', line 440 def stats stats = Hash.new([]) stat_struct, ret = Lib.memcached_stat(@struct, "") check_return_code(ret) keys, ret = Lib.memcached_stat_get_keys(@struct, stat_struct) check_return_code(ret) keys.each do |key| server_structs.size.times do |index| value, ret = Lib.memcached_stat_get_rvalue( @struct, Lib.memcached_select_stat_at(@struct, stat_struct, index), key) check_return_code(ret, key) value = case value when /^\d+\.\d+$/ then value.to_f when /^\d+$/ then value.to_i else value end stats[key.to_sym] += [value] end end Lib.memcached_stat_free(@struct, stat_struct) stats rescue Memcached::SomeErrorsWereReported => _ e = _.class.new("Error getting stats") e.set_backtrace(_.backtrace) raise e end |