Class: Rufus::Tokyo::Cabinet

Inherits:
Object
  • Object
show all
Extended by:
Openable
Includes:
HashMethods, Outlen, Transactions
Defined in:
lib/rufus/tokyo/cabinet/abstract.rb

Overview

A ‘cabinet’, ie a Tokyo Cabinet [abstract] database.

Follows the abstract API described at :

http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi

An usage example :

db = Rufus::Tokyo::Cabinet.new('test_data.tch')
db['pillow'] = 'Shonagon'

db.size # => 1
db['pillow'] # => 'Shonagon'

db.delete('pillow') # => 'Shonagon'
db.size # => 0

db.close

Direct Known Subclasses

Tyrant

Instance Attribute Summary

Attributes included from HashMethods

#default_proc

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Openable

open

Methods included from Outlen

#outlen_op

Methods included from Transactions

#abort, #transaction

Methods included from HashMethods

#[], #default, #default=, #each, #merge, #to_a, #to_h, #values

Constructor Details

#initialize(name, params = {}) ⇒ Cabinet

Creates/opens the cabinet, raises an exception in case of creation/opening failure.

This method accepts a ‘name’ parameter and an optional ‘params’ hash parameter.

‘name’ follows the syntax described at

http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi

under tcadbopen(). For example :

db = Rufus::Tokyo::Cabinet.new('casket.tch#bnum=100000#opts=ld')

will open (eventually create) a hash database backed in the file ‘casket.tch’ with a bucket number of 100000 and the ‘large’ and ‘deflate’ options (opts) turned on.

Note that there is an #open method similar to File#open for openening a db and closing it when it’s no longer needed :

Rufus::Tokyo::Cabinet.new('data.tch') do |db|
  db['key'] = value
end

database name

From tokyocabinet.sourceforge.net/spex-en.html#tcadbapi :

‘If it is “*”, the database will be an on-memory hash database. If it is

"+", the database will be an on-memory tree database. If its suffix is
".tch", the database will be a hash database. If its suffix is ".tcb",
the database will be a B+ tree database. If its suffix is ".tcf", the
database will be a fixed-length database. If its suffix is ".tct", the
database will be a table database.'

You’re supposed to give a path to the database file you want to use and Cabinet expects you to give the proper prefix.

db = Rufus::Tokyo::Cabinet.new('data.tch') # hash database
db = Rufus::Tokyo::Cabinet.new('data.tcb') # B+ tree db
db = Rufus::Tokyo::Cabinet.new('data.tcf') # fixed-length db

will result with the same file names :

db = Rufus::Tokyo::Cabinet.new('data', :type => :hash) # hash database
db = Rufus::Tokyo::Cabinet.new('data', :type => :btree) # B+ tree db
db = Rufus::Tokyo::Cabinet.new('data', :type => :fixed) # fixed-length db

You can open an in-memory hash and an in-memory B+ tree with :

h = Rufus::Tokyo::Cabinet.new(:mem_hash) # or
h = Rufus::Tokyo::Cabinet.new('*')

t = Rufus::Tokyo::Cabinet.new(:mem_tree) # or
t = Rufus::Tokyo::Cabinet.new('+')

parameters

There are two ways to pass parameters at the opening of a db :

db = Rufus::Tokyo::Cabinet.new('data.tch#opts=ld#mode=w') # or
db = Rufus::Tokyo::Cabinet.new('data.tch', :opts => 'ld', :mode => 'w')

most verbose :

db = Rufus::Tokyo::Cabinet.new(
  'data', :type => :hash, :opts => 'ld', :mode => 'w')

:mode

* :mode    a set of chars ('r'ead, 'w'rite, 'c'reate, 't'runcate,
           'e' non locking, 'f' non blocking lock), default is 'wc'

:default and :default_proc

Much like a Ruby Hash, a Cabinet accepts a default value or a default_proc

db = Rufus::Tokyo::Cabinet.new('data.tch', :default => 'xxx')
db['fred'] = 'Astaire'
p db['fred'] # => 'Astaire'
p db['ginger'] # => 'xxx'

db = Rufus::Tokyo::Cabinet.new(
  'data.tch',
  :default_proc => lambda { |cab, key| "not found : '#{k}'" }
p db['ginger'] # => "not found : 'ginger'"

The first arg passed to the default_proc is the cabinet itself, so this opens up interesting possibilities.

other parameters

‘On-memory hash database supports “bnum”, “capnum”, and “capsiz”.

On-memory tree database supports "capnum" and "capsiz".
Hash database supports "mode", "bnum", "apow", "fpow", "opts",
"rcnum", and "xmsiz".
B+ tree database supports "mode", "lmemb", "nmemb", "bnum", "apow",
"fpow", "opts", "lcnum", "ncnum", and "xmsiz".
Fixed-length database supports "mode", "width", and "limsiz"'

 * :opts    a set of chars ('l'arge, 'd'eflate, 'b'zip2, 't'cbs)
            (usually empty or something like 'ld' or 'lb')

 * :bnum    number of elements of the bucket array
 * :apow    size of record alignment by power of 2 (defaults to 4)
 * :fpow    maximum number of elements of the free block pool by
            power of 2 (defaults to 10)
 * :mutex   when set to true, makes sure only 1 thread at a time
            accesses the table (well, Ruby, global thread lock, ...)

 * :rcnum   specifies the maximum number of records to be cached.
            If it is not more than 0, the record cache is disabled.
            It is disabled by default.
 * :lcnum   specifies the maximum number of leaf nodes to be cached.
            If it is not more than 0, the default value is specified.
            The default value is 2048.
 * :ncnum   specifies the maximum number of non-leaf nodes to be
            cached. If it is not more than 0, the default value is
            specified. The default value is 512.

 * :xmsiz   specifies the size of the extra mapped memory. If it is
            not more than 0, the extra mapped memory is disabled.
            The default size is 67108864.

 * :capnum  specifies the capacity number of records.
 * :capsiz  specifies the capacity size of using memory.

 * :dfunit  unit step number. If it is not more than 0,
            the auto defragmentation is disabled. (Since TC 1.4.21)

NOTE :

On reopening a file, Cabinet will tend to stick to the parameters as set when the file was opened. To change that, have a look at the man pages of the various command line tools coming with Tokyo Cabinet.



201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 201

def initialize (name, params={})

  #conf = determine_conf(path, params)
    # not using it

  @db = lib.tcadbnew

  name = '*' if name == :mem_hash # in memory hash database
  name = '+' if name == :mem_tree # in memory B+ tree database

  if type = params.delete(:type)
    name += { :hash => '.tch', :btree => '.tcb', :fixed => '.tcf' }[type]
  end

  @path = name
  @type = File.extname(@path)[1..-1]

  name = name + params.collect { |k, v| "##{k}=#{v}" }.join('')

  lib.tcadbopen(@db, name) || raise(
    TokyoError.new("failed to open/create db '#{name}' #{params.inspect}"))

  #
  # default value|proc

  self.default = params[:default]
  @default_proc ||= params[:default_proc]
end

Class Method Details

.new_hash(params = {}) ⇒ Object

Returns a new in-memory hash. Accepts the same optional params hash as new().



233
234
235
236
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 233

def self.new_hash (params={})

  self.new(:hash, params)
end

.new_tree(params = {}) ⇒ Object

Returns a new in-memory B+ tree. Accepts the same optional params hash as new().



241
242
243
244
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 241

def self.new_tree (params={})

  self.new(:tree, params)
end

Instance Method Details

#[]=(k, v) ⇒ Object

No comment



262
263
264
265
266
267
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 262

def []= (k, v)

  k = k.to_s; v = v.to_s

  lib.abs_put(@db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v))
end

#clear ⇒ Object

Removes all the records in the cabinet (use with care)

Returns self (like Ruby’s Hash does).



324
325
326
327
328
329
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 324

def clear

  lib.abs_vanish(@db)

  self
end

#close ⇒ Object

Closes the cabinet (and frees the datastructure allocated for it), returns true in case of success.



341
342
343
344
345
346
347
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 341

def close

  result = lib.abs_close(@db)
  lib.abs_del(@db)

  result
end

#compact_copy(target_path) ⇒ Object

Copies the current cabinet to a new file.

Does it by copying each entry afresh to the target file. Spares some space, hence the ‘compact’ label…



363
364
365
366
367
368
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 363

def compact_copy (target_path)

  @other_db = Cabinet.new(target_path)
  self.each { |k, v| @other_db[k] = v }
  @other_db.close
end

#copy(target_path) ⇒ Object

Copies the current cabinet to a new file.

Returns true if it was successful.



353
354
355
356
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 353

def copy (target_path)

  lib.abs_copy(@db, target_path)
end

#counter_value(key) ⇒ Object

Returns the current value for a counter (a float or an int).

See #incr



491
492
493
494
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 491

def counter_value (key)

  incr(key, 0.0) rescue incr(key, 0)
end

#defrag ⇒ Object

Triggers a defrag run (TC >= 1.4.21 only)

Raises:

  • (NotImplementedError) 


498
499
500
501
502
503
504
505
506
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 498

def defrag

  raise(NotImplementedError.new(
    "method defrag is supported since Tokyo Cabinet 1.4.21. " +
    "your TC version doesn't support it"
  )) unless lib.respond_to?(:tctdbsetdfunit)

  call_misc('defrag', Rufus::Tokyo::List.new)
end

#delete(k) ⇒ Object

Removes a record from the cabinet, returns the value if successful else nil.



304
305
306
307
308
309
310
311
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 304

def delete (k)

  k = k.to_s

  v = self[k]

  lib.abs_out(@db, k, Rufus::Tokyo.blen(k)) ? v : nil
end

#delete_keys_with_prefix(prefix) ⇒ Object

Deletes all the entries whose keys begin with the given prefix



411
412
413
414
415
416
417
418
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 411

def delete_keys_with_prefix (prefix)

  call_misc(
    'outlist', lib.abs_fwmkeys(@db, prefix, Rufus::Tokyo.blen(prefix), -1))
      # -1 for no limits

  nil
end

#get4(k) ⇒ Object Also known as: getdup

This is a B+ Tree method only, returns all the values for a given key.



562
563
564
565
566
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 562

def get4 (k)

  l = lib.tcbdbget4(as_btree, k, Rufus::Tokyo.blen(k))
  Rufus::Tokyo::List.new(l).release
end

#incr(key, inc = 1) ⇒ Object Also known as: addint, adddouble, add_int, add_double

Increments the value stored under the given key with the given increment (defaults to 1 (integer)).

Accepts an integer or a double value.

Warning : Tokyo Cabinet/Tyrant doesn’t store counter values as regular strings (db won’t yield something that replies properly to #to_i)

Use #counter_value(k) to get the current value set for the counter.

Raises:



467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 467

def incr (key, inc=1)

  key = key.to_s

  v = inc.is_a?(Fixnum) ?
    lib.addint(@db, key, Rufus::Tokyo.blen(key), inc) :
    lib.adddouble(@db, key, Rufus::Tokyo.blen(key), inc)

  raise(TokyoError.new(
    "incr failed, there is probably already a string value set " +
    "for the key '#{key}'. Make sure there is no value before incrementing"
  )) if v == Rufus::Tokyo::INT_MIN || (v.respond_to?(:nan?) && v.nan?)

  v
end

#keys(options = {}) ⇒ Object

Returns an array with all the keys in the databse

With no options given, this method will return all the keys (strings) in a Ruby array.

:prefix --> returns only the keys who match a given string prefix

:limit --> returns a limited number of keys

:native --> returns an instance of Rufus::Tokyo::List instead of
  a Ruby Hash, you have to call #free on that List when done with it !
  Else you're exposing yourself to a memory leak.



391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 391

def keys (options={})

  if @type == "tcf"
    min, max = "min", "max"
    l        = lib.tcfdbrange2( as_fixed, min, Rufus::Tokyo.blen(min),
                                          max, Rufus::Tokyo.blen(max), -1)
  else
    pre = options.fetch(:prefix, "")

    l = lib.abs_fwmkeys(
      @db, pre, Rufus::Tokyo.blen(pre), options[:limit] || -1)
  end

  l = Rufus::Tokyo::List.new(l)

  options[:native] ? l : l.release
end

#ldelete(*keys) ⇒ Object

Given a list of keys, deletes all the matching entries (in one sweep).



450
451
452
453
454
455
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 450

def ldelete (*keys)

  call_misc(
    'outlist',
    Rufus::Tokyo::List.new(keys.flatten.collect { |k| k.to_s }))
end

#lget(*keys) ⇒ Object Also known as: mget

Given a list of keys, returns a Hash { key => value } of the matching entries (in one sweep).



423
424
425
426
427
428
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 423

def lget (*keys)

  keys = keys.flatten.collect { |k| k.to_s }

  Hash[*call_misc('getlist', Rufus::Tokyo::List.new(keys))]
end

#lib ⇒ Object

Using the cabinet lib



248
249
250
251
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 248

def lib

  CabinetLib
end

#merge!(hash) ⇒ Object Also known as: lput

Merges the given hash into this Cabinet (or Tyrant) and returns self.



434
435
436
437
438
439
440
441
442
443
444
445
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 434

def merge! (hash)

  call_misc(
    'putlist',
    hash.inject(Rufus::Tokyo::List.new) { |l, (k, v)|
      l << k.to_s
      l << v.to_s
      l
    })

  self
end

#path ⇒ Object

Returns the path to this database.



255
256
257
258
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 255

def path

  @path
end

#putcat(k, v) ⇒ Object

Appends the given string at the end of the current string value for key k. If there is no record for key k, a new record will be created.

Returns true if successful.



284
285
286
287
288
289
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 284

def putcat (k, v)

  k = k.to_s; v = v.to_s

  lib.abs_putcat(@db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v))
end

#putdup(k, v) ⇒ Object

This is a B+ Tree method only, puts a value for a key who has

potentially

multiple values.



553
554
555
556
557
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 553

def putdup (k, v)

  lib.tcbdbputdup(
    as_btree, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v))
end

#putkeep(k, v) ⇒ Object

Like #put but doesn’t overwrite the value if already set. Returns true only if there no previous entry for k.



272
273
274
275
276
277
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 272

def putkeep (k, v)

  k = k.to_s; v = v.to_s

  lib.abs_putkeep(@db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v))
end

#size ⇒ Object

Returns the number of records in the ‘cabinet’



315
316
317
318
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 315

def size

  lib.abs_rnum(@db)
end

#sync ⇒ Object

“synchronize updated contents of an abstract database object with the file and the device”



373
374
375
376
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 373

def sync

  lib.abs_sync(@db)
end

#tranabort ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction abort’.



537
538
539
540
541
542
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 537

def tranabort

  #check_transaction_support

  libcall(:tcadbtranabort)
end

#tranbegin ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction begin’.



513
514
515
516
517
518
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 513

def tranbegin

  #check_transaction_support

  libcall(:tcadbtranbegin)
end

#trancommit ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction commit’.



525
526
527
528
529
530
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 525

def trancommit

  #check_transaction_support

  libcall(:tcadbtrancommit)
end

#weight ⇒ Object

Returns the ‘weight’ of the db (in bytes)



333
334
335
336
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 333

def weight

  lib.abs_size(@db)
end