Class: Rufus::Tokyo::Cabinet

Inherits:
Object
  • Object
show all
Includes:
HashMethods, Outlen, Transactions
Defined in:
lib/rufus/tokyo/cabinet/abstract.rb

Overview

A ‘cabinet’, ie a Tokyo Cabinet [abstract] database.

Follows the abstract API described at :

http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi

An usage example :

db = Rufus::Tokyo::Cabinet.new('test_data.tch')
db['pillow'] = 'Shonagon'

db.size # => 1
db['pillow'] # => 'Shonagon'

db.delete('pillow') # => 'Shonagon'
db.size # => 0

db.close

Direct Known Subclasses

Tyrant

Instance Attribute Summary

Attributes included from HashMethods

#default_proc

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Outlen

#outlen_op

Methods included from Transactions

#abort, #transaction

Methods included from HashMethods

#[], #default, #default=, #each, #merge, #to_a, #to_h, #values

Constructor Details

#initialize(name, params = {}) ⇒ Cabinet

Creates/opens the cabinet, raises an exception in case of creation/opening failure.

This method accepts a ‘name’ parameter and an optional ‘params’ hash parameter.

‘name’ follows the syntax described at

http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi

under tcadbopen(). For example :

db = Rufus::Tokyo::Cabinet.new('casket.tch#bnum=100000#opts=ld')

will open (eventually create) a hash database backed in the file ‘casket.tch’ with a bucket number of 100000 and the ‘large’ and ‘deflate’ options (opts) turned on.

Note that there is an #open method similar to File#open for openening a db and closing it when it’s no longer needed :

Rufus::Tokyo::Cabinet.new('data.tch') do |db|
  db['key'] = value
end

database name

From tokyocabinet.sourceforge.net/spex-en.html#tcadbapi :

‘If it is “*”, the database will be an on-memory hash database. If it is

"+", the database will be an on-memory tree database. If its suffix is
".tch", the database will be a hash database. If its suffix is ".tcb",
the database will be a B+ tree database. If its suffix is ".tcf", the
database will be a fixed-length database. If its suffix is ".tct", the
database will be a table database.'

You’re supposed to give a path to the database file you want to use and Cabinet expects you to give the proper prefix.

db = Rufus::Tokyo::Cabinet.new('data.tch') # hash database
db = Rufus::Tokyo::Cabinet.new('data.tcb') # B+ tree db
db = Rufus::Tokyo::Cabinet.new('data.tcf') # fixed-length db

will result with the same file names :

db = Rufus::Tokyo::Cabinet.new('data', :type => :hash) # hash database
db = Rufus::Tokyo::Cabinet.new('data', :type => :btree) # B+ tree db
db = Rufus::Tokyo::Cabinet.new('data', :type => :fixed) # fixed-length db

You can open an in-memory hash and an in-memory B+ tree with :

h = Rufus::Tokyo::Cabinet.new(:mem_hash) # or
h = Rufus::Tokyo::Cabinet.new('*')

t = Rufus::Tokyo::Cabinet.new(:mem_tree) # or
t = Rufus::Tokyo::Cabinet.new('+')

parameters

There are two ways to pass parameters at the opening of a db :

db = Rufus::Tokyo::Cabinet.new('data.tch#opts=ld#mode=w') # or
db = Rufus::Tokyo::Cabinet.new('data.tch', :opts => 'ld', :mode => 'w')

most verbose :

db = Rufus::Tokyo::Cabinet.new(
  'data', :type => :hash, :opts => 'ld', :mode => 'w')

mode

* :mode    a set of chars ('r'ead, 'w'rite, 'c'reate, 't'runcate,
           'e' non locking, 'f' non blocking lock), default is 'wc'

other parameters

‘On-memory hash database supports “bnum”, “capnum”, and “capsiz”.

On-memory tree database supports "capnum" and "capsiz".
Hash database supports "mode", "bnum", "apow", "fpow", "opts",
"rcnum", and "xmsiz".
B+ tree database supports "mode", "lmemb", "nmemb", "bnum", "apow",
"fpow", "opts", "lcnum", "ncnum", and "xmsiz".
Fixed-length database supports "mode", "width", and "limsiz"'

 * :opts    a set of chars ('l'arge, 'd'eflate, 'b'zip2, 't'cbs)
            (usually empty or something like 'ld' or 'lb')

 * :bnum    number of elements of the bucket array
 * :apow    size of record alignment by power of 2 (defaults to 4)
 * :fpow    maximum number of elements of the free block pool by
            power of 2 (defaults to 10)
 * :mutex   when set to true, makes sure only 1 thread at a time
            accesses the table (well, Ruby, global thread lock, ...)

 * :rcnum   specifies the maximum number of records to be cached.
            If it is not more than 0, the record cache is disabled.
            It is disabled by default.
 * :lcnum   specifies the maximum number of leaf nodes to be cached.
            If it is not more than 0, the default value is specified.
            The default value is 2048.
 * :ncnum   specifies the maximum number of non-leaf nodes to be
            cached. If it is not more than 0, the default value is
            specified. The default value is 512.

 * :xmsiz   specifies the size of the extra mapped memory. If it is
            not more than 0, the extra mapped memory is disabled.
            The default size is 67108864.

 * :capnum  specifies the capacity number of records.
 * :capsiz  specifies the capacity size of using memory.

 * :dfunit  unit step number. If it is not more than 0,
            the auto defragmentation is disabled. (Since TC 1.4.21)

NOTE :

On reopening a file, Cabinet will tend to stick to the parameters as set when the file was opened. To change that, have a look at the man pages of the various command line tools coming with Tokyo Cabinet.



179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 179

def initialize (name, params={})

  @db = lib.tcadbnew

  name = '*' if name == :mem_hash # in memory hash database
  name = '+' if name == :mem_tree # in memory B+ tree database

  if type = params.delete(:type)
    name += { :hash => '.tch', :btree => '.tcb', :fixed => '.tcf' }[type]
  end

  @path = name

  name = name + params.collect { |k, v| "##{k}=#{v}" }.join('')

  (lib.tcadbopen(@db, name) == 1) || raise(
    TokyoError.new("failed to open/create db '#{name}' #{params.inspect}"))

  self.default = params[:default]
  @default_proc ||= params[:default_proc]
end

Class Method Details

.new_hash(params = {}) ⇒ Object

Returns a new in-memory hash. Accepts the same optional params hash as new().



223
224
225
226
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 223

def self.new_hash (params={})

  self.new(:hash, params)
end

.new_tree(params = {}) ⇒ Object

Returns a new in-memory B+ tree. Accepts the same optional params hash as new().



231
232
233
234
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 231

def self.new_tree (params={})

  self.new(:tree, params)
end

.open(name, params = {}) ⇒ Object

Same args as initialize, but can take a block form that will close the db when done. Similar to File.open



204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 204

def self.open (name, params={})

  db = self.new(name, params)

  if block_given?
    yield db
    nil
  else
    db
  end

ensure

  db.close if block_given? && db
end

Instance Method Details

#[]=(k, v) ⇒ Object

No comment



252
253
254
255
256
257
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 252

def []= (k, v)

  k = k.to_s; v = v.to_s

  lib.abs_put(@db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v))
end

#clear ⇒ Object

Removes all the records in the cabinet (use with care)

Returns self (like Ruby’s Hash does).



316
317
318
319
320
321
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 316

def clear

  lib.abs_vanish(@db)

  self
end

#close ⇒ Object

Closes the cabinet (and frees the datastructure allocated for it), returns true in case of success.



333
334
335
336
337
338
339
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 333

def close

  result = lib.abs_close(@db)
  lib.abs_del(@db)

  (result == 1)
end

#compact_copy(target_path) ⇒ Object

Copies the current cabinet to a new file.

Does it by copying each entry afresh to the target file. Spares some space, hence the ‘compact’ label…



355
356
357
358
359
360
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 355

def compact_copy (target_path)

  @other_db = Cabinet.new(target_path)
  self.each { |k, v| @other_db[k] = v }
  @other_db.close
end

#copy(target_path) ⇒ Object

Copies the current cabinet to a new file.

Returns true if it was successful.



345
346
347
348
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 345

def copy (target_path)

  (lib.abs_copy(@db, target_path) == 1)
end

#defrag ⇒ Object

Triggers a defrag run (TC >= 1.4.21 only)

Raises:

  • (NotImplementedError) 


497
498
499
500
501
502
503
504
505
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 497

def defrag

  raise(NotImplementedError.new(
    "method defrag is supported since Tokyo Cabinet 1.4.21. " +
    "your TC version doesn't support it"
  )) unless lib.respond_to?(:tctdbsetdfunit)

  call_misc('defrag', Rufus::Tokyo::List.new)
end

#delete(k) ⇒ Object

Removes a record from the cabinet, returns the value if successful else nil.



296
297
298
299
300
301
302
303
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 296

def delete (k)

  k = k.to_s

  v = self[k]

  (lib.abs_out(@db, k, Rufus::Tokyo.blen(k)) == 1) ? v : nil
end

#delete_keys_with_prefix(prefix) ⇒ Object

Deletes all the entries whose keys begin with the given prefix



424
425
426
427
428
429
430
431
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 424

def delete_keys_with_prefix (prefix)

  call_misc(
    'outlist', lib.abs_fwmkeys(@db, prefix, Rufus::Tokyo.blen(prefix), -1))
      # -1 for no limits

  nil
end

#get4(k) ⇒ Object Also known as: getdup

This is a B+ Tree method only, returns all the values for a given key.



561
562
563
564
565
566
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 561

def get4 (k)

  l = lib.tcbdbget4(as_btree, k, Rufus::Tokyo.blen(k))

  Rufus::Tokyo::List.new(l).release
end

#incr(key, inc = 1) ⇒ Object Also known as: addint, adddouble, add_int, add_double

Increments the value stored under the given key with the given increment (defaults to 1 (integer)).

Accepts an integer or a double value.

Raises:



475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 475

def incr (key, inc=1)

  key = key.to_s

  v = inc.is_a?(Fixnum) ?
    lib.addint(@db, key, Rufus::Tokyo.blen(key), inc) :
    lib.adddouble(@db, key, Rufus::Tokyo.blen(key), inc)

  raise(TokyoError.new(
    "incr failed, there is probably already a string value set " +
    "for the key '#{key}'"
  )) if v == Rufus::Tokyo::INT_MIN || (v.respond_to?(:nan?) && v.nan?)

  v
end

#keys(options = {}) ⇒ Object

Returns an array with all the keys in the databse

With no options given, this method will return all the keys (strings) in a Ruby array.

:prefix --> returns only the keys who match a given string prefix

:limit --> returns a limited number of keys

:native --> returns an instance of Rufus::Tokyo::List instead of
  a Ruby Hash, you have to call #free on that List when done with it !
  Else you're exposing yourself to a memory leak.



383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 383

def keys (options={})

  outlen = nil

  if pre = options[:prefix]

    l = lib.abs_fwmkeys(
      @db, pre, Rufus::Tokyo.blen(pre), options[:limit] || -1)

    l = Rufus::Tokyo::List.new(l)

    options[:native] ? l : l.release

  else

    limit = options[:limit] || -1
    limit = nil if limit < 1

    l = options[:native] ? Rufus::Tokyo::List.new : []

    lib.abs_iterinit(@db)

    outlen = FFI::MemoryPointer.new(:int)

    loop do
      break if limit and l.size >= limit
      out = lib.abs_iternext(@db, outlen)
      break if out.address == 0
      l << out.get_bytes(0, outlen.get_int(0))
    end

    l
  end

ensure

  outlen.free if outlen
end

#ldelete(keys) ⇒ Object

Given a list of keys, deletes all the matching entries (in one sweep).



463
464
465
466
467
468
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 463

def ldelete (keys)

  keys = keys.collect { |k| k.to_s }

  call_misc('outlist', Rufus::Tokyo::List.new(keys))
end

#lget(keys) ⇒ Object Also known as: mget

Given a list of keys, returns a Hash { key => value } of the matching entries (in one sweep).



436
437
438
439
440
441
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 436

def lget (keys)

  keys = keys.collect { |k| k.to_s }

  Hash[*call_misc('getlist', Rufus::Tokyo::List.new(keys))]
end

#lib ⇒ Object

Using the cabinet lib



238
239
240
241
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 238

def lib

  CabinetLib
end

#merge!(hash) ⇒ Object Also known as: lput

Merges the given hash into this Cabinet (or Tyrant) and returns self.



447
448
449
450
451
452
453
454
455
456
457
458
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 447

def merge! (hash)

  call_misc(
    'putlist',
    hash.inject(Rufus::Tokyo::List.new) { |l, (k, v)|
      l << k.to_s
      l << v.to_s
      l
    })

  self
end

#path ⇒ Object

Returns the path to this database.



245
246
247
248
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 245

def path

  @path
end

#putcat(k, v) ⇒ Object

Appends the given string at the end of the current string value for key k. If there is no record for key k, a new record will be created.

Returns true if successful.



275
276
277
278
279
280
281
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 275

def putcat (k, v)

  k = k.to_s; v = v.to_s

  (lib.abs_putcat(
    @db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v)) == 1)
end

#putdup(k, v) ⇒ Object

This is a B+ Tree method only, puts a value for a key who has

potentially

multiple values.



552
553
554
555
556
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 552

def putdup (k, v)

  lib.tcbdbputdup(
    as_btree, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v))
end

#putkeep(k, v) ⇒ Object

Like #put but doesn’t overwrite the value if already set. Returns true only if there no previous entry for k.



262
263
264
265
266
267
268
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 262

def putkeep (k, v)

  k = k.to_s; v = v.to_s

  (lib.abs_putkeep(
    @db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v)) == 1)
end

#size ⇒ Object

Returns the number of records in the ‘cabinet’



307
308
309
310
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 307

def size

  lib.abs_rnum(@db)
end

#sync ⇒ Object

“synchronize updated contents of an abstract database object with the file and the device”



365
366
367
368
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 365

def sync

  (lib.abs_sync(@db) == 1)
end

#tranabort ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction abort’.



536
537
538
539
540
541
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 536

def tranabort

  #check_transaction_support

  libcall(:tcadbtranabort)
end

#tranbegin ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction begin’.



512
513
514
515
516
517
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 512

def tranbegin

  #check_transaction_support

  libcall(:tcadbtranbegin)
end

#trancommit ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction commit’.



524
525
526
527
528
529
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 524

def trancommit

  #check_transaction_support

  libcall(:tcadbtrancommit)
end

#weight ⇒ Object

Returns the ‘weight’ of the db (in bytes)



325
326
327
328
 
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 325

def weight

  lib.abs_size(@db)
end