Class: Rufus::Tokyo::Table

Inherits:
Object
  • Object
show all
Extended by:
Openable
Includes:
CabinetConfig, HashMethods, Transactions
Defined in:
lib/rufus/tokyo/cabinet/table.rb

Overview

A ‘table’ a table database.

http://alpha.mixi.co.jp/blog/?p=290
http://tokyocabinet.sourceforge.net/spex-en.html#tctdbapi

A short example :

require 'rubygems'
require 'rufus/tokyo/cabinet/table'

t = Rufus::Tokyo::Table.new('table.tdb', :create, :write)
  # '.tdb' suffix is a must

t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
t['pk1'] = { 'name' => 'bob', 'age' => '18' }
t['pk2'] = { 'name' => 'charly', 'age' => '45' }
t['pk3'] = { 'name' => 'doug', 'age' => '77' }
t['pk4'] = { 'name' => 'ephrem', 'age' => '32' }

p t.query { |q|
  q.add_condition 'age', :numge, '32'
  q.order_by 'age'
  q.limit 2
}
  # => [ {"name"=>"ephrem", :pk=>"pk4", "age"=>"32"},
  #      {"name"=>"charly", :pk=>"pk2", "age"=>"45"} ]

t.close

Direct Known Subclasses

TyrantTable

Constant Summary collapse

INDEX_TYPES = 
{
  :lexical => 0,
  :decimal => 1,
  :token => 2,
  :qgram => 3,
  :opt => 9998,
  :optimized => 9998,
  :void => 9999,
  :remove => 9999,
  :keep => 1 << 24
}

Instance Attribute Summary

Attributes included from HashMethods

#default_proc

Instance Method Summary collapse

Methods included from Openable

open

Methods included from Transactions

#abort, #transaction

Methods included from HashMethods

#[], #default, #default=, #each, #merge, #merge!, #to_a, #to_h, #values

Constructor Details

#initialize(path, params = {}) ⇒ Table

Creates a Table instance (creates or opens it depending on the args)

For example,

t = Rufus::Tokyo::Table.new('table.tdb')
  # '.tdb' suffix is a must

will create the table.tdb (or simply open it if already present) and make sure we have write access to it.

parameters

Parameters can be set in the path or via the optional params hash (like in Rufus::Tokyo::Cabinet)

* :mode    a set of chars ('r'ead, 'w'rite, 'c'reate, 't'runcate,
           'e' non locking, 'f' non blocking lock), default is 'wc'
* :opts    a set of chars ('l'arge, 'd'eflate, 'b'zip2, 't'cbs)
           (usually empty or something like 'ld' or 'lb')

* :bnum    number of elements of the bucket array
* :apow    size of record alignment by power of 2 (defaults to 4)
* :fpow    maximum number of elements of the free block pool by
           power of 2 (defaults to 10)
* :mutex   when set to true, makes sure only 1 thread at a time
           accesses the table (well, Ruby, global thread lock, ...)

* :rcnum   specifies the maximum number of records to be cached.
           If it is not more than 0, the record cache is disabled.
           It is disabled by default.
* :lcnum   specifies the maximum number of leaf nodes to be cached.
           If it is not more than 0, the default value is specified.
           The default value is 2048.
* :ncnum   specifies the maximum number of non-leaf nodes to be
           cached. If it is not more than 0, the default value is
           specified. The default value is 512.

* :xmsiz   specifies the size of the extra mapped memory. If it is
           not more than 0, the extra mapped memory is disabled.
           The default size is 67108864.

* :dfunit  unit step number. If it is not more than 0,
           the auto defragmentation is disabled. (Since TC 1.4.21)

Some examples :

t = Rufus::Tokyo::Table.new('table.tdb')
t = Rufus::Tokyo::Table.new('table.tdb#mode=r')
t = Rufus::Tokyo::Table.new('table.tdb', :mode => 'r')
t = Rufus::Tokyo::Table.new('table.tdb#opts=ld#mode=r')
t = Rufus::Tokyo::Table.new('table.tdb', :opts => 'ld', :mode => 'r')



127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 127

def initialize (path, params={})

  conf = determine_conf(path, params, :table)

  @db = lib.tctdbnew

  #
  # tune table

  libcall(:tctdbsetmutex) if conf[:mutex]

  libcall(:tctdbtune, conf[:bnum], conf[:apow], conf[:fpow], conf[:opts])

  # TODO : set indexes here... well, there is already #set_index
  #conf[:indexes]...

  libcall(:tctdbsetcache, conf[:rcnum], conf[:lcnum], conf[:ncnum])

  libcall(:tctdbsetxmsiz, conf[:xmsiz])

  libcall(:tctdbsetdfunit, conf[:dfunit]) \
    if lib.respond_to?(:tctdbsetdfunit) # TC >= 1.4.21

  #
  # open table

  @path = conf[:path]

  libcall(:tctdbopen, @path, conf[:mode])

  #
  # no default

  @default_proc = nil
end

Instance Method Details

#[]=(pk, h_or_a) ⇒ Object

Inserts a record in the table db

table['pk0'] = [ 'name', 'fred', 'age', '45' ]
table['pk1'] = { 'name' => 'jeff', 'age' => '46' }

Accepts both a hash or an array (expects the array to be of the form [ key, value, key, value, … ] else it will raise an ArgumentError)

Raises an error in case of failure.



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 247

def []= (pk, h_or_a)

  pk = pk.to_s
  h_or_a = Rufus::Tokyo.h_or_a_to_s(h_or_a)

  m = Rufus::Tokyo::Map[h_or_a]

  r = lib.tab_put(@db, pk, Rufus::Tokyo.blen(pk), m.pointer)

  m.free

  r || raise_error # raising potential error after freeing map

  h_or_a
end

#clear ⇒ Object

Removes all records in this table database



281
282
283
284
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 281

def clear

  libcall(:tab_vanish)
end

#close ⇒ Object

Closes the table (and frees the datastructure allocated for it), returns true in case of success.



180
181
182
183
184
185
186
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 180

def close

  result = lib.tab_close(@db)
  lib.tab_del(@db)

  result
end

#delete(k) ⇒ Object

Removes an entry in the table

(might raise an error if the delete itself failed, but returns nil if there was no entry for the given key)



268
269
270
271
272
273
274
275
276
277
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 268

def delete (k)

  k = k.to_s

  v = self[k]
  return nil unless v
  libcall(:tab_out, k, Rufus::Tokyo.blen(k))

  v
end

#delete_keys_with_prefix(prefix) ⇒ Object

Deletes all the entries whose key begin with the given prefix.



313
314
315
316
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 313

def delete_keys_with_prefix (prefix)

  query_delete { |q| q.add('', :strbw, prefix) }
end

#difference(*queries) ⇒ Object

Returns the difference of the listed queries

r = table.intersection(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query OR the second but not both.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



502
503
504
505
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 502

def difference (*queries)

  search(:difference, *queries)
end

#do_query(&block) ⇒ Object

Prepares and runs a query, returns a ResultSet instance (takes care of freeing the query structure)



354
355
356
357
358
359
360
361
362
363
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 354

def do_query (&block)

  q = prepare_query(&block)
  rs = q.run

  return rs

ensure
  q && q.free
end

#generate_unique_id ⇒ Object Also known as: genuid

Generates a unique id (in the context of this Table instance)



190
191
192
193
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 190

def generate_unique_id

  lib.tab_genuid(@db)
end

#intersection(*queries) ⇒ Object

Returns the intersection of the listed queries

r = table.intersection(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query AND the second.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



480
481
482
483
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 480

def intersection (*queries)

  search(:intersection, *queries)
end

#keys(options = {}) ⇒ Object

Returns an array of all the primary keys in the table

With no options given, this method will return all the keys (strings) in a Ruby array.

:prefix --> returns only the keys who match a given string prefix

:limit --> returns a limited number of keys

:native --> returns an instance of Rufus::Tokyo::List instead of
  a Ruby Hash, you have to call #free on that List when done with it !
  Else you're exposing yourself to a memory leak.



299
300
301
302
303
304
305
306
307
308
309
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 299

def keys (options={})

  pre = options.fetch(:prefix, "")

  l = lib.tab_fwmkeys(
    @db, pre, Rufus::Tokyo.blen(pre), options[:limit] || -1)

  l = Rufus::Tokyo::List.new(l)

  options[:native] ? l : l.release
end

#lget(*keys) ⇒ Object Also known as: mget

No ‘misc’ methods for the table library, so this lget is equivalent to calling get for each key. Hoping later versions of TC will provide a mget method.



322
323
324
325
326
327
328
329
330
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 322

def lget (*keys)

  keys.flatten.inject({}) { |h, k|
    k = k.to_s
    v = self[k]
    h[k] = v if v
    h
  }
end

#lib ⇒ Object

Using the cabinet lib



165
166
167
168
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 165

def lib

  CabinetLib
end

#path ⇒ Object

Returns the path to the table.



172
173
174
175
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 172

def path

  @path
end

#pointer ⇒ Object

Returns the actual pointer to the Tokyo Cabinet table



436
437
438
439
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 436

def pointer

  @db
end

#prepare_query(&block) ⇒ Object

Prepares a query instance (block is optional)



343
344
345
346
347
348
349
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 343

def prepare_query (&block)

  q = TableQuery.new(self)
  block.call(q) if block

  q
end

#query(&block) ⇒ Object

Prepares and runs a query, returns an array of hashes (all Ruby) (takes care of freeing the query and the result set structures)



368
369
370
371
372
373
374
375
376
377
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 368

def query (&block)

  rs = do_query(&block)
  a = rs.to_a

  return a

ensure
  rs && rs.free
end

#query_count(&block) ⇒ Object

Prepares a query and then runs it and deletes all the results.



394
395
396
397
398
399
400
401
402
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 394

def query_count (&block)

  q = prepare_query { |q|
    q.pk_only  # improve efficiency, since we have to do the query
  }
  q.count
ensure
  q.free if q
end

#query_delete(&block) ⇒ Object

Prepares a query and then runs it and deletes all the results.



381
382
383
384
385
386
387
388
389
390
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 381

def query_delete (&block)

  q = prepare_query(&block)
  rs = q.delete

  return rs

ensure
  q && q.free
end

#search(type, *queries) ⇒ Object

A #search a la ruby-tokyotyrant (github.com/actsasflinn/ruby-tokyotyrant/tree)

r = table.search(
  :intersection,
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

Accepts the symbols :union, :intersection, :difference or :diff as first parameter.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.

Raises:

  • (ArgumentError) 


526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 526

def search (type, *queries)

  run_query = true
  run_query = queries.pop if queries.last == false

  raise(
    ArgumentError.new("pass at least one prepared query")
  ) if queries.size < 1

  raise(
    ArgumentError.new("pass instances of Rufus::Tokyo::TableQuery only")
  ) if queries.find { |q| q.class != TableQuery }

  t = META_TYPES[type]

  raise(
    ArgumentError.new("no search type #{type.inspect}")
  ) unless t

  qs = FFI::MemoryPointer.new(:pointer, queries.size)
  qs.write_array_of_pointer(queries.collect { |q| q.pointer })

  r = lib.tab_metasearch(qs, queries.size, t)

  qs.free

  pks = Rufus::Tokyo::List.new(r).release

  run_query ? lget(pks) : pks
end

#set_index(column_name, *types) ⇒ Object

Sets an index on a column of the table.

Types maybe be :lexical or :decimal.

Recently (TC 1.4.26 and 1.4.27) inverted indexes have been added, they are :token and :qgram. There is an :opt index as well.

Sorry couldn’t find any good doc about those inverted indexes apart from :

http://alpha.mixi.co.jp/blog/?p=1147
http://www.excite-webtl.jp/world/english/web/?wb_url=http%3A%2F%2Falpha.mixi.co.jp%2Fblog%2F%3Fp%3D1147&wb_lp=JAEN&wb_dis=2&wb_submit=+%96%7C+%96%F3+

Use :keep to “add” and :remove (or :void) to “remove” an index.

If column_name is :pk or “”, the index will be set on the primary key.

Returns true in case of success.



227
228
229
230
231
232
233
234
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 227

def set_index (column_name, *types)

  column_name = column_name == :pk ? '' : column_name.to_s

  ii = types.inject(0) { |i, t| i = i | INDEX_TYPES[t]; i }

  lib.tab_setindex(@db, column_name, ii)
end

#size ⇒ Object

Returns the number of records in this table db



336
337
338
339
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 336

def size

  lib.tab_rnum(@db)
end

#tranabort ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction abort’.



429
430
431
432
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 429

def tranabort

  libcall(:tctdbtranabort)
end

#tranbegin ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction begin’.



409
410
411
412
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 409

def tranbegin

  libcall(:tctdbtranbegin)
end

#trancommit ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction commit’.



419
420
421
422
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 419

def trancommit

  libcall(:tctdbtrancommit)
end

#union(*queries) ⇒ Object

Returns the union of the listed queries

r = table.union(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query OR the second.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



458
459
460
461
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 458

def union (*queries)

  search(:union, *queries)
end