Class: SlateDb::Reader

Inherits:
Object
  • Object
show all
Defined in:
lib/slatedb/reader.rb

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.open(path, url: nil, checkpoint_id: nil, follow_latest: nil, manifest_poll_interval: nil, checkpoint_lifetime: nil, max_memtable_bytes: nil, cache_root: nil, max_open_file_handles: nil, merge_operator: nil) {|reader| ... } ⇒ Reader

Open a read-only reader at the given path.

Examples:

Open a reader

reader = SlateDb::Reader.open("/tmp/mydb")
value = reader.get("key")
reader.close

Open with block (auto-close)

SlateDb::Reader.open("/tmp/mydb") do |reader|
  reader.get("key")
end # automatically closed

Open at a specific checkpoint

reader = SlateDb::Reader.open("/tmp/mydb", checkpoint_id: "uuid-here")

Follow the latest state without managing a checkpoint

reader = SlateDb::Reader.open("/tmp/mydb", follow_latest: true)

Enable the on-disk cache and cap its open file handles

reader = SlateDb::Reader.open("/tmp/mydb",
                              cache_root: "/var/cache/slatedb",
                              max_open_file_handles: 256)

Parameters:

  • path (String) —

    The path identifier for the database

  • url (String, nil) (defaults to: nil) —

    Optional object store URL

  • checkpoint_id (String, nil) (defaults to: nil) —

    Optional checkpoint UUID to read at. When given, the reader is pinned to that checkpoint and does not follow new writes.

  • follow_latest (Boolean, nil) (defaults to: nil) —

    When true, the reader tails the latest manifest without creating or maintaining its own checkpoint. This performs no object-store writes but provides no protection from garbage collection, so it is best suited to read-only or mirrored databases. Mutually exclusive with checkpoint_id. When neither is set (the default), the reader manages its own checkpoint and refreshes it periodically. (Requires SlateDB >= 0.15.0)

  • manifest_poll_interval (Integer, nil) (defaults to: nil) —

    Poll interval in milliseconds

  • checkpoint_lifetime (Integer, nil) (defaults to: nil) —

    Checkpoint lifetime in milliseconds

  • max_memtable_bytes (Integer, nil) (defaults to: nil) —

    Maximum memtable size in bytes

  • cache_root (String, nil) (defaults to: nil) —

    Root folder for the reader's local on-disk object-store cache. Setting this enables the cached object store; when it is not set the cache (and max_open_file_handles) has no effect.

  • max_open_file_handles (Integer, nil) (defaults to: nil) —

    Maximum number of file handles to keep open in the reader's file-handle cache. When the limit is reached, the least recently used handle is closed (default: 1000). Only takes effect when cache_root is set. (Requires SlateDB >= 0.13.0)

  • merge_operator (Symbol, String, nil) (defaults to: nil) —

    Optional merge operator ("string_concat" or "concat")

Yields:

  • (reader) —

    If a block is given, yields the reader and ensures it's closed

Returns:

  • (Reader) —

    The opened reader (or block result if block given)



53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/slatedb/reader.rb', line 53

def open(path, url: nil, checkpoint_id: nil, follow_latest: nil,
         manifest_poll_interval: nil, checkpoint_lifetime: nil,
         max_memtable_bytes: nil, cache_root: nil, max_open_file_handles: nil,
         merge_operator: nil)
  opts = {}
  opts[:follow_latest] = follow_latest unless follow_latest.nil?
  opts[:manifest_poll_interval] = manifest_poll_interval if manifest_poll_interval
  opts[:checkpoint_lifetime] = checkpoint_lifetime if checkpoint_lifetime
  opts[:max_memtable_bytes] = max_memtable_bytes if max_memtable_bytes
  opts[:cache_root] = cache_root if cache_root
  opts[:max_open_file_handles] = max_open_file_handles if max_open_file_handles
  opts[:merge_operator] = merge_operator.to_s if merge_operator

  reader = _open(path, url, checkpoint_id, opts)

  if block_given?
    begin
      yield reader
    ensure
      begin
        reader.close
      rescue StandardError
        nil
      end
    end
  else
    reader
  end
end

Instance Method Details

#get(key, durability_filter: nil, dirty: nil, cache_blocks: nil) ⇒ String?

Get a value by key.

Parameters:

  • key (String) —

    The key to look up

  • durability_filter (String, nil) (defaults to: nil) —

    Filter by durability level ("remote" or "memory")

  • dirty (Boolean, nil) (defaults to: nil) —

    Whether to include uncommitted data

  • cache_blocks (Boolean, nil) (defaults to: nil) —

    Whether to cache blocks

Returns:

  • (String, nil) —

    The value, or nil if not found



92
93
94
95
96
97
98
99
100
101
102
103
 
# File 'lib/slatedb/reader.rb', line 92

def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
  opts = {}
  opts[:durability_filter] = durability_filter.to_s if durability_filter
  opts[:dirty] = dirty unless dirty.nil?
  opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?

  if opts.empty?
    _get(key)
  else
    _get_with_options(key, opts)
  end
end

#scan(start_key, end_key = nil, durability_filter: nil, dirty: nil, read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil) ⇒ Iterator

Scan a range of keys.

Parameters:

  • start_key (String) —

    The start key (inclusive)

  • end_key (String, nil) (defaults to: nil) —

    The end key (exclusive)

Returns:

  • (Iterator) —

    An iterator over key-value pairs



111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/slatedb/reader.rb', line 111

def scan(start_key, end_key = nil, durability_filter: nil, dirty: nil,
         read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, &)
  opts = {}
  opts[:durability_filter] = durability_filter.to_s if durability_filter
  opts[:dirty] = dirty unless dirty.nil?
  opts[:read_ahead_bytes] = read_ahead_bytes if read_ahead_bytes
  opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
  opts[:max_fetch_tasks] = max_fetch_tasks if max_fetch_tasks

  iter = if opts.empty?
           _scan(start_key, end_key)
         else
           _scan_with_options(start_key, end_key, opts)
         end

  if block_given?
    iter.each(&)
  else
    iter
  end
end

#scan_prefix(prefix, durability_filter: nil, dirty: nil, read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, from: nil, to: nil) ⇒ Iterator

Scan all keys with a given prefix.

Parameters:

  • prefix (String) —

    The key prefix to scan

  • durability_filter (String, nil) (defaults to: nil) —

    Filter by durability level

  • dirty (Boolean, nil) (defaults to: nil) —

    Whether to include uncommitted data

  • read_ahead_bytes (Integer, nil) (defaults to: nil) —

    Number of bytes to read ahead

  • cache_blocks (Boolean, nil) (defaults to: nil) —

    Whether to cache blocks

  • max_fetch_tasks (Integer, nil) (defaults to: nil) —

    Maximum number of fetch tasks

  • from (String, nil) (defaults to: nil) —

    Inclusive lower bound suffix, appended to the prefix, to start scanning from (e.g. prefix "user:" with from "100" starts at "user:100"). Defaults to the start of the prefix.

  • to (String, nil) (defaults to: nil) —

    Exclusive upper bound suffix, appended to the prefix, to stop scanning at. Defaults to the end of the prefix.

Returns:

  • (Iterator) —

    An iterator over key-value pairs



148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
 
# File 'lib/slatedb/reader.rb', line 148

def scan_prefix(prefix, durability_filter: nil, dirty: nil,
                read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil,
                from: nil, to: nil, &)
  opts = {
    durability_filter: durability_filter&.to_s,
    dirty: dirty,
    read_ahead_bytes: read_ahead_bytes,
    cache_blocks: cache_blocks,
    max_fetch_tasks: max_fetch_tasks,
    subrange_from: from,
    subrange_to: to
  }.compact

  iter = if opts.empty?
           _scan_prefix(prefix)
         else
           _scan_prefix_with_options(prefix, opts)
         end

  if block_given?
    iter.each(&)
  else
    iter
  end
end