Class: Zvec::Collection

Inherits:
Object
  • Object
show all
Defined in:
lib/zvec/collection.rb

Overview

A vector collection backed by the zvec C++ engine. Provides CRUD operations, vector similarity search, and index management.

Collections must be explicitly closed via #close before they can be reopened from the same path. Use the closed? method to check state.

All mutating operations are thread-safe (protected by a Monitor).

Examples:

Create, populate, search, and close

schema = Zvec::Schema.new("articles") do
  string "title"
  vector "embedding", dimension: 4,
         index: Zvec::Ext::HnswIndexParams.new(Zvec::COSINE)
end

col = Zvec::Collection.create_and_open("/tmp/articles", schema)
col.add(pk: "1", title: "Hello", embedding: [0.1, 0.2, 0.3, 0.4])
results = col.search([0.1, 0.2, 0.3, 0.4], top_k: 5)
col.close

Reopen an existing collection

col = Zvec::Collection.open("/tmp/articles")
puts col.doc_count
col.close

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(ext_collection, schema: nil, name: nil) ⇒ Collection

Returns a new instance of Collection.

Parameters:

  • ext_collection (Ext::Collection) —

    the underlying C++ collection

  • schema (Zvec::Schema, nil) (defaults to: nil) —

    optional schema for type-aware access

  • name (String, nil) (defaults to: nil) —

    optional collection name



36
37
38
39
40
41
42
 
# File 'lib/zvec/collection.rb', line 36

def initialize(ext_collection, schema: nil, name: nil)
  @ext = ext_collection
  @schema = schema
  @name = name
  @monitor = Monitor.new
  @closed = false
end

Instance Attribute Details

#schema ⇒ Zvec::Schema? (readonly)

Returns the schema, if provided at creation time.

Returns:

  • (Zvec::Schema, nil) —

    the schema, if provided at creation time



31
32
33
 
# File 'lib/zvec/collection.rb', line 31

def schema
  @schema
end

Class Method Details

.create_and_open(path, schema, read_only: false, enable_mmap: true) ⇒ Zvec::Collection

Create a new collection on disk and open it.

Examples:

col = Zvec::Collection.create_and_open("/tmp/my_col", schema)

Parameters:

  • path (String) —

    directory path for the collection data

  • schema (Zvec::Schema) —

    the collection schema

  • read_only (Boolean) (defaults to: false) —

    open in read-only mode

  • enable_mmap (Boolean) (defaults to: true) —

    use memory-mapped I/O (default: true)

Returns:

Raises:

  • (ArgumentError) —

    if path is blank or schema is not a Zvec::Schema



55
56
57
58
59
60
61
62
63
64
 
# File 'lib/zvec/collection.rb', line 55

def self.create_and_open(path, schema, read_only: false, enable_mmap: true)
  validate_path!(path)
  raise ArgumentError, "schema must be a Zvec::Schema" unless schema.is_a?(Schema)

  opts = Ext::CollectionOptions.new
  opts.read_only = read_only
  opts.enable_mmap = enable_mmap
  ext = Ext::Collection.create_and_open(path, schema.ext_schema, opts)
  new(ext, schema: schema, name: schema.name)
end

.open(path, read_only: false, enable_mmap: true) ⇒ Zvec::Collection

Open an existing collection from disk.

Examples:

col = Zvec::Collection.open("/tmp/my_col", read_only: true)

Parameters:

  • path (String) —

    directory path of an existing collection

  • read_only (Boolean) (defaults to: false) —

    open in read-only mode

  • enable_mmap (Boolean) (defaults to: true) —

    use memory-mapped I/O (default: true)

Returns:

Raises:

  • (ArgumentError) —

    if path is blank



76
77
78
79
80
81
82
83
84
 
# File 'lib/zvec/collection.rb', line 76

def self.open(path, read_only: false, enable_mmap: true)
  validate_path!(path)

  opts = Ext::CollectionOptions.new
  opts.read_only = read_only
  opts.enable_mmap = enable_mmap
  ext = Ext::Collection.open(path, opts)
  new(ext)
end

.validate_path!(path) ⇒ Object

Raises:

  • (ArgumentError) 


422
423
424
 
# File 'lib/zvec/collection.rb', line 422

def self.validate_path!(path)
  raise ArgumentError, "path must be a non-empty string" if path.nil? || path.to_s.strip.empty?
end

Instance Method Details

#add(pk:, **fields) ⇒ Array

Convenience method to insert a document from keyword arguments.

Examples:

col.add(pk: "1", title: "Hello", embedding: [0.1, 0.2, 0.3, 0.4])

Parameters:

  • pk (String, Integer) —

    the primary key (required)

  • fields (Hash) —

    field name/value pairs

Returns:

  • (Array) —

    write results

Raises:



377
378
379
380
381
382
 
# File 'lib/zvec/collection.rb', line 377

def add(pk:, **fields)
  ensure_open!
  raise ArgumentError, "#{error_prefix}pk must not be nil" if pk.nil?
  doc = Doc.new(pk: pk, fields: fields, schema: @schema)
  insert(doc)
end

#close ⇒ void

This method returns an undefined value.

Close the collection, releasing the underlying C++ resources. The collection must be closed before it can be reopened from the same path.

Examples:

col.close
col.closed?  #=> true

Raises:



125
126
127
128
129
130
131
132
133
134
135
136
 
# File 'lib/zvec/collection.rb', line 125

def close
  raise CollectionError, "#{error_prefix}Collection is already closed" if @closed

  @monitor.synchronize do
    begin
      @ext.close
    rescue NoMethodError
      # C++ extension may not expose a close method; the GC will handle it.
    end
    @closed = true
  end
end

#closed? ⇒ Boolean

Returns true if the collection has been closed.

Returns:

  • (Boolean) —

    true if the collection has been closed



111
112
113
 
# File 'lib/zvec/collection.rb', line 111

def closed?
  @closed
end

#collection_name ⇒ String?

Returns the collection name (from schema or explicit).

Returns:

  • (String, nil) —

    the collection name (from schema or explicit)



87
88
89
 
# File 'lib/zvec/collection.rb', line 87

def collection_name
  @name || (@schema ? @schema.name : nil)
end

#create_index(field_name, index_params) ⇒ self

Create an index on a field.

Examples:

col.create_index("embedding",
  Ext::HnswIndexParams.new(Zvec::COSINE, m: 32, ef_construction: 400))

Parameters:

  • field_name (String, Symbol) —

    the field to index

  • index_params (Ext::HnswIndexParams, Ext::FlatIndexParams, Ext::IVFIndexParams, Ext::InvertIndexParams) —

    index configuration

Returns:

  • (self)

Raises:



152
153
154
155
156
157
158
159
160
 
# File 'lib/zvec/collection.rb', line 152

def create_index(field_name, index_params)
  ensure_open!
  raise ArgumentError, "field_name must be a non-empty string" if field_name.nil? || field_name.to_s.strip.empty?

  @monitor.synchronize do
    @ext.create_index(field_name.to_s, index_params)
  end
  self
end

#delete(*pks) ⇒ Array<Array(Boolean, String)>

Delete documents by primary key(s).

Examples:

col.delete("doc-1", "doc-2")

Parameters:

  • pks (Array<String>) —

    one or more primary keys to delete

Returns:

  • (Array<Array(Boolean, String)>) —

    write results

Raises:



273
274
275
276
277
278
279
280
 
# File 'lib/zvec/collection.rb', line 273

def delete(*pks)
  ensure_open!
  pks = pks.flatten
  raise ArgumentError, "#{error_prefix}No primary keys provided for delete" if pks.empty?
  pks = pks.map(&:to_s)
  results = @monitor.synchronize { @ext.delete_pks(pks) }
  check_write_results!(results)
end

#delete_by_filter(filter) ⇒ void

This method returns an undefined value.

Delete documents matching a filter expression.

Examples:

col.delete_by_filter("year < 2020")

Parameters:

  • filter (String) —

    the filter expression (see VectorQuery for syntax)

Raises:



291
292
293
294
295
 
# File 'lib/zvec/collection.rb', line 291

def delete_by_filter(filter)
  ensure_open!
  raise ArgumentError, "#{error_prefix}filter must be a non-empty string" if filter.nil? || filter.to_s.strip.empty?
  @monitor.synchronize { @ext.delete_by_filter(filter) }
end

#destroy ⇒ void

This method returns an undefined value.

Destroy the collection, removing all data from disk.

Raises:



202
203
204
205
206
207
208
 
# File 'lib/zvec/collection.rb', line 202

def destroy
  ensure_open!
  @monitor.synchronize do
    @ext.destroy
    @closed = true
  end
end

#doc_count ⇒ Integer

Returns the number of documents in the collection.

Returns:

  • (Integer) —

    the number of documents in the collection

Raises:



105
106
107
108
 
# File 'lib/zvec/collection.rb', line 105

def doc_count
  ensure_open!
  @ext.stats.doc_count
end

#drop_index(field_name) ⇒ self

Drop an index on a field.

Parameters:

  • field_name (String, Symbol) —

    the field whose index to drop

Returns:

  • (self)

Raises:



168
169
170
171
172
173
174
175
176
 
# File 'lib/zvec/collection.rb', line 168

def drop_index(field_name)
  ensure_open!
  raise ArgumentError, "field_name must be a non-empty string" if field_name.nil? || field_name.to_s.strip.empty?

  @monitor.synchronize do
    @ext.drop_index(field_name.to_s)
  end
  self
end

#fetch(*pks) ⇒ Hash{String => Zvec::Doc}

Fetch documents by primary key(s).

Examples:

docs = col.fetch("doc-1", "doc-2")
docs["doc-1"]["title"]  #=> "Hello"

Parameters:

  • pks (Array<String>) —

    one or more primary keys

Returns:

  • (Hash{String => Zvec::Doc}) —

    mapping of pk to document

Raises:

  • (Zvec::CollectionError) —

    if the collection is closed

  • (ArgumentError) —

    if no primary keys provided



356
357
358
359
360
361
362
363
364
365
 
# File 'lib/zvec/collection.rb', line 356

def fetch(*pks)
  ensure_open!
  pks = pks.flatten
  raise ArgumentError, "#{error_prefix}No primary keys provided for fetch" if pks.empty?
  pks = pks.map(&:to_s)
  raw = @monitor.synchronize { @ext.fetch(pks) }
  raw.transform_values do |h|
    Doc.new(pk: nil, fields: h, schema: @schema)
  end
end

#flush ⇒ self

Flush pending writes to disk.

Returns:

  • (self)

Raises:



192
193
194
195
196
 
# File 'lib/zvec/collection.rb', line 192

def flush
  ensure_open!
  @monitor.synchronize { @ext.flush }
  self
end

#insert(docs) ⇒ Array<Array(Boolean, String)>

Insert one or more documents.

Examples:

doc = Zvec::Doc.new(pk: "1", schema: schema)
doc["title"] = "Hello"
col.insert(doc)

Parameters:

Returns:

  • (Array<Array(Boolean, String)>) —

    write results

Raises:



224
225
226
227
228
229
230
231
 
# File 'lib/zvec/collection.rb', line 224

def insert(docs)
  ensure_open!
  docs = [docs] unless docs.is_a?(Array)
  validate_docs!(docs)
  ext_docs = docs.map { |d| d.is_a?(Doc) ? d.ext_doc : d }
  results = @monitor.synchronize { @ext.insert(ext_docs) }
  check_write_results!(results)
end

#optimize ⇒ self

Optimize the collection (compact segments, rebuild indexes).

Returns:

  • (self)

Raises:



182
183
184
185
186
 
# File 'lib/zvec/collection.rb', line 182

def optimize
  ensure_open!
  @monitor.synchronize { @ext.optimize }
  self
end

#path ⇒ String

Returns the on-disk path of the collection.

Returns:

  • (String) —

    the on-disk path of the collection



92
93
94
 
# File 'lib/zvec/collection.rb', line 92

def path
  @ext.path
end

#query(field_name:, vector:, topk: 10, filter: nil, include_vector: false, output_fields: nil, query_params: nil) ⇒ Array<Zvec::Doc>

Execute a vector similarity search with full control over parameters.

Examples:

results = col.query(
  field_name: "embedding",
  vector: [0.1, 0.2, 0.3, 0.4],
  topk: 5,
  filter: "year > 2024"
)
results.each { |doc| puts "#{doc.pk}: #{doc.score}" }

Parameters:

  • field_name (String, Symbol) —

    the vector field to search

  • vector (Array<Numeric>) —

    the query vector

  • topk (Integer) (defaults to: 10) —

    maximum number of results (default: 10)

  • filter (String, nil) (defaults to: nil) —

    optional filter expression

  • include_vector (Boolean) (defaults to: false) —

    include stored vectors in results

  • output_fields (Array<String>, nil) (defaults to: nil) —

    specific fields to return

  • query_params (Ext::HnswQueryParams, Ext::IVFQueryParams, Ext::FlatQueryParams, nil) (defaults to: nil) —

    search tuning params

Returns:

  • (Array<Zvec::Doc>) —

    result documents with pk and score set

Raises:



322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
 
# File 'lib/zvec/collection.rb', line 322

def query(field_name:, vector:, topk: 10, filter: nil,
          include_vector: false, output_fields: nil, query_params: nil)
  ensure_open!
  validate_query_vector!(vector, field_name)

  vq = VectorQuery.new(
    field_name: field_name,
    vector: vector,
    topk: topk,
    filter: filter,
    include_vector: include_vector,
    output_fields: output_fields,
    query_params: query_params
  )
  raw_results = @monitor.synchronize { @ext.query(vq.ext_query) }
  raw_results.map do |h|
    Doc.new(
      pk: h["pk"],
      fields: h.reject { |k, _| %w[pk score doc_id].include?(k) },
      schema: @schema
    ).tap { |d| d.instance_variable_set(:@score, h["score"]) }
  end
end

#search(vector, field: nil, top_k: 10, filter: nil) ⇒ Array<Zvec::Doc>

Convenience method for simple vector similarity search.

Auto-detects the vector field from the schema if not specified.

Examples:

results = col.search([0.1, 0.2, 0.3, 0.4], top_k: 5)
results.first.pk     #=> "doc-1"
results.first.score  #=> 0.95

Parameters:

  • vector (Array<Numeric>) —

    the query vector

  • field (String, Symbol, nil) (defaults to: nil) —

    vector field name (auto-detected if nil)

  • top_k (Integer) (defaults to: 10) —

    number of results (default: 10)

  • filter (String, nil) (defaults to: nil) —

    optional filter expression

Returns:

Raises:



400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
 
# File 'lib/zvec/collection.rb', line 400

def search(vector, field: nil, top_k: 10, filter: nil)
  ensure_open!
  raise ArgumentError, "#{error_prefix}vector must be a non-empty Array" unless vector.is_a?(Array) && !vector.empty?

  # Auto-detect vector field if not specified
  fname = field&.to_s
  unless fname
    if @schema
      vfield = @schema.ext_schema.vector_fields.first
      raise CollectionError, "#{error_prefix}No vector fields in schema" unless vfield
      fname = vfield.name
    else
      vfields = @ext.schema.vector_fields
      raise CollectionError, "#{error_prefix}No vector fields in schema" if vfields.empty?
      fname = vfields.first.name
    end
  end
  query(field_name: fname, vector: vector, topk: top_k, filter: filter)
end

#stats ⇒ Ext::CollectionStats

Returns collection statistics.

Returns:

  • (Ext::CollectionStats) —

    collection statistics

Raises:



98
99
100
101
 
# File 'lib/zvec/collection.rb', line 98

def stats
  ensure_open!
  @ext.stats
end

#update(docs) ⇒ Array<Array(Boolean, String)>

Update one or more existing documents.

Parameters:

Returns:

  • (Array<Array(Boolean, String)>) —

    write results

Raises:



254
255
256
257
258
259
260
261
 
# File 'lib/zvec/collection.rb', line 254

def update(docs)
  ensure_open!
  docs = [docs] unless docs.is_a?(Array)
  validate_docs!(docs)
  ext_docs = docs.map { |d| d.is_a?(Doc) ? d.ext_doc : d }
  results = @monitor.synchronize { @ext.update(ext_docs) }
  check_write_results!(results)
end

#upsert(docs) ⇒ Array<Array(Boolean, String)>

Upsert (insert or update) one or more documents.

Parameters:

Returns:

  • (Array<Array(Boolean, String)>) —

    write results

Raises:



239
240
241
242
243
244
245
246
 
# File 'lib/zvec/collection.rb', line 239

def upsert(docs)
  ensure_open!
  docs = [docs] unless docs.is_a?(Array)
  validate_docs!(docs)
  ext_docs = docs.map { |d| d.is_a?(Doc) ? d.ext_doc : d }
  results = @monitor.synchronize { @ext.upsert(ext_docs) }
  check_write_results!(results)
end