Class: Langchain::Chunker::Base

Inherits:

Object

• Object
• Langchain::Chunker::Base

Defined in:

lib/langchain/chunker/base.rb

Overview

Chunkers

Chunkers are used to split documents into smaller chunks before indexing into vector search databases. Otherwise large documents, when retrieved and passed to LLMs, may hit the context window limits.

Available chunkers

• RecursiveText

• Text

• Semantic

• Sentence

Direct Known Subclasses

Markdown, RecursiveText, Semantic, Sentence, Text

Instance Method Summary

• #chunks ⇒ Array<Langchain::Chunk>

Instance Method Details

#chunks ⇒ Array<Langchain::Chunk>

Returns:

• (Array<Langchain::Chunk>)

Raises:

• (NotImplementedError)

# File 'lib/langchain/chunker/base.rb', line 17

def chunks
  raise NotImplementedError
end