Class: Langchain::Chunker::Sentence

Inherits:
Base
  • Object
show all
Defined in:
lib/langchain/chunker/sentence.rb

Overview

This chunker splits text by sentences.

Usage:

Langchain::Chunker::Sentence.new(text).chunks

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(text) ⇒ Langchain::Chunker::Sentence

Parameters:

  • text (String) 


16
17
18
 
# File 'lib/langchain/chunker/sentence.rb', line 16

def initialize(text)
  @text = text
end

Instance Attribute Details

#text ⇒ Object (readonly)

Returns the value of attribute text.



12
13
14
 
# File 'lib/langchain/chunker/sentence.rb', line 12

def text
  @text
end

Instance Method Details

#chunks ⇒ Array<Langchain::Chunk>

Returns:



21
22
23
24
25
26
 
# File 'lib/langchain/chunker/sentence.rb', line 21

def chunks
  ps = PragmaticSegmenter::Segmenter.new(text: text)
  ps.segment.map do |chunk|
    Langchain::Chunk.new(text: chunk)
  end
end