Class: Textoken::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/textoken/base.rb

Overview

Inits options, findings and responds to tokens Does not raise error when text or options are nil Splits the text and makes it ready for other operations

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(text, opt = nil) ⇒ Base

Returns a new instance of Base.



8
9
10
11
 
# File 'lib/textoken/base.rb', line 8

def initialize(text, opt = nil)
  @text     = initial_split(text)
  @options  = Options.new(opt)
end

Instance Attribute Details

#dont_split ⇒ Object (readonly)

Returns the value of attribute dont_split.



6
7
8
 
# File 'lib/textoken/base.rb', line 6

def dont_split
  @dont_split
end

#findings ⇒ Object (readonly)

Returns the value of attribute findings.



6
7
8
 
# File 'lib/textoken/base.rb', line 6

def findings
  @findings
end

#options ⇒ Object (readonly)

Returns the value of attribute options.



6
7
8
 
# File 'lib/textoken/base.rb', line 6

def options
  @options
end

#text ⇒ Object (readonly)

Returns the value of attribute text.



6
7
8
 
# File 'lib/textoken/base.rb', line 6

def text
  @text
end

Instance Method Details

#tokens ⇒ Object

we do take intersection array of results returning from multiple options



15
16
17
18
19
20
21
22
23
24
25
 
# File 'lib/textoken/base.rb', line 15

def tokens
  options.collection.each do |option|
    if @findings.nil?
      @findings = option.tokenize(self)
    else
      @findings &= option.tokenize(self)
    end
  end

  Tokenizer.new(self).tokens
end

#words ⇒ Object 



27
28
29
30
31
 
# File 'lib/textoken/base.rb', line 27

def words
  # tokenize options but do not make the last split
  @dont_split = true
  tokens
end