Class: Tokenizer

Inherits:

Object

Object
Tokenizer

show all

Defined in:: lib/pingilish/tokenizer.rb

Instance Method Summary collapse

#initialize(text) ⇒ Tokenizer constructor

A new instance of Tokenizer.
#tokens ⇒ Object

Constructor Details

#initialize(text) ⇒ `Tokenizer`

Returns a new instance of Tokenizer.



3
4
5

# File 'lib/pingilish/tokenizer.rb', line 3

def initialize(text)
  @text = text    
end

Instance Method Details

#tokens ⇒ `Object`



7
8
9

# File 'lib/pingilish/tokenizer.rb', line 7

def tokens
  @text.split(' ')
end