Class: NLPBackpack::Tokenizer::Tab

Inherits:

Object

Object
NLPBackpack::Tokenizer::Tab

show all

Defined in:: lib/nlp_backpack/tokenizers/tab.rb

Class Method Summary collapse

.tokenize(string) ⇒ Object

Class Method Details

.tokenize(string) ⇒ `Object`



6
7
8

# File 'lib/nlp_backpack/tokenizers/tab.rb', line 6

def tokenize(string)
  string.split(/\t{1}/)
end