Class: Tokenizer::Tokenizer Deprecated

Inherits:
WhitespaceTokenizer show all
Defined in:
lib/tokenizer/tokenizer.rb

Overview

Deprecated.

Use WhitespaceTokenizer instead.

Constant Summary

Constants inherited from WhitespaceTokenizer

WhitespaceTokenizer::FS, WhitespaceTokenizer::PAIR_POST, WhitespaceTokenizer::PAIR_PRE, WhitespaceTokenizer::PRE_N_POST, WhitespaceTokenizer::SIMPLE_POST, WhitespaceTokenizer::SIMPLE_PRE

Instance Method Summary collapse

Methods inherited from WhitespaceTokenizer

#sanitize_input, #tokenize

Constructor Details

#initialize(*args) ⇒ Tokenizer

Returns a new instance of Tokenizer.


76
77
78
79
# File 'lib/tokenizer/tokenizer.rb', line 76

def initialize(*args)
  warn '[Deprecated!] Use WhitespaceTokenizer instead.'
  super(*args)
end