Class: Tokenizer::Tokenizer Deprecated

Inherits:

WhitespaceTokenizer

• Object
• WhitespaceTokenizer
• Tokenizer::Tokenizer

Defined in:

lib/tokenizer/tokenizer.rb

Overview

Deprecated.

Use WhitespaceTokenizer instead.

Constant Summary

Constants inherited from WhitespaceTokenizer

WhitespaceTokenizer::FS, WhitespaceTokenizer::PAIR_POST, WhitespaceTokenizer::PAIR_PRE, WhitespaceTokenizer::PRE_N_POST, WhitespaceTokenizer::SIMPLE_POST, WhitespaceTokenizer::SIMPLE_PRE

Instance Method Summary

• #initialize(*args) ⇒ Tokenizer constructor

  A new instance of Tokenizer.

Methods inherited from WhitespaceTokenizer

#sanitize_input, #tokenize

Constructor Details

#initialize(*args) ⇒ Tokenizer

# File 'lib/tokenizer/tokenizer.rb', line 76

def initialize(*args)
  warn '[Deprecated!] Use WhitespaceTokenizer instead.'
  super(*args)
end