Class: Ferret::Analysis::Tokenizer

Inherits:

TokenStream

• Object
• TokenStream
• Ferret::Analysis::Tokenizer

Defined in:

lib/ferret/analysis/tokenizers.rb

Overview

A Tokenizer is a TokenStream whose input is a Reader.

This is an abstract class.

Direct Known Subclasses

RegExpTokenizer

Instance Method Summary

• #close ⇒ Object

  By default, closes the input Reader.

Methods inherited from TokenStream

#each, #next

Instance Method Details

#close ⇒ Object

By default, closes the input Reader.

# File 'lib/ferret/analysis/tokenizers.rb', line 9

def close()
  @input.close()
end