Class: Ferret::Analysis::WhiteSpaceTokenizer

Inherits:

RegExpTokenizer

• Object
• TokenStream
• Tokenizer
• RegExpTokenizer
• Ferret::Analysis::WhiteSpaceTokenizer

Defined in:

lib/ferret/analysis/tokenizers.rb

Overview

A WhiteSpaceTokenizer is a tokenizer that divides text at whiteSpace. Adjacent sequences of non-WhiteSpace characters form tokens.

Method Summary

Methods inherited from RegExpTokenizer

#close, #initialize, #next

Methods inherited from Tokenizer

#close

Methods inherited from TokenStream

#close, #each, #next

Constructor Details

This class inherits a constructor from Ferret::Analysis::RegExpTokenizer