Class: SClust::Util::DocumentTokenizer

Inherits:
TokenizerFilter show all
Defined in:
lib/sclust/util/filters.rb

Overview

A tokenizer that applies a few overall document filters.

Instance Method Summary collapse

Methods inherited from TokenizerFilter

#filter

Methods inherited from Filter

#after, #apply, #before, #filter

Constructor Details

#initializeDocumentTokenizer

Returns a new instance of DocumentTokenizer.



176
177
178
179
# File 'lib/sclust/util/filters.rb', line 176

def initialize()
    super()
    after(HTMLFilter.new())
end