Class: Ferret::Analysis::StandardAnalyzer

Inherits:

Object

Object
Ferret::Analysis::StandardAnalyzer

show all

Defined in:: ext/isomorfeus_ferret_ext/frb_analysis.c

Overview

Summary

The StandardAnalyzer is the most advanced of the available analyzers. If it were implemented in Ruby it would look like this;

class StandardAnalyzer
def initialize(stop_words = FRT_FULL_ENGLISH_STOP_WORDS, lower = true)
  @lower = lower
  @stop_words = stop_words
end

def token_stream(field, str)
  ts = StandardTokenizer.new(str)
  ts = LowerCaseFilter.new(ts) if @lower
  ts = StopFilter.new(ts, @stop_words)
  ts = HyphenFilter.new(ts)
end
end

As you can see it makes use of the StandardTokenizer and you can also add your own list of stopwords if you wish.