Class: OpenNlp::Tokenizer

Inherits:

Tool

• Object
• Tool
• OpenNlp::Tokenizer

Defined in:

lib/open_nlp/tokenizer.rb

Instance Attribute Summary

Attributes inherited from Tool

#j_instance

Instance Method Summary

• #tokenize(str) ⇒ Array

  Tokenizes a string.

Methods inherited from Tool

#initialize

Methods included from JavaClass

included

Constructor Details

This class inherits a constructor from OpenNlp::Tool

Instance Method Details

#tokenize(str) ⇒ Array

Tokenizes a string

Parameters:

• str (String) —

  string to tokenize

Returns:

• (Array) —

  array of string tokens

# File 'lib/open_nlp/tokenizer.rb', line 9

def tokenize(str)
  fail ArgumentError, 'str must be a String' unless str.is_a?(String)
  j_instance.tokenize(str).to_ary
end