Class: Syntax::Default

Inherits:

Object

• Object
• Syntax::Default

Defined in:

lib/syntax.rb

Overview

A default tokenizer for handling syntaxes that are not explicitly handled elsewhere. It simply yields the given text as a single token.

Instance Method Summary

• #tokenize(text) {|Token.new( text, :normal )| ... } ⇒ Object

  Yield the given text as a single token.

Instance Method Details

#tokenize(text) {|Token.new( text, :normal )| ... } ⇒ Object

Yield the given text as a single token.

Yields:

• (Token.new( text, :normal ))

# File 'lib/syntax.rb', line 10

def tokenize( text )
  yield Token.new( text, :normal )
end