Class: Syntax::Token

Inherits:

String

• Object
• String
• Syntax::Token

Defined in:

lib/syntax/common.rb

Overview

A single token extracted by a tokenizer. It is simply the lexeme itself, decorated with a ‘group’ attribute to identify the type of the lexeme.

Instance Attribute Summary

• #group ⇒ Object readonly

  the type of the lexeme that was extracted.

• #instruction ⇒ Object readonly

  the instruction associated with this token (:none, :region_open, or :region_close).

Instance Method Summary

• #initialize(text, group, instruction = :none) ⇒ Token constructor

  Create a new Token representing the given text, and belonging to the given group.

Constructor Details

#initialize(text, group, instruction = :none) ⇒ Token

Create a new Token representing the given text, and belonging to the given group.

# File 'lib/syntax/common.rb', line 19

def initialize( text, group, instruction = :none )
  super text
  @group = group
  @instruction = instruction
end

Instance Attribute Details

#group ⇒ Object (readonly)

the type of the lexeme that was extracted.

# File 'lib/syntax/common.rb', line 11

def group
  @group
end

#instruction ⇒ Object (readonly)

the instruction associated with this token (:none, :region_open, or :region_close)

# File 'lib/syntax/common.rb', line 15

def instruction
  @instruction
end