Class: Dhaka::LexerSpecification

Inherits:
Object
  • Object
show all
Defined in:
lib/lexer/specification.rb

Overview

Abstract base class for lexer specifications.

Use this to specify the transformations that will be performed when the lexer recognizes a given pattern. Actions are listed in descending order of priority. For example in the following lexer specification:

class LexerSpec < Dhaka::LexerSpecification
  for_pattern 'zz' do
    "recognized two zs"
  end

  for_pattern '\w(\w|\d)*' do
    "recognized word token #{current_lexeme.value}"
  end

  for_pattern '(\d)+(\.\d+)?' do
    "recognized number #{current_lexeme.value}"
  end

  for_pattern ' +' do
    #ignores whitespace
  end

  for_pattern "\n+" do
    "recognized newline"
  end
end

the pattern ‘zz’ takes precedence over the pattern immediately below it, so the lexer will announce that it has recognized two ‘z’s instead of a word token.

The patterns are not Ruby regular expressions - a lot of operators featured in Ruby’s regular expression engine are not yet supported. See dhaka.rubyforge.org/regex_grammar.html for the current syntax.

Class Method Summary collapse

Class Method Details

.for_pattern(pattern, &blk) ⇒ Object

Associates blk as the action to be performed when a lexer recognizes pattern. When Lexer#lex is invoked, it creates a LexerRun object that provides the context for blk to be evaluated in. Methods available in this block are LexerRun#current_lexeme and LexerRun#create_token.



40
41
42
43
 
# File 'lib/lexer/specification.rb', line 40

def for_pattern(pattern, &blk)
  items[pattern] = LexerSpecificationItem.new(pattern, priority, blk)
  self.priority += 1
end