Class: Taxonifi::Splitter::Lexer

Inherits:
Object
  • Object
show all
Defined in:
lib/taxonifi/splitter/lexer.rb

Overview

Lexer taken verbatim from OboParser and other mjy gems.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input, token_list = nil) ⇒ Lexer

Returns a new instance of Lexer.

Raises:



6
7
8
9
10
11
12
13
14
 
# File 'lib/taxonifi/splitter/lexer.rb', line 6

def initialize(input, token_list = nil)

  raise Taxonifi::Splitter::SplitterError, "Invalid token list passed to Lexer." if (!token_list.nil? && !Taxonifi::Splitter::TOKEN_LISTS.include?(token_list)  )
  token_list = :global_token_list if token_list.nil?

  @input = input
  @token_list = token_list 
  @next_token = nil
end

Instance Attribute Details

#input ⇒ Object (readonly)

Returns the value of attribute input.



5
6
7
 
# File 'lib/taxonifi/splitter/lexer.rb', line 5

def input
  @input
end

#token_list ⇒ Object (readonly)

Returns the value of attribute token_list.



5
6
7
 
# File 'lib/taxonifi/splitter/lexer.rb', line 5

def token_list
  @token_list
end

Instance Method Details

#peek(token_class, token_list = nil) ⇒ Object

Checks whether the next token is of the specified class.



17
18
19
20
 
# File 'lib/taxonifi/splitter/lexer.rb', line 17

def peek(token_class, token_list = nil)
  token = read_next_token(token_class)
  return token.class == token_class
end

#pop(token_class) ⇒ Object

Return (and delete) the next token from the input stream, or raise an exception if the next token is not of the given class.



24
25
26
27
28
29
30
31
32
 
# File 'lib/taxonifi/splitter/lexer.rb', line 24

def pop(token_class)
  token = read_next_token(token_class)
  @next_token = nil
  if token.class != token_class
    raise(Taxonifi::Splitter::SplitterError, "expected #{token_class.to_s} but received #{token.class.to_s} at #{@input[0..10]}...", caller)
  else
    return token
  end
end