Class: EBNF::LL1::Scanner

Inherits:
StringScanner
  • Object
show all
Defined in:
lib/ebnf/ll1/scanner.rb

Overview

Overload StringScanner with file operations

  • Reloads scanner as required until EOF.

  • Loads to a high-water and reloads when remaining size reaches a low-water.

FIXME: Only implements the subset required by the Lexer for now.

Constant Summary collapse

HIGH_WATER = 
10240
LOW_WATER =

Hopefully large enough to deal with long multi-line comments

2048

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input, options = {}) ⇒ Scanner

Create a scanner, from an IO or String

Parameters:

  • input (String, IO, #read)
  • options (Hash{Symbol => Object}) (defaults to: {})
  • options[Integer] (Hash) —

    a customizable set of options



27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/ebnf/ll1/scanner.rb', line 27

def initialize(input, options = {})
  @options = options.merge(:high_water => HIGH_WATER, :low_water => LOW_WATER)

  if input.respond_to?(:read)
    @input = input
    super("")
    feed_me
  else
    super(input.to_s)
  end
end

Instance Attribute Details

#input ⇒ IO, StringIO (readonly)

Returns:

  • (IO, StringIO) 


17
18
19
 
# File 'lib/ebnf/ll1/scanner.rb', line 17

def input
  @input
end

Instance Method Details

#rest ⇒ String

Returns the “rest” of the line, or the next line if at EOL (i.e. everything after the scan pointer). If there is no more data (eos? = true), it returns “”.

Returns:

  • (String) 


44
45
46
47
 
# File 'lib/ebnf/ll1/scanner.rb', line 44

def rest
  feed_me
  encode_utf8 super
end

#scan(pattern) ⇒ String

Tries to match with ‘pattern` at the current position.

If there is a match, the scanner advances the “scan pointer” and returns the matched string. Otherwise, the scanner returns nil.

If the scanner begins with the multi-line start expression

Examples:

s = StringScanner.new('test string')
p s.scan(/\w+/)   # -> "test"
p s.scan(/\w+/)   # -> nil
p s.scan(/\s+/)   # -> " "
p s.scan(/\w+/)   # -> "string"
p s.scan(/./)     # -> nil

Parameters:

  • pattern (Regexp)

Returns:

  • (String) 


78
79
80
81
 
# File 'lib/ebnf/ll1/scanner.rb', line 78

def scan(pattern)
  feed_me
  encode_utf8 super
end

#skip(pattern) ⇒ Object

Attempts to skip over the given ‘pattern` beginning with the scan pointer. If it matches, the scan pointer is advanced to the end of the match, and the length of the match is returned. Otherwise, `nil` is returned.

similar to ‘scan`, but without returning the matched string.

Parameters:

  • pattern (Regexp) 


56
57
58
59
 
# File 'lib/ebnf/ll1/scanner.rb', line 56

def skip(pattern)
  feed_me
  super
end