Class: Crass::Scanner

Inherits:
Object
  • Object
show all
Defined in:
lib/crass/scanner.rb

Overview

Similar to a StringScanner, but with extra functionality needed to tokenize CSS while preserving the original text.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input) ⇒ Scanner

Creates a Scanner instance for the given input string or IO instance.



22
23
24
25
26
27
 
# File 'lib/crass/scanner.rb', line 22

def initialize(input)
  @string = input.is_a?(IO) ? input.read : input.to_s
  @chars  = @string.chars.to_a

  reset
end

Instance Attribute Details

#current ⇒ Object (readonly)

Current character, or ‘nil` if the scanner hasn’t yet consumed a character, or is at the end of the string.



8
9
10
 
# File 'lib/crass/scanner.rb', line 8

def current
  @current
end

#marker ⇒ Object

Current marker position. Use #marked to get the substring between #marker and #pos.



12
13
14
 
# File 'lib/crass/scanner.rb', line 12

def marker
  @marker
end

#pos ⇒ Object

Position of the next character that will be consumed. This is a character position, not a byte position, so it accounts for multi-byte characters.



16
17
18
 
# File 'lib/crass/scanner.rb', line 16

def pos
  @pos
end

#string ⇒ Object (readonly)

The string being scanned.



19
20
21
 
# File 'lib/crass/scanner.rb', line 19

def string
  @string
end

Instance Method Details

#consume ⇒ Object

Consumes the next character and returns it, advancing the pointer, or an empty string if the end of the string has been reached.



31
32
33
34
35
 
# File 'lib/crass/scanner.rb', line 31

def consume
  @current = @chars[@pos] || ''
  @pos += 1 if @current
  @current
end

#consume_rest ⇒ Object

Consumes the rest of the string and returns it, advancing the pointer to the end of the string. Returns an empty string is the end of the string has already been reached.



40
41
42
43
44
45
46
 
# File 'lib/crass/scanner.rb', line 40

def consume_rest
  rest     = @string[@pos..@len] || ''
  @current = rest[-1] || ''
  @pos     = @len

  rest
end

#eos? ⇒ Boolean

Returns ‘true` if the end of the string has been reached, `false` otherwise.

Returns:

  • (Boolean) 


50
51
52
 
# File 'lib/crass/scanner.rb', line 50

def eos?
  @pos == @len
end

#mark ⇒ Object

Sets the marker to the position of the next character that will be consumed.



56
57
58
 
# File 'lib/crass/scanner.rb', line 56

def mark
  @marker = @pos
end

#marked ⇒ Object

Returns the substring between #marker and #pos, without altering the pointer.



62
63
64
65
66
67
68
 
# File 'lib/crass/scanner.rb', line 62

def marked
  if result = @chars[@marker...@pos]
    result.join('')
  else
    ''
  end
end

#peek(length = 1) ⇒ Object

Returns up to length characters starting at the current position, but doesn’t consume them. The number of characters returned may be less than length if the end of the string is reached.



73
74
75
76
77
78
79
 
# File 'lib/crass/scanner.rb', line 73

def peek(length = 1)
  if result = @chars[@pos, length]
    result.join('')
  else
    ''
  end
end

#reconsume ⇒ Object

Moves the pointer back one character without changing the value of #current. The next call to #consume will re-consume the current character.



84
85
86
 
# File 'lib/crass/scanner.rb', line 84

def reconsume
  @pos -= 1 if @pos > 0
end

#reset ⇒ Object

Resets the pointer to the beginning of the string.



89
90
91
92
93
94
 
# File 'lib/crass/scanner.rb', line 89

def reset
  @current = nil
  @len     = @string.length
  @marker  = 0
  @pos     = 0
end

#scan(pattern) ⇒ Object

Tries to match pattern at the current position. If it matches, the matched substring will be returned and the pointer will be advanced. Otherwise, ‘nil` will be returned.



99
100
101
102
103
104
105
106
107
 
# File 'lib/crass/scanner.rb', line 99

def scan(pattern)
  match = pattern.match(@string, @pos)
  return nil if match.nil? || match.begin(0) != @pos

  @pos     = match.end(0)
  @current = @chars[@pos - 1]

  match[0]
end

#scan_until(pattern) ⇒ Object

Scans the string until the pattern is matched. Returns the substring up to and including the end of the match, and advances the pointer. If there is no match, ‘nil` is returned and the pointer is not advanced.



112
113
114
115
116
117
118
119
120
121
122
 
# File 'lib/crass/scanner.rb', line 112

def scan_until(pattern)
  start = @pos
  match = pattern.match(@string, @pos)

  return nil if match.nil?

  @pos     = match.end(0)
  @current = @chars[@pos - 1]

  @string[start...@pos]
end