Class: LanguageParser::SQLT_WhitespaceTokenizer

Inherits:
SQLT_State
  • Object
show all
Defined in:
lib/cgialib/lp/SQLTokenizer.rb

Overview

SQLT_WhitespaceTokenizer

Handles whitespace in the character stream

Instance Method Summary collapse

Constructor Details

#initialize(newstate, addtoken) ⇒ SQLT_WhitespaceTokenizer

initialize( newstate, addtoken )

newstate - A method to be called to change state addtoken - The method to be called to add a token



197
198
199
200
201
202
203
204
205
 
# File 'lib/cgialib/lp/SQLTokenizer.rb', line 197

def initialize( newstate, addtoken )
  
  super( newstate, addtoken )
  
  # Initialize the text buffer to blank
  
  @text = ""
  
end

Instance Method Details

#next(ch) ⇒ Object

next( ch )

ch - The character

Handles the character in the parsing stream



213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
 
# File 'lib/cgialib/lp/SQLTokenizer.rb', line 213

def next( ch )
  
  if ( ch =~ /\s/ )
  
    # If the character is whitespace add it to
    # the buffer
  
    @text += ch
    return true
  
  else
  
    # Otherwise return to the normal state and
    # add the token
  
    @addtoken.call( WhitespaceToken.new( @text ) )
    @newstate.call( SQLT_NormalState )
  
    # Return false because we want the tokenizer
    # to re-run on the current character
  
    return false
  
  end
  
end