Class: Parser

Inherits:
Recognizer show all
Defined in:
lib/antlr4/Parser.rb

Overview

self is all the parsing support code essentially; most of it is error recovery stuff.#

Direct Known Subclasses

ParserInterpreter

Constant Summary collapse

@@bypassAltsAtnCache =

self field maps from the serialized ATN string to the deserialized ATN with bypass alternatives.

See Also:

  • ATNDeserializationOptions#isGenerateRuleBypassTransitions()
Hash.new

Instance Attribute Summary collapse

Attributes inherited from Recognizer

#interp, #listeners, #ruleIndexMapCache, #state, #tokenTypeMapCache

Instance Method Summary collapse

Methods inherited from Recognizer

#addErrorListener, #checkVersion, #extractVersion, #getErrorHeader, #getErrorListenerDispatch, #getRuleIndexMap, #getState, #getTokenErrorDisplay, #getTokenType, #getTokenTypeMap, #sempred

Constructor Details

#initialize(input) ⇒ Parser

Returns a new instance of Parser.



14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
 
# File 'lib/antlr4/Parser.rb', line 14

def initialize(input)
    super()
    # The input stream.
    self.input = nil
    # The error handling strategy for the parser. The default value is a new
    # instance of {@link DefaultErrorStrategy}.
    self.errHandler = DefaultErrorStrategy.new()
    @precedenceStack = Array.new()
    @precedenceStack.push(0)
    # The {@link ParserRuleContext} object for the currently executing rule.
    # self is always non-null during the parsing process.
    @ctx = nil
    # Specifies whether or not the parser should construct a parse tree during
    # the parsing process. The default value is {@code true}.
    @buildParseTrees = true
    # When {@link #setTrace}{@code (true)} is called, a reference to the
    # {@link TraceListener} is stored here so it can be easily removed in a
    # later call to {@link #setTrace}{@code (false)}. The listener itself is
    # implemented as a parser listener so self field is not directly used by
    # other parser methods.
    @tracer = nil
    # The list of {@link ParseTreeListener} listeners registered to receive
    # events during the parse.
    @parseListeners = Array.new()
    # The number of syntax errors reported during parsing. self value is
    # incremented each time {@link #notifyErrorListeners} is called.
    @syntaxErrors = 0
    self.setInputStream(input)
end

Instance Attribute Details

#buildParseTrees ⇒ Object

Returns the value of attribute buildParseTrees.



11
12
13
 
# File 'lib/antlr4/Parser.rb', line 11

def buildParseTrees
  @buildParseTrees
end

#ctx ⇒ Object

Returns the value of attribute ctx.



11
12
13
 
# File 'lib/antlr4/Parser.rb', line 11

def ctx
  @ctx
end

#errHandler ⇒ Object

Returns the value of attribute errHandler.



11
12
13
 
# File 'lib/antlr4/Parser.rb', line 11

def errHandler
  @errHandler
end

#input ⇒ Object

Returns the value of attribute input.



11
12
13
 
# File 'lib/antlr4/Parser.rb', line 11

def input
  @input
end

#parseListeners ⇒ Object

Returns the value of attribute parseListeners.



12
13
14
 
# File 'lib/antlr4/Parser.rb', line 12

def parseListeners
  @parseListeners
end

#precedenceStack ⇒ Object

Returns the value of attribute precedenceStack.



11
12
13
 
# File 'lib/antlr4/Parser.rb', line 11

def precedenceStack
  @precedenceStack
end

#syntaxErrors ⇒ Object

Returns the value of attribute syntaxErrors.



12
13
14
 
# File 'lib/antlr4/Parser.rb', line 12

def syntaxErrors
  @syntaxErrors
end

#tokenNames ⇒ Object

Returns the value of attribute tokenNames.



13
14
15
 
# File 'lib/antlr4/Parser.rb', line 13

def tokenNames
  @tokenNames
end

#tracer ⇒ Object

Returns the value of attribute tracer.



12
13
14
 
# File 'lib/antlr4/Parser.rb', line 12

def tracer
  @tracer
end

Instance Method Details

#addContextToParseTree ⇒ Object 



325
326
327
328
329
330
 
# File 'lib/antlr4/Parser.rb', line 325

def addContextToParseTree()
    # add current context to parent if we have a parent
    if self.ctx.parentCtx then
        self.ctx.parentCtx.addChild(self.ctx)
    end
end

#addParseListener(listener) ⇒ Object

Registers listener to receive events during the parsing process.

<p>To support output-preserving grammar transformations (including but not limited to left-recursion removal, automated left-factoring, and optimized code generation), calls to listener methods during the parse may differ substantially from calls made by ParseTreeWalker#DEFAULT used after the parse is complete. In particular, rule entry and exit events may occur in a different order during the parse than after the parser. In addition, calls to certain rule entry methods may be omitted.</p>

<p>With the following specific exceptions, calls to listener events are deterministic, i.e. for identical input the calls to listener methods will be the same.</p>

<ul> <li>Alterations to the grammar used to generate code may change the behavior of the listener calls.</li> <li>Alterations to the command line options passed to ANTLR 4 when generating the parser may change the behavior of the listener calls.</li> <li>Changing the version of the ANTLR Tool used to generate the parser may change the behavior of the listener calls.</li> </ul>

Parameters:

  • listener —

    the listener to add

Raises:

  • (ReferenceError) 


153
154
155
156
157
 
# File 'lib/antlr4/Parser.rb', line 153

def addParseListener(listener)
    raise ReferenceError.new("listener is nil") if listener.nil? 
#        @parseListeners = Array.new if @parseListeners.nil? 
    self.parseListeners.push(listener)
end

#compileParseTreePattern(pattern, patternRuleIndex, lexer = nil) ⇒ Object

The preferred method of getting a tree pattern. For example, here’s a sample use:

<pre> ParseTree t = parser.expr(); ParseTreePattern p = parser.compileParseTreePattern(“&lt;ID&gt;+0”, MyParser.RULE_expr); ParseTreeMatch m = p.match(t); String id = m.get(“ID”); </pre>



238
239
240
241
242
243
244
245
246
247
248
249
250
 
# File 'lib/antlr4/Parser.rb', line 238

def compileParseTreePattern(pattern, patternRuleIndex, lexer=nil)
    if lexer.nil?  then
        if not self.getTokenStream().nil? then
            tokenSource = self.getTokenStream().getTokenSource()
        end
        lexer = tokenSource if tokenSource.kind_of? Lexer 
    end
    if lexer.nil? 
        raise UnsupportedOperationException.new("Parser can't discover a lexer to use")
    end
    m = ParseTreePatternMatcher.new(lexer, self)
    return m.compile(pattern, patternRuleIndex)
end

#consume ⇒ Object

Consume and return the #getCurrentToken current symbol.

<p>E.g., given the following input with A being the current lookahead symbol, self function moves the cursor to B and returns A.</p>

<pre>

A B
^

</pre>

If the parser is not in error recovery mode, the consumed symbol is added to the parse tree using ParserRuleContext#addChild(Token), and ParseTreeListener#visitTerminal is called on any parse listeners. If the parser is in error recovery mode, the consumed symbol is added to the parse tree using ParserRuleContext#addErrorNode(Token), and ParseTreeListener#visitErrorNode is called on any parse listeners.



308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
 
# File 'lib/antlr4/Parser.rb', line 308

def consume()
    o = self.getCurrentToken()
    if o.type != Token::EOF then
        self.getInputStream().consume()
    end
    hasListener = ! self.parseListeners.empty? 
    if self.buildParseTrees or hasListener then
        if self.errHandler.inErrorRecoveryMode(self) then
            node = self.ctx.addErrorNode(o)
        else
            node = self.ctx.addTokenNode(o)
        end
        self.parseListeners.each {|listener| listener.visitTerminal(node) }
    end
    return o
end

#dumpDFA ⇒ Object

For debugging and other purposes.#



534
535
536
537
538
539
540
541
542
 
# File 'lib/antlr4/Parser.rb', line 534

def dumpDFA()
    seenOne = false
    self.interp.decisionToDFA.each {|dfa| 
        if dfa.states.length > 0
            puts "Decision #{dfa.decision}:"
            puts dfa.toString(self.tokenNames)
        end
   }
end

#enterOuterAlt(localctx, altNum) ⇒ Object 



350
351
352
353
354
355
356
357
358
359
360
 
# File 'lib/antlr4/Parser.rb', line 350

def enterOuterAlt(localctx, altNum)
    # if we have new localctx, make sure we replace existing ctx
    # that is previous child of parse tree
    if self.buildParseTrees and self.ctx != localctx
        if not self.ctx.parentCtx.nil? then
            self.ctx.parentCtx.removeLastChild()
            self.ctx.parentCtx.addChild(localctx)
        end
    end
    self.ctx = localctx
end

#enterRecursionRule(localctx, state, ruleIndex, precedence) ⇒ Object 



375
376
377
378
379
380
381
 
# File 'lib/antlr4/Parser.rb', line 375

def enterRecursionRule(localctx, state, ruleIndex, precedence)
    self.state = state
    self.precedenceStack.push(precedence)
    self.ctx = localctx
    self.ctx.start = self.input.LT(1)
    self.triggerEnterRuleEvent() # simulates rule entry for left-recursive rules
end

#enterRule(localctx, state, ruleIndex) ⇒ Object

Always called by generated parsers upon entry to a rule. Access field #ctx get the current context.



334
335
336
337
338
339
340
 
# File 'lib/antlr4/Parser.rb', line 334

def enterRule(localctx, state, ruleIndex)
    self.state = state
    self.ctx = localctx
    self.ctx.start = self.input.LT(1)
    self.addContextToParseTree() if self.buildParseTrees
    self.triggerEnterRuleEvent()
end

#exitRule ⇒ Object 



342
343
344
345
346
347
348
 
# File 'lib/antlr4/Parser.rb', line 342

def exitRule()
    self.ctx.stop = self.input.LT(-1)
    # trigger event on ctx, before it reverts to parent
    self.triggerExitRuleEvent()
    self.state = self.ctx.invokingState
    self.ctx = self.ctx.parentCtx
end

#getATNWithBypassAlts ⇒ Object

The ATN with bypass alternatives is expensive to create so we create it lazily.

implement the #getSerializedATN() method.



213
214
215
216
217
218
219
220
221
222
223
224
225
226
 
# File 'lib/antlr4/Parser.rb', line 213

def getATNWithBypassAlts()
    serializedAtn = self.getSerializedATN()
    if serializedAtn.nil? 
        raise UnsupportedOperationException.new("The current parser does not support an ATN with bypass alternatives.")
    end
    result = self.bypassAltsAtnCache.get(serializedAtn)
    if result.nil? then
        deserializationOptions = ATNDeserializationOptions.new()
        deserializationOptions.generateRuleBypassTransitions = true
        result = ATNDeserializer(deserializationOptions).deserialize(serializedAtn)
        self.bypassAltsAtnCache[serializedAtn] = result
    end
    return result
end

#getCurrentToken ⇒ Object

Match needs to return the current input symbol, which gets put

into the label for the associated token ref; e.g., x=ID.



273
274
275
 
# File 'lib/antlr4/Parser.rb', line 273

def getCurrentToken()
    return self.input.LT(1)
end

#getDFAStrings ⇒ Object

For debugging and other purposes.#



530
531
532
 
# File 'lib/antlr4/Parser.rb', line 530

def getDFAStrings
    self.interp.decisionToDFA.map {|dfa| dfa.to_s  }
end

#getExpectedTokens ⇒ Object

Computes the set of input symbols which could follow the current parser state and context, as given by #getState and #getContext, respectively.

See Also:

  • RuleContext)


487
488
489
 
# File 'lib/antlr4/Parser.rb', line 487

def getExpectedTokens()
    return self.interp.atn.getExpectedTokens(self.state, self.ctx)
end

#getExpectedTokensWithinCurrentRule ⇒ Object 



491
492
493
494
495
 
# File 'lib/antlr4/Parser.rb', line 491

def getExpectedTokensWithinCurrentRule()
    atn = self.interp.atn
    s = atn.states[self.state]
    return atn.nextTokens(s)
end

#getInputStream ⇒ Object 



252
253
254
 
# File 'lib/antlr4/Parser.rb', line 252

def getInputStream()
    return self.getTokenStream()
end

#getInvokingContext(ruleIndex) ⇒ Object 



419
420
421
422
423
424
425
426
427
428
 
# File 'lib/antlr4/Parser.rb', line 419

def getInvokingContext(ruleIndex)
    ctx = self.ctx
    while not ctx.nil? do
        if ctx.ruleIndex == ruleIndex
            return ctx
        end
        ctx = ctx.parentCtx
    end
    return nil
end

#getParseListeners ⇒ Object 



121
122
123
 
# File 'lib/antlr4/Parser.rb', line 121

def getParseListeners
    self.parseListeners
end

#getPrecedence ⇒ Object

Get the precedence level for the top-most precedence rule.

the parser context is not nested within a precedence rule.

Returns:

  • The precedence level for the top-most precedence rule, or -1 if



367
368
369
370
371
372
373
 
# File 'lib/antlr4/Parser.rb', line 367

def getPrecedence()
    if @precedenceStack.length==0
        return -1
    else
        return @precedenceStack[-1]
    end
end

#getRuleIndex(ruleName) ⇒ Object

Get a rule’s index (i.e., RULE_ruleName field) or -1 if not found.#



498
499
500
501
502
503
504
505
 
# File 'lib/antlr4/Parser.rb', line 498

def getRuleIndex(ruleName)
    ruleIndex = self.getRuleIndexMap().get(ruleName)
    if ruleIndex then
        return ruleIndex
    else
        return -1
    end
end

#getRuleInvocationStack(p = nil) ⇒ Object

Return List&lt;String&gt; of the rule names in your parser instance

leading up to a call to the current rule.  You could override if
you want more details such as the file/line info of where
in the ATN a rule is invoked.

this is very useful for error messages.



514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
 
# File 'lib/antlr4/Parser.rb', line 514

def getRuleInvocationStack(p=nil)
    p = self.ctx if p.nil? 
    stack = Array.new
    while p do
        # compute what follows who invoked us
        ruleIndex = p.getRuleIndex()
        if ruleIndex<0
            stack.push("n/a")
        else
            stack.push(self.ruleNames[ruleIndex])
        end
        p = p.parentCtx
    end
    return stack
end

#getSourceName ⇒ Object 



543
544
545
 
# File 'lib/antlr4/Parser.rb', line 543

def getSourceName
    return self.input.sourceName
end

#getTokenFactory ⇒ Object 



198
199
200
 
# File 'lib/antlr4/Parser.rb', line 198

def getTokenFactory
    return self.input.tokenSource.factory
end

#getTokenStream ⇒ Object 



260
261
262
 
# File 'lib/antlr4/Parser.rb', line 260

def getTokenStream()
    return self.input
end

#inContext(context) ⇒ Object 



435
436
437
438
 
# File 'lib/antlr4/Parser.rb', line 435

def inContext(context)
    # TODO: useful in parser?
    return false
end

#isExpectedToken(symbol) ⇒ @code true

Checks whether or not symbol can follow the current state in the ATN. The behavior of self method is equivalent to the following, but is implemented such that the complete context-sensitive follow set does not need to be explicitly constructed.

<pre> return getExpectedTokens().contains(symbol); </pre>

the ATN, otherwise false.

Parameters:

  • symbol —

    the symbol type to check

Returns:

  • (@code true) —

    if symbol can follow the current state in



454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
 
# File 'lib/antlr4/Parser.rb', line 454

def isExpectedToken(symbol)
    atn = self.interp.atn
    ctx = self.ctx
    s = atn.states[self.state]
    following = atn.nextTokens(s)
#        print "\nisExpectedToken: #{following.toString(tokenNames)}: #{s}"
    if following.member?(symbol) then
#            puts " true "
        return true
    end
    if not following.member? Token::EPSILON then
#            puts " FAIL "
        return false
    end
    while ctx and ctx.invokingState >= 0 and following.member?(Token::EPSILON) do
        invokingState = atn.states[ctx.invokingState]
        rt = invokingState.transitions[0]
        following = atn.nextTokens(rt.followState)
        return true if following.member?(symbol)
        ctx = ctx.parentCtx
    end
    if following.member?( Token::EPSILON) and symbol == Token::EOF
        return true
    else
        return false
    end
end

#match(ttype) ⇒ Object

Match current input symbol against ttype. If the symbol type matches, ANTLRErrorStrategy#reportMatch and #consume are called to complete the match process.

<p>If the symbol type does not match, ANTLRErrorStrategy#recoverInline is called on the current error strategy to attempt recovery. If #getBuildParseTree is true and the token index of the symbol returned by ANTLRErrorStrategy#recoverInline is -1, the symbol is added to the parse tree by calling ParserRuleContext#addErrorNode.</p>

ttype and the error strategy could not recover from the mismatched symbol

Parameters:

  • ttype —

    the token type to match

Returns:

  • the matched symbol



73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
 
# File 'lib/antlr4/Parser.rb', line 73

def match(ttype)
    t = self.getCurrentToken()
    if t.type==ttype
        self.errHandler.reportMatch(self)
        self.consume()
    else
        t = self.errHandler.recoverInline(self)
        if self.buildParseTrees and t.tokenIndex==-1
            # we must have conjured up a new token during single token insertion
            # if it's not the current symbol
            self.ctx.addErrorNode(t)
        end
    end
    return t
end

#matchWildcard ⇒ Object

Match current input symbol as a wildcard. If the symbol type matches (i.e. has a value greater than 0), ANTLRErrorStrategy#reportMatch and #consume are called to complete the match process.

<p>If the symbol type does not match, ANTLRErrorStrategy#recoverInline is called on the current error strategy to attempt recovery. If #getBuildParseTree is true and the token index of the symbol returned by ANTLRErrorStrategy#recoverInline is -1, the symbol is added to the parse tree by calling ParserRuleContext#addErrorNode.</p>

a wildcard and the error strategy could not recover from the mismatched symbol

Returns:

  • the matched symbol



105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
 
# File 'lib/antlr4/Parser.rb', line 105

def matchWildcard()
    t = self.getCurrentToken()
    if t.type > 0 then
        self.errHandler.reportMatch(self)
        self.consume()
    else
        t = self.errHandler.recoverInline(self)
        if self.buildParseTrees and t.tokenIndex == -1 then
            # we must have conjured up a new token during single token insertion
            # if it's not the current symbol
            self.ctx.addErrorNode(t)
        end
    end
    return t
end

#notifyErrorListeners(msg, offendingToken = nil, e = nil) ⇒ Object

RecognitionException



277
278
279
280
281
282
283
284
285
286
 
# File 'lib/antlr4/Parser.rb', line 277

def notifyErrorListeners(msg, offendingToken=nil ,e=nil) #RecognitionException 
    if offendingToken.nil?
        offendingToken = self.getCurrentToken()
    end
    @syntaxErrors = @syntaxErrors + 1
    line = offendingToken.line
    column = offendingToken.column
    listener = self.getErrorListenerDispatch()
    listener.syntaxError(self, offendingToken, line, column, msg, e)
end

#precpred(localctx, precedence) ⇒ Object 



431
432
433
 
# File 'lib/antlr4/Parser.rb', line 431

def precpred(localctx, precedence)
    return precedence >= self.precedenceStack[-1]
end

#pushNewRecursionContext(localctx, state, ruleIndex) ⇒ Object

Like #enterRule but for recursive rules.



386
387
388
389
390
391
392
393
394
395
396
 
# File 'lib/antlr4/Parser.rb', line 386

def pushNewRecursionContext(localctx, state, ruleIndex)
    previous = self.ctx
    previous.parentCtx = localctx
    previous.invokingState = state
    previous.stop = self.input.LT(-1)

    self.ctx = localctx
    self.ctx.start = previous.start
    self.ctx.addChild(previous) if self.buildParseTrees
    self.triggerEnterRuleEvent() # simulates rule entry for left-recursive rules
end

#removeParseListener(listener) ⇒ Object

Remove listener from the list of parse listeners.

<p>If listener is null or has not been added as a parse listener, self method does nothing.</p>

Parameters:

  • listener —

    the listener to remove



166
167
168
169
 
# File 'lib/antlr4/Parser.rb', line 166

def removeParseListener(listener)
#        return if self.parseListeners.nil? 
    self.parseListeners.delete(listener)
end

#removeParseListeners ⇒ Object

Remove all parse listeners.



172
173
174
 
# File 'lib/antlr4/Parser.rb', line 172

def removeParseListeners
    @parseListeners = Array.new
end

#reset ⇒ Object

reset the parser’s state#



45
46
47
48
49
50
51
52
53
54
 
# File 'lib/antlr4/Parser.rb', line 45

def reset()
    self.input.seek(0) unless @input.nil?
    @errHandler.reset(self)
    @ctx = nil
    @syntaxErrors = 0
    self.setTrace(false)
    @precedenceStack = Array.new
    @precedenceStack.push(0)
    @interp.reset() unless @interp.nil?
end

#setInputStream(input) ⇒ Object 



256
257
258
 
# File 'lib/antlr4/Parser.rb', line 256

def setInputStream(input)
    self.setTokenStream(input)
end

#setTokenFactory(factory) ⇒ Object

Tell our token source and error strategy about a new way to create tokens.#



203
204
205
 
# File 'lib/antlr4/Parser.rb', line 203

def setTokenFactory(factory)
    self.input.tokenSource.factory = factory
end

#setTokenStream(input) ⇒ Object

Set the token stream and reset the parser.#



265
266
267
268
269
 
# File 'lib/antlr4/Parser.rb', line 265

def setTokenStream(input)
    self.input = nil
    self.reset()
    self.input = input
end

#setTrace(trace) ⇒ Object

During a parse is sometimes useful to listen in on the rule entry and exit

events as well as token matches. self is for quick and dirty debugging.



550
551
552
553
554
555
556
557
558
559
560
561
 
# File 'lib/antlr4/Parser.rb', line 550

def setTrace(trace)
    if not trace then
        self.removeParseListener(self.tracer)
        self.tracer = nil
    else
        if self.tracer 
            self.removeParseListener(self.tracer)
        end
        self.tracer = TraceListener.new(self)
        self.addParseListener(self.tracer)
    end
end

#triggerEnterRuleEvent ⇒ Object

Notify any parse listeners of an enter rule event.



177
178
179
180
181
182
 
# File 'lib/antlr4/Parser.rb', line 177

def triggerEnterRuleEvent
    self.parseListeners do |listener|
         listener.enterEveryRule(self.ctx)
         self.ctx.enterRule(listener)
    end
end

#triggerExitRuleEvent ⇒ Object

Notify any parse listeners of an exit rule event.

See Also:



189
190
191
192
193
194
195
 
# File 'lib/antlr4/Parser.rb', line 189

def triggerExitRuleEvent
    # reverse order walk of listeners
    self.parseListeners.reverse do |listener|
        self.ctx.exitRule(listener)
        listener.exitEveryRule(self.ctx)
    end
end

#unrollRecursionContexts(parentCtx) ⇒ Object 



398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
 
# File 'lib/antlr4/Parser.rb', line 398

def unrollRecursionContexts(parentCtx)
    self.precedenceStack.pop()
    self.ctx.stop = self.input.LT(-1)
    retCtx = self.ctx # save current ctx (return value)
    # unroll so ctx is as it was before call to recursive method
    if not self.parseListeners.empty? then
        while self.ctx != parentCtx do
            self.triggerExitRuleEvent()
            self.ctx = self.ctx.parentCtx
        end
    else
        self.ctx = parentCtx
    end
    # hook into tree
    retCtx.parentCtx = parentCtx

    if self.buildParseTrees and parentCtx then
        # add return ctx into invoking rule's tree
        parentCtx.addChild(retCtx)
    end
end