Module: RDocF95::TokenStream

Included in:
AnyMethod, RubyParser
Defined in:
lib/rdoc-f95/tokenstream.rb

Overview

A TokenStream is a list of tokens, gathered during the parse of some entity (say a method). Entities populate these streams by being registered with the lexer. Any class can collect tokens by including TokenStream. From the outside, you use such an object by calling the start_collecting_tokens method, followed by calls to add_token and pop_token.

Instance Method Summary collapse

Instance Method Details

#add_token(tk) ⇒ Object 



20
21
22
 
# File 'lib/rdoc-f95/tokenstream.rb', line 20

def add_token(tk)
  @token_stream << tk
end

#add_tokens(tks) ⇒ Object 



24
25
26
 
# File 'lib/rdoc-f95/tokenstream.rb', line 24

def add_tokens(tks)
  tks.each  {|tk| add_token(tk)}
end

#pop_token ⇒ Object 



28
29
30
 
# File 'lib/rdoc-f95/tokenstream.rb', line 28

def pop_token
  @token_stream.pop
end

#start_collecting_tokens ⇒ Object 



16
17
18
 
# File 'lib/rdoc-f95/tokenstream.rb', line 16

def start_collecting_tokens
  @token_stream = []
end

#token_stream ⇒ Object 



12
13
14
 
# File 'lib/rdoc-f95/tokenstream.rb', line 12

def token_stream
  @token_stream
end