Class: BibTeX::Lexer

Inherits:
Object
  • Object
show all
Extended by:
Forwardable
Defined in:
lib/bibtex/lexer.rb

Overview

The BibTeX::Lexer handles the lexical analysis of BibTeX bibliographies.

Constant Summary collapse

MODE = 
Hash.new(:meta).merge(
  bibtex: :bibtex, entry: :bibtex,
  string: :bibtex, preamble: :bibtex,
  comment: :bibtex,  meta: :meta,
  literal: :literal, content: :content
).freeze

Class Attribute Summary collapse

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Lexer

Creates a new instance. Possible options and their respective default values are:

  • :include => [:errors] A list that may contain :meta_content, and :errors; depending on whether or not these are present, the respective tokens are included in the parse tree.

  • :strict => true In strict mode objects can start anywhere; therefore the ‘@’ symbol is not possible except inside literals or @comment objects; for a more lenient lexer set to false and objects are expected to start after a new line (leading white space is permitted).

  • :strip => true When enabled, newlines will be stripped from quoted string values.



90
91
92
93
 
# File 'lib/bibtex/lexer.rb', line 90

def initialize(options = {})
  @options = Lexer.defaults.merge(options)
  reset
end

Class Attribute Details

.defaults ⇒ Object (readonly)

Returns the value of attribute defaults.



73
74
75
 
# File 'lib/bibtex/lexer.rb', line 73

def defaults
  @defaults
end

.patterns ⇒ Object (readonly)

Returns the value of attribute patterns.



73
74
75
 
# File 'lib/bibtex/lexer.rb', line 73

def patterns
  @patterns
end

Instance Attribute Details

#mode ⇒ Object

Returns the value of attribute mode.



28
29
30
 
# File 'lib/bibtex/lexer.rb', line 28

def mode
  @mode
end

#options ⇒ Object (readonly)

Returns the value of attribute options.



28
29
30
 
# File 'lib/bibtex/lexer.rb', line 28

def options
  @options
end

#scanner ⇒ Object (readonly)

Returns the value of attribute scanner.



28
29
30
 
# File 'lib/bibtex/lexer.rb', line 28

def scanner
  @scanner
end

#stack ⇒ Object (readonly)

Returns the value of attribute stack.



28
29
30
 
# File 'lib/bibtex/lexer.rb', line 28

def stack
  @stack
end

Instance Method Details

#active?(object) ⇒ Boolean

Returns true if the lexer is currently parsing the given object type.

Returns:

  • (Boolean) 


133
134
135
 
# File 'lib/bibtex/lexer.rb', line 133

def active?(object)
  @active_object == object
end

#allow_missing_keys? ⇒ Boolean

Returns:

  • (Boolean) 


142
143
144
 
# File 'lib/bibtex/lexer.rb', line 142

def allow_missing_keys?
  !!@options[:allow_missing_keys]
end

#analyse(string = nil) ⇒ Object

Start the lexical analysis.

Raises:



174
175
176
177
178
179
180
181
182
183
 
# File 'lib/bibtex/lexer.rb', line 174

def analyse(string = nil)
  raise(ArgumentError, 'Lexer: failed to start analysis: no source given!') unless
    string || @scanner

  self.data = string || @scanner.string

  send("parse_#{MODE[@mode]}") until @scanner.eos?

  push([false, '$end'])
end

#bibtex_mode? ⇒ Boolean

Returns true if the lexer is currenty parsing a BibTeX object.

Returns:

  • (Boolean) 


124
125
126
 
# File 'lib/bibtex/lexer.rb', line 124

def bibtex_mode?
  MODE[@mode] == :bibtex
end

#data=(data) ⇒ Object

Sets the source for the lexical analysis and resets the internal state.



109
110
111
112
 
# File 'lib/bibtex/lexer.rb', line 109

def data=(data)
  @scanner = StringScanner.new(data)
  reset
end

#next_token ⇒ Object

Returns the next token from the parse stack.



119
120
121
 
# File 'lib/bibtex/lexer.rb', line 119

def next_token
  @stack.shift
end

#push(value) ⇒ Object

Pushes a value onto the parse stack. Returns the Lexer.



151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
 
# File 'lib/bibtex/lexer.rb', line 151

def push(value)
  case value[0]
  when :CONTENT, :STRING_LITERAL
    value[1].gsub!(/\n\s*/, ' ') if strip_line_breaks?

    if !@stack.empty? && value[0] == @stack[-1][0]
      @stack[-1][1] << value[1]
    else
      @stack.push(value)
    end
  when :ERROR
    @stack.push(value) if @include_errors
    leave_object
  when :META_CONTENT
    @stack.push(value) if @include_meta_content
  else
    @stack.push(value)
  end

  self
end

#reset ⇒ Object 



95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/bibtex/lexer.rb', line 95

def reset
  @stack = []
  @brace_level = 0
  @mode = :meta
  @active_object = nil

  # cache options for speed
  @include_meta_content = @options[:include].include?(:meta_content)
  @include_errors = @options[:include].include?(:errors)

  self
end

#strict? ⇒ Boolean

Returns true if the lexer is currently in strict mode.

Returns:

  • (Boolean) 


138
139
140
 
# File 'lib/bibtex/lexer.rb', line 138

def strict?
  !!@options[:strict]
end

#strip_line_breaks? ⇒ Boolean

Returns:

  • (Boolean) 


146
147
148
 
# File 'lib/bibtex/lexer.rb', line 146

def strip_line_breaks?
  !!options[:strip] && !active?(:comment)
end

#symbols ⇒ Object 



114
115
116
 
# File 'lib/bibtex/lexer.rb', line 114

def symbols
  @stack.map(&:first)
end