Class: EBNF::LL1::Lexer::Token

Inherits:
Object
  • Object
show all
Defined in:
lib/ebnf/ll1/lexer.rb

Overview

Represents a lexer token.

Examples:

Creating a new token

token = EBNF::LL1::Lexer::Token.new(:LANGTAG, "en")
token.type   #=> :LANGTAG
token.value  #=> "en"

See Also:

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(type, value, options = {}) ⇒ Token

Initializes a new token instance.

Parameters:

  • type (Symbol)
  • value (String)
  • options (Hash{Symbol => Object}) (defaults to: {})

Options Hash (options):

  • :lineno (Integer) — default: nil 


397
398
399
400
401
402
 
# File 'lib/ebnf/ll1/lexer.rb', line 397

def initialize(type, value, options = {})
  @type = type.to_s.to_sym if type
  @value = value.to_s
  @options = options.dup
  @lineno  = @options.delete(:lineno)
end

Instance Attribute Details

#lineno ⇒ Integer (readonly)

The line number where the token was encountered.

Returns:

  • (Integer) 


382
383
384
 
# File 'lib/ebnf/ll1/lexer.rb', line 382

def lineno
  @lineno
end

#options ⇒ Hash (readonly)

Any additional options for the token.

Returns:

  • (Hash) 


388
389
390
 
# File 'lib/ebnf/ll1/lexer.rb', line 388

def options
  @options
end

#type ⇒ Symbol (readonly)

The token’s symbol type.

Returns:

  • (Symbol) 


370
371
372
 
# File 'lib/ebnf/ll1/lexer.rb', line 370

def type
  @type
end

#value ⇒ String (readonly)

The token’s value.

Returns:

  • (String) 


376
377
378
 
# File 'lib/ebnf/ll1/lexer.rb', line 376

def value
  @value
end

Instance Method Details

#===(value) ⇒ Boolean

Returns ‘true` if the given `value` matches either the type or value of this token.

Examples:

Matching using the symbolic type

EBNF::LL1::Lexer::Token.new(:NIL) === :NIL     #=> true

Matching using the string value

EBNF::LL1::Lexer::Token.new(nil, "{") === "{"  #=> true

Parameters:

  • value (Symbol, String)

Returns:

  • (Boolean) 


430
431
432
433
434
435
436
437
438
 
# File 'lib/ebnf/ll1/lexer.rb', line 430

def ===(value)
  case value
    when Symbol
      value == @type
    when ::String
      @value == (@options[:case_insensitive] ? value.to_s.downcase : value.to_s)
    else value == @value
  end
end

#[](key) ⇒ Object

Returns the attribute named by ‘key`.

Parameters:

  • key (Symbol)

Returns:

  • (Object) 


409
410
411
412
413
414
415
416
 
# File 'lib/ebnf/ll1/lexer.rb', line 409

def [](key)
  key = key.to_s.to_sym unless key.is_a?(Integer) || key.is_a?(Symbol)
  case key
    when 0, :type    then @type
    when 1, :value   then @value
    else nil
  end
end

#inspect ⇒ String

Returns a developer-friendly representation of this token.

Returns:

  • (String) 


472
473
474
 
# File 'lib/ebnf/ll1/lexer.rb', line 472

def inspect
  "#{@value.inspect}#{'(' + @type.to_s + ')' if @type}"
end

#representation ⇒ Object

Returns type, if not nil, otherwise value



456
457
458
 
# File 'lib/ebnf/ll1/lexer.rb', line 456

def representation
  @type ? @type : @value
end

#to_a ⇒ Array

Returns an array representation of this token.

Returns:

  • (Array) 


464
465
466
 
# File 'lib/ebnf/ll1/lexer.rb', line 464

def to_a
  [@type, @value]
end

#to_hash ⇒ Hash

Returns a hash table representation of this token.

Returns:

  • (Hash) 


444
445
446
 
# File 'lib/ebnf/ll1/lexer.rb', line 444

def to_hash
  {:type => @type, :value => @value}
end

#to_s ⇒ Object

Readable version of token



450
451
452
 
# File 'lib/ebnf/ll1/lexer.rb', line 450

def to_s
  @type ? @type.inspect : @value
end