Class: Lingo::Language::Token

Inherits:
WordForm
  • Object
show all
Defined in:
lib/lingo/language/token.rb

Overview

– Die Klasse Token, abgeleitet von der Klasse WordForm, stellt den Container für ein einzelnes Wort eines Textes dar. Das Wort wird mit einem Attribut versehen, welches der Regel entspricht, die dieses Wort identifiziert hat.

Steht z.B. in ruby.cfg eine Regel zur Erkennung einer Zahl, die mit NUM bezeichnet wird, so wird dies dem Token angeheftet, z.B. Token.new(‘100’, ‘NUM’) -> #100/NUM# ++

Constant Summary collapse

POSITION_SEP = 
':'

Instance Attribute Summary collapse

Attributes inherited from WordForm

#attr, #form, #gender, #head, #src, #token

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from WordForm

#<=>, #eql?, #hash, #identified?, #inspect, #unknown?

Constructor Details

#initialize(form, attr, position = nil, offset = nil) ⇒ Token

Returns a new instance of Token.



48
49
50
51
 
# File 'lib/lingo/language/token.rb', line 48

def initialize(form, attr, position = nil, offset = nil)
  @position, @offset = position, offset
  super(form, self.class.clean(attr))
end

Instance Attribute Details

#offset ⇒ Object (readonly)

Returns the value of attribute offset.



53
54
55
 
# File 'lib/lingo/language/token.rb', line 53

def offset
  @offset
end

#position ⇒ Object (readonly)

Returns the value of attribute position.



53
54
55
 
# File 'lib/lingo/language/token.rb', line 53

def position
  @position
end

Class Method Details

.clean(attr) ⇒ Object 



44
45
46
 
# File 'lib/lingo/language/token.rb', line 44

def self.clean(attr)
  attr.sub(/:.*/, '')
end

Instance Method Details

#number? ⇒ Boolean

Returns:

  • (Boolean) 


61
62
63
 
# File 'lib/lingo/language/token.rb', line 61

def number?
  attr == TA_NUMBER
end

#position_and_offset ⇒ Object 



65
66
67
 
# File 'lib/lingo/language/token.rb', line 65

def position_and_offset
  "#{position}#{POSITION_SEP}#{offset}"
end

#to_a ⇒ Object 



69
70
71
 
# File 'lib/lingo/language/token.rb', line 69

def to_a
  [form, attr, position, offset]
end

#to_s ⇒ Object 



73
74
75
 
# File 'lib/lingo/language/token.rb', line 73

def to_s
  ":#{super}:"
end

#word? ⇒ Boolean Also known as: word_token?

Returns:

  • (Boolean) 


55
56
57
 
# File 'lib/lingo/language/token.rb', line 55

def word?
  attr == TA_WORD
end