Class: TwitterCldr::Tokenizers::CompositeToken

Inherits:
Object
  • Object
show all
Defined in:
lib/twitter_cldr/tokenizers/composite_token.rb

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(tokens) ⇒ CompositeToken

Returns a new instance of CompositeToken.



12
13
14
# File 'lib/twitter_cldr/tokenizers/composite_token.rb', line 12

def initialize(tokens)
  @tokens = tokens
end

Instance Attribute Details

#tokensObject

Returns the value of attribute tokens.



10
11
12
# File 'lib/twitter_cldr/tokenizers/composite_token.rb', line 10

def tokens
  @tokens
end

Instance Method Details

#to_sObject



20
21
22
# File 'lib/twitter_cldr/tokenizers/composite_token.rb', line 20

def to_s
  @tokens.map { |t| t.to_s }.join('')
end

#typeObject



16
17
18
# File 'lib/twitter_cldr/tokenizers/composite_token.rb', line 16

def type
  :composite
end