Class: Token::Resolver::Document

Inherits:
Object
  • Object
show all
Defined in:
lib/token/resolver/document.rb

Overview

Parses input text and provides access to the resulting text and token nodes.

Document is the primary public API for parsing. It uses Grammar to parse the input and Transform to convert the parse tree into node objects.

Examples:

Basic usage

doc = Token::Resolver::Document.new("Hello {KJ|NAME}!")
doc.nodes       # => [Text("Hello "), Token(["KJ", "NAME"]), Text("!")]
doc.tokens      # => [Token(["KJ", "NAME"])]
doc.token_keys  # => ["KJ|NAME"]
doc.to_s        # => "Hello {KJ|NAME}!"
doc.text_only?  # => false

Fast-path for text without tokens

doc = Token::Resolver::Document.new("No tokens here")
doc.text_only?  # => true

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input, config: Config.default) ⇒ Document

Parse input text into a Document.

Parameters:

  • input (String) —

    Text to parse for tokens

  • config (Config) (defaults to: Config.default) —

    Token configuration (default: Config.default)



33
34
35
36
37
 
# File 'lib/token/resolver/document.rb', line 33

def initialize(input, config: Config.default)
  @config = config
  @input = input
  @nodes = parse(input)
end

Instance Attribute Details

#config ⇒ Config (readonly)

Returns The config used for parsing.

Returns:

  • (Config) —

    The config used for parsing



27
28
29
 
# File 'lib/token/resolver/document.rb', line 27

def config
  @config
end

#nodes ⇒ Array<Node::Text, Node::Token> (readonly)

Returns Parsed nodes.

Returns:



24
25
26
 
# File 'lib/token/resolver/document.rb', line 24

def nodes
  @nodes
end

Instance Method Details

#text_only? ⇒ Boolean

Whether the input contains no tokens.

Returns:

  • (Boolean) 


63
64
65
 
# File 'lib/token/resolver/document.rb', line 63

def text_only?
  tokens.empty?
end

#to_s ⇒ String

Reconstruct the original input from nodes (roundtrip fidelity).

Returns:

  • (String) 


56
57
58
 
# File 'lib/token/resolver/document.rb', line 56

def to_s
  @nodes.map(&:to_s).join
end

#token_keys ⇒ Array<String>

Return the unique token keys found in the input.

Returns:

  • (Array<String>) 


49
50
51
 
# File 'lib/token/resolver/document.rb', line 49

def token_keys
  @token_keys ||= tokens.map(&:key).uniq
end

#tokens ⇒ Array<Node::Token>

Return only the Token nodes.

Returns:



42
43
44
 
# File 'lib/token/resolver/document.rb', line 42

def tokens
  @tokens ||= @nodes.select(&:token?)
end