Class: Yaml::Converter::Parser

Inherits:
Object
  • Object
show all
Defined in:
lib/yaml/converter/parser.rb

Overview

Tokenizes input YAML lines (with inline annotations) into structured tokens consumable by the StateMachine.

Input assumptions:

  • Comment titles: lines starting with # become title tokens.
  • Validation marker: a comment line starting with # YAML validation: is recognized.
  • Separator lines (---) are recognized and currently ignored by the state machine.
  • Inline notes: fragments after #note: are captured as out-of-band NOTE tokens.
  • Other non-empty lines are treated as YAML content.

Constant Summary collapse

Token =

Lightweight token structure used by the parser/state machine pipeline.

Class.new do
  attr_accessor :type, :text, :meta

  def initialize(type:, text:, meta: nil)
    @type = type
    @text = text
    @meta = meta
  end
end
VALIDATION_PREFIX =

Comment line prefix indicating a validation status line will be injected

"# YAML validation:"
NOTE_MARK =

Inline note marker captured from right side of a line

"#note:"

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Parser 



39
40
41
 
# File 'lib/yaml/converter/parser.rb', line 39

def initialize(options = {})
  @options = options
end

Instance Attribute Details

#meta ⇒ Hash? 



23
24
25
26
27
28
29
30
31
 
# File 'lib/yaml/converter/parser.rb', line 23

Token = Class.new do
  attr_accessor :type, :text, :meta

  def initialize(type:, text:, meta: nil)
    @type = type
    @text = text
    @meta = meta
  end
end

#text ⇒ String 



23
24
25
26
27
28
29
30
31
 
# File 'lib/yaml/converter/parser.rb', line 23

Token = Class.new do
  attr_accessor :type, :text, :meta

  def initialize(type:, text:, meta: nil)
    @type = type
    @text = text
    @meta = meta
  end
end

#type ⇒ Symbol 



23
24
25
26
27
28
29
30
31
 
# File 'lib/yaml/converter/parser.rb', line 23

Token = Class.new do
  attr_accessor :type, :text, :meta

  def initialize(type:, text:, meta: nil)
    @type = type
    @text = text
    @meta = meta
  end
end

Instance Method Details

#tokenize(lines) ⇒ Array<Token>

Convert raw lines into token objects.



47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/yaml/converter/parser.rb', line 47

def tokenize(lines)
  tokens = []
  lines.each do |raw|
    line = raw.rstrip
    if line.empty?
      tokens << Token.new(type: :blank, text: "")
      next
    end

    if line.start_with?("# ") || line == "#"
      if line.start_with?(VALIDATION_PREFIX)
        tokens << Token.new(type: :validation, text: line[2..])
      else
        content = (line == "#") ? "" : line[2..]
        tokens << Token.new(type: :title, text: content)
      end
      next
    end

    if line == "---"
      tokens << Token.new(type: :separator, text: line)
      next
    end

    if line.start_with?("-")
      tokens << Token.new(type: :dash_heading, text: line[1..].strip)
    end

    note_idx = line.index(NOTE_MARK)
    if note_idx
      base = line[0...note_idx].rstrip
      note = line[(note_idx + NOTE_MARK.length)..].to_s.strip
      tokens << Token.new(type: :yaml_line, text: base)
      tokens << Token.new(type: :note, text: note)
    else
      tokens << Token.new(type: :yaml_line, text: line)
    end
  end
  tokens
end