Parse

The framework for building parsers.

Features

• Position tracking. You can always obtain the current position and the current rule's start position while parsing.
• Position remapping. Ever wanted to implement something like C preprocessor's #line command? Now it is possible!
• Automatic error handling. If the parsing fails, the raised error contains the failure position and what is expected at that position. No additional coding is required.
• Flexible and easy to use. Parsers are written in plain Ruby. You may implement any parsing algorithms you wish! Also no clumsy DSLs or code generation are used - you may use your favorite IDE!

Install

Command line:

gem install parse-framework

With Bundler:

echo 'gem "parse-framework"' >>Gemfile
bundle install

Usage

Write subclass of Parse class.

Also you may find Parse::MapPosition and Code classes useful!

Examples

First parser:

require 'parse'

class HelloWorldParser < Parse

  def start
    scan("hello world")
  end

end

Usage:

puts HelloWorldParser.new.("hello world")
  #=> hello world

Sequence:

class HelloWorldParser < Parse

  def start
    scan("hello ") and scan("world")
  end

end

puts HelloWorldParser.new.("hello world")
  #=> world

Analyze errors:

begin
  HelloWorldParser.new.("Hello everyone")
rescue Parse::Error => e
  puts "error at #{e.pos.line + 1}:#{e.pos.column + 1}: #{e.message}"
end

#=> error at 1:7: "world" expected

Regexps:

class HelloWorldParser < Parse

  def start
    scan(/[Hh]ello/) and scan(/\s+/) and scan("world")
  end

end

puts HelloWorldParser.new.("Hello world")
  #=> "world"

Alteration:

class HelloWorldParser < Parse

  def start
    scan(/[Hh]ello/) and scan(/\s+/) and (
      _{ scan("world") } or
      _{ scan("everyone") }
    )
  end

end

puts HelloWorldParser.new.("Hello everyone")
  #=> everyone

Separate rules:

class HelloWorldParser < Parse

  def start
    scan(/[Hh]ello/) and scan(/\s+/) and who
  end

  def who
    _{ scan("world") } or
    _{ scan("everyone") }
  end

end

puts HelloWorldParser.new.("Hello everyone")
  #=> everyone

Repetition:

class MillionThankYousParser < Parse

  def start
    many { scan("thank you") and scan(/\s*/) }
  end

end

MillionThankYousParser.new.("thank you thank you thank you")

many is a Parser Combinator. There are many other Parser Combinators, see Parse's doc.

Counting:

class MillionThankYousParser < Parse

  def start
    count = 0
    many {
      scan("thank you") and scan(/\s*/) and
      act { count += 1 }  # `act` always returns true regardless of 
                          # the block's result.
    } and
    count  # the last non-nil value in the sequence is the
           # sequence's result.
  end

end

puts MillionThankYousParser.new.("thank you thank you thank you")
  #=> 3

Capture variables:

class ChatBot < Parse

  def start
    scan("My name is ") and n = scan(/\w+/) and scan(/[\!\.]/) and n
  end

end

puts ChatBot.new.("My name is John!")
  #=> John

Parse LISP expressions (using AST nodes):

class ParseLISP < Parse

  # a shorthand for declaring a Struct with member "children" and a method
  # "to_ruby_value" and including the module "ASTNode" into it.
  Cons = ASTNode.new :children do

    def to_ruby_value
      children.map(&:to_ruby_value)
    end

  end

  Atom = ASTNode.new :val do

    def to_ruby_value
      val
    end

  end

  def start
    skipped and
    r = (_{ cons } or _{ atom }) and
    r.to_ruby_value
  end

  # "rule" is similar to "def" but it allows to use "_(node)" in its body.
  # "_(node)" sets {ASTNode#pos} to the rule's start position.
  rule :atom do
    _{ n = scan(/\d+/) and skipped and act { n = n.to_i } and _(Atom[n]) } or
    _{ s = scan(/"[^"]*"/) and skipped and act { s = s[1...-1] } and _(Atom[s]) } or
    _{ s = scan(/[^\(\)\"\s\;]+/) and skipped and act { s = s.to_sym } and _(Atom[s]) }
  end

  rule :cons do
    scan("(") and skipped and
    a = many { _{ atom } or _{ cons } } and  # "many" results in array of
                                             # successfully parsed values
    scan(")") and skipped and
    _(Cons[a])
  end

  def skipped
    opt {
      _{ scan(/\s+/) } or
      _{ scan(/;.*\n/) }
    }
  end

end

p ParseLISP.new.(' (a b (c d) (e 12 "s")) ')
  #=> [:a, :b, [:c, :d], [:e, 12, "s"]]

The same but using token macro:

class ParseLISP < Parse

  Cons = ASTNode.new :children do

    def to_ruby_value
      children.map(&:to_ruby_value)
    end

  end

  Atom = ASTNode.new :val do

    def to_ruby_value
      val
    end

  end

  def start
    skipped and
    r = (_{ cons } or _{ atom }) and
    r.to_ruby_value
  end

  rule :atom do
    _{ n = number and _(Atom[n]) } or        # !!!
    _{ s = string and _(Atom[s]) } or        # !!!
    _{ s = symbol and _(Atom[s]) }                    # !!!
  end

  rule :cons do
    lbrace and                               # !!!
    a = many { _{ atom } or _{ cons } } and
    rbrace and                               # !!!
    _(Cons[a])
  end

  def skipped
    opt {
      _{ scan(/\s+/) } or
      _{ scan(/;.*\n/) }
    }
  end

  # !!!

  # `token` is like `rule` but:
  # 1) it handles errors slightly different, see doc.
  # 2) it skips `whitespace_and_comments` after the token
  token :number, "integer number" do
    n = scan(/\d+/) and n.to_i
  end

  # you may omit description.
  token :string do
    s = scan(/"[^"]*"/) and s[1...-1]
  end

  token :symbol do
    s = scan(/[^\(\)\"\s\;]+/) and s.to_sym
  end

  # a shorthand for ``token :lbrace, "`('" do scan("(") end''
  token :lbrace, "("

  token :rbrace, ")"

  # required by `token`.
  alias whitespace_and_comments skipped

  # !!!

end

p ParseLISP.new.(' (a b (c d) (e 12 "s")) ')
  #=> [:a, :b, [:c, :d], [:e, 12, "s"]]

There are many other features, see Parse's documentation, it is pretty detailed!

License

The MIT License (MIT)

Copyright (c) 2016 Various Furriness

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.