Sexpistol is a very fast and easy-to-use library for parsing S-Expressions in Ruby. Sexpistol takes an S-Expression in string form and turns it into a native Ruby data structure made up of nested sets of arrays.
(define test (lambda () ( (print "Hello world!\n") (print 1) (print 9.01) (print 2.0e10) (print (+ 10 12 13)) )))
would be parsed by Sexpistol like so:
[:define, :test, [:lambda, , [ [:print, "Hello world!\n"], [:print, 1], [:print, 9.01], [:print, 2.0e10], [:print, [:+, 10, 12, 13]] ]]]
Sexpistol supports all of the standard datatypes and converts them directly to their Ruby equivalents:
Lists (a b c)
Integers (1 2 3)
Floats (1.0 2.0 42.9 3e6 1.2e2)
Strings (“t"Hello world!"n”)
Symbols (symbol Symbol __symbol____ symbo_l symbol? symbol! + - / ++ a+ e$, etc…)
Sexpistol also supports mapping the Ruby keyword literals (nil, true, false) to their native Ruby types, although this is disabled by default for compatibility. To enable it use `@parser.ruby_keyword_literals = true`, eg:
@parser = .new @parser.parse_string("nil false true") #=> [:nil, :false, :true] @parser.ruby_keyword_literals = true @parser.parse_string("nil false true") #=> [nil, false, true]
Above all Sexpistol strives to be compatible with Scheme-style S-Expressions. This means that Sexpistol supports comma quoting, though quasi-quoting is not yet implemented. Sexpistol can also generate Scheme compatible external representations when the 'scheme_compatability' options is set to true:
@parser = .new @parser.scheme_compatability = true @parser.to_sexp([:test, false, true, nil]) #=> "(test #f #t ())"
For convenience Sexpistol is packaged as a RubyGem, to install it simply enter the following at your command line:
gem install sexpistol
# Create a new parser instance @parser = .new # Parse a string ast = @parser.parse_string("(string (to (parse)))") #=> [:string, [:to, [:parse]]] # Change the representation ast = :is ast = :parsed #=> [:string, [:is, [:parsed]]] # Turn the array structure back into an S-Expression @parser.to_sexp( ast ) #=> "( string ( is ( parsed ) ) )"
The core of Sexpistol was recently re-written using StringScanner and the new version is roughly twice as fast as the older ones.
Parsing throughput on my test machine (2Ghz Core 2 Duo, 4GB RAM, Ruby 1.9) is approximately 1 Megabytes/sec. This is fairly high given that Sexpistol is pure Ruby. Benchmarking Sexpistol against other popular S-Expression parser gems shows that it is roughly 8x faster than the nearest competitor.