Sexpistol is a very fast and easy-to-use library for parsing S-Expressions in Ruby. Sexpistol takes an S-Expression in string form and turns it into a native Ruby data structure made up of nested sets of arrays.
(define test (lambda () ( (print "Hello world!\n") (print 1) (print 9.01) (print 2.0e10) (print (+ 10 12 13)) )))
would be parsed by Sexpistol like so:
[:define, :test, [:lambda, [], [ [:print, "Hello world!\n"], [:print, 1], [:print, 9.01], [:print, 2.0e10], [:print, [:+, 10, 12, 13]] ]]]
Sexpistol supports all of the standard datatypes and converts them directly to their Ruby equivalents:
-
Lists (a b c)
-
Integers (1 2 3)
-
Floats (1.0 2.0 42.9 3e6 1.2e2)
-
Strings (“t"Hello world!"n”)
-
Symbols (symbol Symbol __symbol____ symbo_l symbol? symbol! + - / ++ a+ e$, etc…)
Sexpistol also supports mapping the Ruby keyword literals (nil, true, false) to their native Ruby types, although this is disabled by default for compatibility. To enable it use ‘@parser.ruby_keyword_literals = true`, eg:
@parser = Sexpistol.new @parser.parse_string("nil false true") #=> [:nil, :false, :true] @parser.ruby_keyword_literals = true @parser.parse_string("nil false true") #=> [nil, false, true]
Above all Sexpistol strives to be compatible with Scheme-style S-Expressions. This means that Sexpistol supports comma quoting, though quasi-quoting is not yet implemented. Sexpistol can also generate Scheme compatible external representations when the ‘scheme_compatability’ options is set to true:
@parser = Sexpistol.new @parser.scheme_compatability = true @parser.to_sexp([:test, false, true, nil]) #=> "(test #f #t ())"
For convenience Sexpistol is packaged as a RubyGem, to install it simply enter the following at your command line:
gem install sexpistol
# Create a new parser instance @parser = Sexpistol.new # Parse a string ast = @parser.parse_string("(string (to (parse)))") #=> [:string, [:to, [:parse]]] # Change the representation ast[1][0] = :is ast[1][1][0] = :parsed #=> [:string, [:is, [:parsed]]] # Turn the array structure back into an S-Expression @parser.to_sexp( ast ) #=> "( string ( is ( parsed ) ) )"
The core of Sexpistol was recently re-written using StringScanner and the new version is roughly twice as fast as the older ones.
Parsing throughput on my test machine (2Ghz Core 2 Duo, 4GB RAM, Ruby 1.9) is approximately 1 Megabytes/sec. This is fairly high given that Sexpistol is pure Ruby. Benchmarking Sexpistol against other popular S-Expression parser gems shows that it is roughly 8x faster than the nearest competitor.
- Author
- Contributors
Copyright © 2010 Aaron Gough (thingsaaronmade.com), released under the MIT license