FastJsonparser
Fastest Json parser for Ruby
Ruby wrapper for simdjson (Parsing gigabytes of JSON per second)
The simdjson library uses commonly available SIMD instructions and microparallel algorithms to parse JSON 2.5 times faster than anything else out there.
Initial code imported from simdjson_ruby
Comparison with Oj gem & JSON.parse
- Parsing JSON from file
FastJsonparser.load(src) is 40% faster than Oj.load_file(src) FastJsonparser.load(src) is 155% faster than JSON.parse(File.read(src))
Benchmark result
# virus-info.json 1.2M
# user system total real
# FastJsonparser 4.772700 0.321748 5.094448 ( 5.100354)
# OJ 6.944966 0.267082 7.212048 ( 7.253477) ~ 42% more
# standard JSON 13.311654 0.167076 13.478730 ( 13.504489) ~ 164% more
Parsing local JSON
FastJsonparser.parse(json) is 19% faster than Oj.load(json) FastJsonparser.parse(json) is 96% faster than JSON.parse(json)
# user system total real # FastJsonparser 6.732325 0.005716 6.738041 ( 6.750285) # OJ 8.029406 0.006582 8.035988 ( 8.055000) ~ 19% more # standard JSON 13.025884 0.031287 13.057171 ( 13.264931) ~ 96% more
Streaming json from file
FastJsonparser.load_many(src) is 153% faster than Yajl::Parser.new.parse(File.new(src, 'r'))
# user system total real # FastJsonparser 3.844446 0.141822 3.986268 ( 3.884655) # YAJL 9.699621 0.110060 9.809681 ( 9.826104) ~ 150% more
Installation
Add this line to your application's Gemfile:
gem 'fast_jsonparser'
And then execute:
$ bundle
Or install it yourself as:
$ gem install fast_jsonparser
Usage
- Parsing JSON from file
FastJsonparser.load(src) # src is source path for json file
Example:
FastJsonparser.load("./benchmark/graduation.json")
- Parsing local JSON
FastJsonparser.parse(json)
Example:
FastJsonparser.parse('{"one": 1, "two": 2}')
- Streaming JSON from file
File with multiple json can be stream with load_many
method
Example: logs.json with following content
{"time": "17/May/2015:08:05:32 +0000", "remote_ip": "93.180.71.3", "remote_user": "-"}
{"time": "17/May/2015:08:05:23 +0000", "remote_ip": "93.180.71.3", "remote_user": "-"}
{"time": "17/May/2015:08:05:24 +0000", "remote_ip": "80.91.33.133", "remote_user": "-"}
load_many
accepts file_path & block as arguments
> FastJsonparser.load_many(file_path) { |obj| p obj[:time]}
"17/May/2015:08:05:32 +0000"
"17/May/2015:08:05:23 +0000"
"17/May/2015:08:05:24 +0000"
- Raise FastJsonparser::ParseError when invalid JSON provided for parsing
FastJsonparser.parse("123: 1") # FastJsonparser::ParseError (parse error)
Example
> sample_json = '{"a":"Alpha","b":true,"c":12345,"d":[true,[false,[-123456789,null],3.9676,["Something else.",false],null]],"e":{"zero":null,"one":1,"two":2,"three":[3],"four":[0,1,2,3,4]},"f":null,"h":{"a":{"b":{"c":{"d":{"e":{"f":{"g":null}}}}}}},"i":[[[[[[[null]]]]]]]}'
> FastJsonparser.parse(sample_json)
=> {:a=>"Alpha", :b=>true, :c=>12345, :d=>[true, [false, [-123456789, nil], 3.9676, ["Something else.", false], nil]], :e=>{:zero=>nil, :one=>1, :two=>2, :three=>[3], :four=>[0, 1, 2, 3, 4]}, :f=>nil, :h=>{:a=>{:b=>{:c=>{:d=>{:e=>{:f=>{:g=>nil}}}}}}}, :i=>[[[[[[[nil]]]]]]]}
Development
After checking out the repo, run bin/setup
to install dependencies. Then, run rake test
to run the tests. You can also run bin/console
for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run bundle exec rake install
. To release a new version, update the version number in version.rb
, and then run bundle exec rake release
, which will create a git tag for the version, push git commits and tags, and push the .gem
file to rubygems.org.
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/anilmaurya/fast_jsonparser. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
License
The gem is available as open source under the terms of the MIT License.
Code of Conduct
Everyone interacting in the FastJsonparser project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.