Class: EM::Voldemort::BinaryJson

Inherits:
Object
  • Object
show all
Defined in:
lib/em-voldemort/binary_json.rb

Overview

Codec for Voldemort’s custom binary serialization format. The Voldemort codebase itself refers to this format as “json”, even though it has virtually nothing in common with JSON. It’s actually more like Avro, but with less sophisticated schema evolution, and less compact. We’re only using it because the Hadoop job for building read-only stores requires it. The format is roughly documented at github.com/voldemort/voldemort/wiki/Binary-JSON-Serialization

This code is adapted from Alejandro Crosa’s voldemort-rb gem (MIT License). github.com/acrosa/voldemort-rb

Constant Summary collapse

BYTE_MIN_VAL = 
-2**7
BYTE_MAX_VAL = 
2**7 - 1
SHORT_MIN_VAL = 
-2**15
SHORT_MAX_VAL = 
2**15 - 1
INT_MIN_VAL = 
-2**31
INT_MAX_VAL = 
2**31 - 1
LONG_MIN_VAL = 
-2**63
LONG_MAX_VAL = 
2**63 - 1
FLOAT_MIN_VAL = 
2.0**-149
DOUBLE_MIN_VAL = 
2.0**-1074
STRING_MAX_LEN = 
0x3FFFFFFF

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(schema_by_version, has_version_tag = true) ⇒ BinaryJson

Returns a new instance of BinaryJson.



27
28
29
30
31
32
 
# File 'lib/em-voldemort/binary_json.rb', line 27

def initialize(schema_by_version, has_version_tag=true)
  @has_version_tag = has_version_tag
  @schema_versions = schema_by_version.each_with_object({}) do |(version, schema), hash|
    hash[version.to_i] = parse_schema(schema)
  end
end

Instance Attribute Details

#has_version_tag ⇒ Object (readonly)

Returns the value of attribute has_version_tag.



12
13
14
 
# File 'lib/em-voldemort/binary_json.rb', line 12

def has_version_tag
  @has_version_tag
end

#schema_versions ⇒ Object (readonly)

Returns the value of attribute schema_versions.



13
14
15
 
# File 'lib/em-voldemort/binary_json.rb', line 13

def schema_versions
  @schema_versions
end

Instance Method Details

#decode(bytes) ⇒ Object

Parses a binary JSON string into Ruby objects

Raises:



45
46
47
48
49
50
51
52
 
# File 'lib/em-voldemort/binary_json.rb', line 45

def decode(bytes)
  bytes.force_encoding(Encoding::BINARY)
  input = StringIO.new(bytes)
  version = has_version_tag ? input.read(1).ord : 0
  schema = schema_versions[version]
  raise ClientError, "no registered schema for version #{version}" unless schema
  read(input, schema)
end

#encode(object) ⇒ Object

Serializes a Ruby object to binary JSON



35
36
37
38
39
40
41
42
 
# File 'lib/em-voldemort/binary_json.rb', line 35

def encode(object)
  ''.force_encoding(Encoding::BINARY).tap do |bytes|
    newest_version = schema_versions.keys.max
    schema = schema_versions[newest_version]
    bytes << newest_version.chr if has_version_tag
    write(object, bytes, schema)
  end
end