Module: Rimless::AvroHelpers

Extended by:
ActiveSupport::Concern
Included in:
Rimless
Defined in:
lib/rimless/avro_helpers.rb

Overview

The top-level Apache Avro helpers.

Class Method Summary collapse

Class Method Details

.avro_decode(data, **opts) ⇒ Mixed Also known as: decode

A shortcut to parse a blob of Apache Avro data.

Parameters:

  • data (String) —

    the Apache Avro blob

  • opts (Hash{Symbol => Mixed}) —

    additional options

Returns:

  • (Mixed) —

    the decoded data structure



42
43
44
 
# File 'lib/rimless/avro_helpers.rb', line 42

def avro_decode(data, **opts)
  avro.decode(data, **opts).deep_symbolize_keys!
end

.avro_encode(data, schema:, **opts) ⇒ String Also known as: encode

A shortcut to encode data using the specified schema to the Apache Avro format. This also applies data sanitation to avoid issues with the low level Apache Avro library (symbolized keys, etc) and it allows deep-relative schema names. When you pass .deep.deep for example (leading period) it will prefix the schema name with the local namespace (so it becomes absolute).

Parameters:

  • data (Mixed) —

    the data structure to encode

  • schema (String, Symbol) —

    name of the schema that should be used

  • opts (Hash{Symbol => Mixed}) —

    additional options

Returns:

  • (String) —

    the Apache Avro blob



25
26
27
28
29
30
31
32
33
34
 
# File 'lib/rimless/avro_helpers.rb', line 25

def avro_encode(data, schema:, **opts)
  data = avro_sanitize(data)

  # When the deep-relative form (+.deep.deep[..]+) is present, we add our
  # local namespace, so Avro can resolve it
  schema = avro_utils.namespace + schema.to_s \
    if schema.to_s.start_with? '.'

  avro.encode(data, schema_name: schema.to_s, **opts)
end

.avro_schemaless_h(hash) ⇒ Hash{String => String} Also known as: avro_schemaless_map

Convert the given deep hash into a sparsed flat hash while transforming all values to strings. This allows to convert a schema-less hash to a Apache Avro compatible map.

Examples:

Convert schema-less hash

avro_schemaless_map(a: { b: { c: true } })
# => { "a.b.c" => "true" }

Parameters:

  • hash (Hash{Mixed => Mixed}) —

    the deep hash

Returns:

  • (Hash{String => String}) —

    the flatted and sparsed hash

See Also:



73
74
75
76
77
 
# File 'lib/rimless/avro_helpers.rb', line 73

def avro_schemaless_h(hash)
  Sparsify(hash, sparse_array: true)
    .transform_values(&:to_s)
    .transform_keys { |key| key.delete('\\') }
end

.avro_to_h(hash) ⇒ Hash{String => Mixed} Also known as: avro_sanitize

The Apache Avro Ruby gem requires simple typed hashes for encoding. This forces us to convert eg. Grape entity representations into simple string-keyed hashes. Use this method to prepare a hash for the Apache Avro serialization.

Note about the implementation: JSON serialization and parsing is the simplest and fastest way to accomplish this.

Parameters:

  • hash (Hash{Mixed => Mixed}) —

    the hash to sanitize

Returns:

  • (Hash{String => Mixed}) —

    the simple typed input hash



57
58
59
 
# File 'lib/rimless/avro_helpers.rb', line 57

def avro_to_h(hash)
  JSON.parse(hash.to_json)
end