Class: AvroTurf

Inherits:
Object
  • Object
show all
Defined in:
lib/avro_turf/test/fake_server.rb,
lib/avro_turf.rb,
lib/avro_turf/version.rb,
lib/avro_turf/messaging.rb,
lib/avro_turf/mutable_schema_store.rb,
lib/avro_turf/schema_to_avro_patch.rb

Overview

Ensure AvroTurf class exists so we can add modules to it This is defined as a class (not module) in lib/avro_turf/version.rb

Defined Under Namespace

Modules: AvroGemPatch, Test Classes: CachedConfluentSchemaRegistry, ConfluentSchemaRegistry, DiskCache, Error, InMemoryCache, IncompatibleSchemaError, Messaging, MutableSchemaStore, SchemaError, SchemaNotFoundError, SchemaStore

Constant Summary collapse

DEFAULT_SCHEMAS_PATH = 
"./schemas"
VERSION = 
"1.20.1"
SchemaRegistry =

AvroTurf::SchemaRegistry is deprecated and will be removed in a future release. Use AvroTurf::ConfluentSchemaRegistry instead.

AvroTurf::ConfluentSchemaRegistry
CachedSchemaRegistry =

AvroTurf::CachedSchemaRegistry is deprecated and will be removed in a future release. Use AvroTurf::CachedConfluentSchemaRegistry instead.

AvroTurf::CachedConfluentSchemaRegistry

Instance Method Summary collapse

Constructor Details

#initialize(schemas_path: nil, schema_store: nil, namespace: nil, codec: nil) ⇒ AvroTurf

Create a new AvroTurf instance with the specified configuration.

schemas_path - The String path to the root directory containing Avro schemas (default: “./schemas”). schema_store - A schema store object that responds to #find(schema_name, namespace). namespace - The String namespace that should be used to qualify schema names (optional). codec - The String name of a codec that should be used to compress messages (optional).

Currently, the only valid codec name is ‘deflate`.



36
37
38
39
40
41
 
# File 'lib/avro_turf.rb', line 36

def initialize(schemas_path: nil, schema_store: nil, namespace: nil, codec: nil)
  @namespace = namespace
  @schema_store = schema_store ||
    SchemaStore.new(path: schemas_path || DEFAULT_SCHEMAS_PATH)
  @codec = codec
end

Instance Method Details

#decode_all(encoded_data, schema_name: nil, namespace: @namespace) ⇒ Object

Returns all entries encoded in the data.



106
107
108
109
 
# File 'lib/avro_turf.rb', line 106

def decode_all(encoded_data, schema_name: nil, namespace: @namespace)
  stream = StringIO.new(encoded_data)
  decode_all_from_stream(stream, schema_name: schema_name, namespace: namespace)
end

#decode_all_from_stream(stream, schema_name: nil, namespace: @namespace) ⇒ Object

Returns all entries encoded in the stream.



127
128
129
130
131
 
# File 'lib/avro_turf.rb', line 127

def decode_all_from_stream(stream, schema_name: nil, namespace: @namespace)
  schema = schema_name && @schema_store.find(schema_name, namespace)
  reader = Avro::IO::DatumReader.new(nil, schema)
  Avro::DataFile::Reader.new(stream, reader)
end

#decode_first(encoded_data, schema_name: nil, namespace: @namespace) ⇒ Object Also known as: decode

Decodes Avro data.

encoded_data - A String containing Avro-encoded data. schema_name - The String name of the schema that should be used to read

the data. If nil, the writer schema will be used.

namespace - The namespace of the Avro schema used to decode the data.

Returns whatever is encoded in the data.



98
99
100
101
 
# File 'lib/avro_turf.rb', line 98

def decode_first(encoded_data, schema_name: nil, namespace: @namespace)
  stream = StringIO.new(encoded_data)
  decode_stream(stream, schema_name: schema_name, namespace: namespace)
end

#decode_first_from_stream(stream, schema_name: nil, namespace: @namespace) ⇒ Object Also known as: decode_stream

Decodes the first entry from an IO stream containing Avro data.

stream - An IO object containing Avro data. schema_name - The String name of the schema that should be used to read

the data. If nil, the writer schema will be used.

namespace - The namespace of the Avro schema used to decode the data.

Returns first entry encoded in the stream.



119
120
121
122
 
# File 'lib/avro_turf.rb', line 119

def decode_first_from_stream(stream, schema_name: nil, namespace: @namespace)
  data = decode_all_from_stream(stream, schema_name: schema_name, namespace: namespace)
  data.first
end

#encode(data, schema_name: nil, namespace: @namespace, validate: false) ⇒ Object

Encodes data to Avro using the specified schema.

data - The data that should be encoded. schema_name - The name of a schema in the ‘schemas_path`. validate - The boolean for performing complete data validation before

encoding it, Avro::SchemaValidator::ValidationError with
a descriptive message will be raised in case of invalid message.

Returns a String containing the encoded data.



52
53
54
55
56
57
58
 
# File 'lib/avro_turf.rb', line 52

def encode(data, schema_name: nil, namespace: @namespace, validate: false)
  stream = StringIO.new

  encode_to_stream(data, stream: stream, schema_name: schema_name, namespace: namespace, validate: validate)

  stream.string
end

#encode_to_stream(data, schema_name: nil, stream: nil, namespace: @namespace, validate: false, validate_options: {recursive: true, encoded: false, fail_on_extra_fields: true}) ⇒ Object

Encodes data to Avro using the specified schema and writes it to the specified stream.

data - The data that should be encoded. schema_name - The name of a schema in the ‘schemas_path`. stream - An IO object that the encoded data should be written to (optional). validate - The boolean for performing complete data validation before

encoding it, Avro::SchemaValidator::ValidationError with
a descriptive message will be raised in case of invalid message.

validate_options - Hash for the Avro::SchemaValidator, default

{recursive: true, encoded: false, fail_on_extra_fields: true}

Returns nothing.



73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
 
# File 'lib/avro_turf.rb', line 73

def encode_to_stream(data, schema_name: nil, stream: nil, namespace: @namespace,
  validate: false,
  validate_options: {recursive: true,
                     encoded: false,
                     fail_on_extra_fields: true})
  schema = @schema_store.find(schema_name, namespace)
  writer = Avro::IO::DatumWriter.new(schema)

  if validate
    Avro::SchemaValidator.validate!(schema, data, **validate_options)
  end

  dw = Avro::DataFile::Writer.new(stream, writer, schema, @codec)
  dw << data.as_avro
  dw.close
end

#load_schemas! ⇒ Object

Loads all schema definition files in the ‘schemas_dir`.



147
148
149
 
# File 'lib/avro_turf.rb', line 147

def load_schemas!
  @schema_store.load_schemas!
end

#valid?(data, schema_name: nil, namespace: @namespace, validate_options: {}) ⇒ Boolean

Validates data against an Avro schema.

data - The data that should be validated. schema - The String name of the schema that should be used to validate

the data.

namespace - The namespace of the Avro schema (optional).

Returns true if the data is valid, false otherwise.

Returns:

  • (Boolean) 


141
142
143
144
 
# File 'lib/avro_turf.rb', line 141

def valid?(data, schema_name: nil, namespace: @namespace, validate_options: {})
  schema = schema_name && @schema_store.find(schema_name, namespace)
  Avro::Schema.validate(schema, data.as_avro, **validate_options)
end