Class: AvroTurf
- Inherits:
-
Object
- Object
- AvroTurf
- Defined in:
- lib/avro_turf.rb,
lib/avro_turf/version.rb,
lib/avro_turf/messaging.rb,
lib/avro_turf/mutable_schema_store.rb,
lib/avro_turf/schema_to_avro_patch.rb
Defined Under Namespace
Modules: AvroGemPatch Classes: CachedConfluentSchemaRegistry, ConfluentSchemaRegistry, DiskCache, Error, InMemoryCache, Messaging, MutableSchemaStore, SchemaError, SchemaNotFoundError, SchemaStore
Constant Summary collapse
- DEFAULT_SCHEMAS_PATH =
"./schemas"
- VERSION =
"1.10.0"
- SchemaRegistry =
AvroTurf::SchemaRegistry is deprecated and will be removed in a future release. Use AvroTurf::ConfluentSchemaRegistry instead.
AvroTurf::ConfluentSchemaRegistry
- CachedSchemaRegistry =
AvroTurf::CachedSchemaRegistry is deprecated and will be removed in a future release. Use AvroTurf::CachedConfluentSchemaRegistry instead.
AvroTurf::CachedConfluentSchemaRegistry
Instance Method Summary collapse
-
#decode(encoded_data, schema_name: nil, namespace: @namespace) ⇒ Object
Decodes Avro data.
-
#decode_stream(stream, schema_name: nil, namespace: @namespace) ⇒ Object
Decodes Avro data from an IO stream.
-
#encode(data, schema_name: nil, namespace: @namespace, validate: false) ⇒ Object
Encodes data to Avro using the specified schema.
-
#encode_to_stream(data, schema_name: nil, stream: nil, namespace: @namespace, validate: false, validate_options: { recursive: true, encoded: false, fail_on_extra_fields: true }) ⇒ Object
Encodes data to Avro using the specified schema and writes it to the specified stream.
-
#initialize(schemas_path: nil, schema_store: nil, namespace: nil, codec: nil) ⇒ AvroTurf
constructor
Create a new AvroTurf instance with the specified configuration.
-
#load_schemas! ⇒ Object
Loads all schema definition files in the ‘schemas_dir`.
-
#valid?(data, schema_name: nil, namespace: @namespace, validate_options: {}) ⇒ Boolean
Validates data against an Avro schema.
Constructor Details
#initialize(schemas_path: nil, schema_store: nil, namespace: nil, codec: nil) ⇒ AvroTurf
Create a new AvroTurf instance with the specified configuration.
schemas_path - The String path to the root directory containing Avro schemas (default: “./schemas”). schema_store - A schema store object that responds to #find(schema_name, namespace). namespace - The String namespace that should be used to qualify schema names (optional). codec - The String name of a codec that should be used to compress messages (optional).
Currently, the only valid codec name is ‘deflate`.
34 35 36 37 38 39 |
# File 'lib/avro_turf.rb', line 34 def initialize(schemas_path: nil, schema_store: nil, namespace: nil, codec: nil) @namespace = namespace @schema_store = schema_store || SchemaStore.new(path: schemas_path || DEFAULT_SCHEMAS_PATH) @codec = codec end |
Instance Method Details
#decode(encoded_data, schema_name: nil, namespace: @namespace) ⇒ Object
Decodes Avro data.
encoded_data - A String containing Avro-encoded data. schema_name - The String name of the schema that should be used to read
the data. If nil, the writer schema will be used.
namespace - The namespace of the Avro schema used to decode the data.
Returns whatever is encoded in the data.
96 97 98 99 |
# File 'lib/avro_turf.rb', line 96 def decode(encoded_data, schema_name: nil, namespace: @namespace) stream = StringIO.new(encoded_data) decode_stream(stream, schema_name: schema_name, namespace: namespace) end |
#decode_stream(stream, schema_name: nil, namespace: @namespace) ⇒ Object
Decodes Avro data from an IO stream.
stream - An IO object containing Avro data. schema_name - The String name of the schema that should be used to read
the data. If nil, the writer schema will be used.
namespace - The namespace of the Avro schema used to decode the data.
Returns whatever is encoded in the stream.
109 110 111 112 113 114 |
# File 'lib/avro_turf.rb', line 109 def decode_stream(stream, schema_name: nil, namespace: @namespace) schema = schema_name && @schema_store.find(schema_name, namespace) reader = Avro::IO::DatumReader.new(nil, schema) dr = Avro::DataFile::Reader.new(stream, reader) dr.first end |
#encode(data, schema_name: nil, namespace: @namespace, validate: false) ⇒ Object
Encodes data to Avro using the specified schema.
data - The data that should be encoded. schema_name - The name of a schema in the ‘schemas_path`. validate - The boolean for performing complete data validation before
encoding it, Avro::SchemaValidator::ValidationError with
a descriptive message will be raised in case of invalid message.
Returns a String containing the encoded data.
50 51 52 53 54 55 56 |
# File 'lib/avro_turf.rb', line 50 def encode(data, schema_name: nil, namespace: @namespace, validate: false) stream = StringIO.new encode_to_stream(data, stream: stream, schema_name: schema_name, namespace: namespace, validate: validate) stream.string end |
#encode_to_stream(data, schema_name: nil, stream: nil, namespace: @namespace, validate: false, validate_options: { recursive: true, encoded: false, fail_on_extra_fields: true }) ⇒ Object
Encodes data to Avro using the specified schema and writes it to the specified stream.
data - The data that should be encoded. schema_name - The name of a schema in the ‘schemas_path`. stream - An IO object that the encoded data should be written to (optional). validate - The boolean for performing complete data validation before
encoding it, Avro::SchemaValidator::ValidationError with
a descriptive message will be raised in case of invalid message.
validate_options - Hash for the Avro::SchemaValidator, default
{recursive: true, encoded: false, fail_on_extra_fields: true}
Returns nothing.
71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
# File 'lib/avro_turf.rb', line 71 def encode_to_stream(data, schema_name: nil, stream: nil, namespace: @namespace, validate: false, validate_options: { recursive: true, encoded: false, fail_on_extra_fields: true }) schema = @schema_store.find(schema_name, namespace) writer = Avro::IO::DatumWriter.new(schema) if validate Avro::SchemaValidator.validate!(schema, data, **) end dw = Avro::DataFile::Writer.new(stream, writer, schema, @codec) dw << data.as_avro dw.close end |
#load_schemas! ⇒ Object
Loads all schema definition files in the ‘schemas_dir`.
130 131 132 |
# File 'lib/avro_turf.rb', line 130 def load_schemas! @schema_store.load_schemas! end |
#valid?(data, schema_name: nil, namespace: @namespace, validate_options: {}) ⇒ Boolean
Validates data against an Avro schema.
data - The data that should be validated. schema - The String name of the schema that should be used to validate
the data.
namespace - The namespace of the Avro schema (optional).
Returns true if the data is valid, false otherwise.
124 125 126 127 |
# File 'lib/avro_turf.rb', line 124 def valid?(data, schema_name: nil, namespace: @namespace, validate_options: {}) schema = schema_name && @schema_store.find(schema_name, namespace) Avro::Schema.validate(schema, data.as_avro, **) end |