Class: LogStash::Codecs::Avro

Inherits:
Base
  • Object
show all
Defined in:
lib/logstash/codecs/avro.rb

Overview

Read serialized Avro records as Logstash events

This plugin is used to serialize Logstash events as Avro datums, as well as deserializing Avro datums into Logstash events.

Encoding

This codec is for serializing individual Logstash events as Avro datums that are Avro binary blobs. It does not encode Logstash events into an Avro file.

Decoding

This codec is for deserializing individual Avro records. It is not for reading Avro files. Avro files have a unique format that must be handled upon input.

Usage

Example usage with Kafka input.

source,ruby

input {

kafka {
  codec => avro {
      schema_uri => "/tmp/schema.avsc"
  }
}

} filter 

...

output 

...

Instance Method Summary collapse

Instance Method Details

#decode(data) {|LogStash::Event.new(datum_reader.read(decoder))| ... } ⇒ Object

Yields:

  • (LogStash::Event.new(datum_reader.read(decoder))) 


69
70
71
72
73
74
 
# File 'lib/logstash/codecs/avro.rb', line 69

def decode(data)
  datum = StringIO.new(data)
  decoder = Avro::IO::BinaryDecoder.new(datum)
  datum_reader = Avro::IO::DatumReader.new(@schema)
  yield LogStash::Event.new(datum_reader.read(decoder))
end

#encode(event) ⇒ Object 



77
78
79
80
81
82
83
 
# File 'lib/logstash/codecs/avro.rb', line 77

def encode(event)
  dw = Avro::IO::DatumWriter.new(@schema)
  buffer = StringIO.new
  encoder = Avro::IO::BinaryEncoder.new(buffer)
  dw.write(event.to_hash, encoder)
  @on_event.call(event, buffer.string)
end

#open_and_read(uri_string) ⇒ Object 



59
60
61
 
# File 'lib/logstash/codecs/avro.rb', line 59

def open_and_read(uri_string)
  open(uri_string).read
end

#register ⇒ Object 



64
65
66
 
# File 'lib/logstash/codecs/avro.rb', line 64

def register
  @schema = Avro::Schema.parse(open_and_read(schema_uri))
end