Fluent plugin for Amazon Kinesis - Intuit

PURPOSE

Fluent plugin for Amazon Kinesis modified to be able to pass Splunk HEC Overrides

DESCRIPTION

In addition to the features provided by Amazon Kinesis fluentd plugin, the following Splunk HEC Overrides can be passed as config params. The same overrides are reflected at the Splunk endpoint.

  1. Splunk index
  2. Source
  3. Source type
  4. Host name

Example: The config params can be passed as given below:

 @type kinesis_streams

  # your kinesis stream name
  stream_name test
  index awesome
  source ${tag}
  sourcetype _json
  host test-host

If the host name is not specified, the instance host name is sent across.

This configuration will

  1. send all events to the awesome index, and
  2. set their source to the event tags. $tag is a special value which will be replaced by the event tags, and
  3. set their sourcetype to _json.

Installation Procedure:

The gem can be installed using 

    gem install pkg/fluent-plugin-kinesis-intuit-2.1.1.gem

You can also build and install using

    git clone https://github.intuit.com/cloud-logging/aws-fluent-plugin-kinesis-intuit.git
    cd aws-fluent-plugin-kinesis-intuit
    bundle install
    bundle exec rake build
    bundle exec rake install

Dependencies

Ruby 2.1.0+
Fluentd 0.14.10+

Please refer to the following link to know more about Fluent plugin for Amazon Kinesis

https://github.com/awslabs/aws-fluent-plugin-kinesis