Fluent::Plugin::Rambler
A variant of S3 output plugin for Amazon Redshift.
This plugin works with a command line utility called "rambler":
https://github.com/k24d/rambler
Installation
Add this line to your application's Gemfile:
gem 'fluent-plugin-rambler'
And then execute:
$ bundle
Or install it yourself as:
$ gem install fluent-plugin-rambler
Configuration
RamblerOutput is a subclass of S3Output, and most configuration parameters are inherited from out_s3.
http://docs.fluentd.org/articles/out_s3
There are additional parameters specific to out_rambler:
<match myapp.event1>
type rambler
# for fluent-plugin-s3
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET/KEY
s3_bucket YOUR_S3_BUCKET_NAME
s3_endpoint s3-ap-northeast-1.amazonaws.com
# NOTE: don't forget to put '/' after %{time_slice}.
s3_object_key_format %{path}%{time_slice}/%{hostname}-%{index}.%{file_extension}
# remote path in the convension of Rambler time-series tables
path redshift/DATABASE/VIEW_%Y%m/
buffer_path /var/log/fluent/DATABASE.VIEW
# the buffer will be flushed with this time slice
time_slice_format %Y%m%d
time_slice_wait 10m
utc
# The default value corresponds to Redshift's default timestamp: YYYY-MM-DD HH:MI:SS
#time_format %Y-%m-%d %H:%M:%S
# for Rambler
columns %{time}|key1|key2|...
</match>
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create new Pull Request