Aliyun DataHub Plugin for LogStash

Getting Started

---

介绍

• 该插件是基于logstash开发的输入插件，它主要完成消费DataHub(阿里云产品)服务上的数据。

安装

• 环境要求linux, jdk1.7+, logstash（可选，如果没安装也没关系）
• 从流计算官网下载tar包，使用以下命令安装

如果之前没安装过logstash，请使用以下步骤安装

$ tar -xzvf logstash-with-datahub-2.3.0.tar.gz
$ cd logstash-with-datahub-2.3.0

如果之前安装过logstash，拿到logstash-input-datahub-1.0.0.gem，再使用以下命令安装

$ ${LOGSTASH_HOME}/bin/logstash-plugin install --local logstash-input-datahub-1.0.0.gem

样例

logstash的配置如下：

input {
    datahub {
        access_id => "Your accessId"
        access_key => "Your accessKey"
        endpoint => "http://dh-cn-hangzhou.aliyuncs.com"
        project_name => "test_project"
        topic_name => "test_logstash"
        interval=> 5
        #cursor => {
        #    "0"=>"20000000000000000000000003110091"
        #    "2"=>"20000000000000000000000003110091"
        #    "1"=>"20000000000000000000000003110091"
        #    "4"=>"20000000000000000000000003110091"
        #    "3"=>"2000000000000000000000000311000"
        #}
        shard_ids => []
        pos_file => "/home/admin/logstash/logstash-2.4.0/pos_file"
    }
}

output {
    file {
        path => "/home/admin/logstash/logstash-2.4.0/output"
    }
}

参数介绍

access_id(Required): 

相关参考

---

• LogStash主页
• LogStash插件开发

Authors && Contributors

---

• Dong Xiao

License

---

licensed under the Apache License 2.0