Class: LogStash::Filters::KV

Inherits:
Base
  • Object
show all
Defined in:
lib/logstash/filters/kv.rb

Overview

This filter helps automatically parse messages (or specific event fields) which are of the ‘foo=bar’ variety.

For example, if you have a log message which contains ‘ip=1.2.3.4 error=REFUSED’, you can parse those automatically by configuring:

filter {
  kv { }
}

The above will result in a message of “ip=1.2.3.4 error=REFUSED” having the fields:

  • ip: 1.2.3.4

  • error: REFUSED

This is great for postfix, iptables, and other types of logs that tend towards ‘key=value’ syntax.

You can configure any arbitrary strings to split your data on, in case your data is not structured using ‘=’ signs and whitespace. For example, this filter can also be used to parse query parameters like ‘foo=bar&baz=fizz’ by setting the ‘field_split` parameter to “&”.

Instance Method Summary collapse

Instance Method Details

#filter(event) ⇒ Object

def register



166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
 
# File 'lib/logstash/filters/kv.rb', line 166

def filter(event)
  return unless filter?(event)

  kv = Hash.new

  value = event[@source]

  case value
    when nil; # Nothing to do
    when String; kv = parse(value, event, kv)
    when Array; value.each { |v| kv = parse(v, event, kv) }
    else
      @logger.warn("kv filter has no support for this type of data",
                   :type => value.class, :value => value)
  end # case value

  # Add default key-values for missing keys
  kv = @default_keys.merge(kv)

  # If we have any keys, create/append the hash
  if kv.length > 0
    if @target.nil?
      # Default is to write to the root of the event.
      dest = event.to_hash
    else
      if !event[@target].is_a?(Hash)
        @logger.debug("Overwriting existing target field", :target => @target)
        dest = event[@target] = {}
      else
        dest = event[@target]
      end
    end

    dest.merge!(kv)
    filter_matched(event)
  end
end

#register ⇒ Object 



160
161
162
163
164
 
# File 'lib/logstash/filters/kv.rb', line 160

def register
  @trim_re = Regexp.new("[#{@trim}]") if !@trim.nil?
  @trimkey_re = Regexp.new("[#{@trimkey}]") if !@trimkey.nil?
  @scan_re = Regexp.new("((?:\\\\ |[^"+@field_split+@value_split+"])+)["+@value_split+"](?:\"([^\"]+)\"|'([^']+)'|((?:\\\\ |[^"+@field_split+"])+))")
end