Class: LogStash::Filters::KV

Inherits:
Base
  • Object
show all
Defined in:
lib/logstash/filters/kv.rb

Overview

This filter helps automatically parse messages (or specific event fields) which are of the ‘foo=bar` variety.

For example, if you have a log message which contains ‘ip=1.2.3.4 error=REFUSED`, you can parse those automatically by configuring:

source,ruby

filter {

kv { }

}

The above will result in a message of ‘ip=1.2.3.4 error=REFUSED` having the fields:

  • ‘ip: 1.2.3.4`

  • ‘error: REFUSED`

This is great for postfix, iptables, and other types of logs that tend towards ‘key=value` syntax.

You can configure any arbitrary strings to split your data on, in case your data is not structured using ‘=` signs and whitespace. For example, this filter can also be used to parse query parameters like `foo=bar&baz=fizz` by setting the `field_split` parameter to `&`.

Instance Method Summary collapse

Instance Method Details

#filter(event) ⇒ Object 



225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
 
# File 'lib/logstash/filters/kv.rb', line 225

def filter(event)
  kv = Hash.new
  value = event[@source]

  case value
  when nil
    # Nothing to do
  when String
    kv = parse(value, event, kv)
  when Array
    value.each { |v| kv = parse(v, event, kv) }
  else
    @logger.warn("kv filter has no support for this type of data", :type => value.class, :value => value)
  end

  # Add default key-values for missing keys
  kv = @default_keys.merge(kv)

  return if kv.empty?

  if @target
    @logger.debug? && @logger.debug("Overwriting existing target field", :target => @target)
    event[@target] = kv
  else
    kv.each{|k, v| event[k] = v}
  end

  filter_matched(event)
end

#register ⇒ Object 



214
215
216
217
218
219
220
221
222
223
 
# File 'lib/logstash/filters/kv.rb', line 214

def register
  @trim_re = Regexp.new("[#{@trim}]") if @trim
  @trimkey_re = Regexp.new("[#{@trimkey}]") if @trimkey

  valueRxString = "(?:\"([^\"]+)\"|'([^']+)'"
  valueRxString += "|\\(([^\\)]+)\\)|\\[([^\\]]+)\\]" if @include_brackets
  valueRxString += "|((?:\\\\ |[^" + @field_split + "])+))"
  @scan_re = Regexp.new("((?:\\\\ |[^" + @field_split + @value_split + "])+)\s*[" + @value_split + "]\s*" + valueRxString)
  @value_split_re = /[#{@value_split}]/
end