Class: LogStash::Filters::Split

Inherits:
Base
  • Object
show all
Defined in:
lib/logstash/filters/split.rb

Overview

The split filter clones an event by splitting one of its fields and placing each value resulting from the split into a clone of the original event. The field being split can either be a string or an array.

An example use case of this filter is for taking output from the <<plugins-inputs-exec,exec input plugin>> which emits one event for the whole output of a command and splitting that output by newline - making each line an event.

Split filter can also be used to split array fields in events into individual events. A very common pattern in JSON & XML is to make use of lists to group data together.

For example, a json structure like this:

source,js

{ field1: …,

results: [
  { result ... },
  { result ... },
  { result ... },
  ...

] }

The split filter can be used on the above data to create separate events for each value of results field

source,js

filter {

split {
  field => "results"
}

}

The end result of each split is a complete copy of the event with only the current split section of the given field changed.

Constant Summary collapse

PARSE_FAILURE_TAG = 
'_split_type_failure'.freeze

Instance Method Summary collapse

Instance Method Details

#filter(event) ⇒ Object 



67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/logstash/filters/split.rb', line 67

def filter(event)
  

  original_value = event.get(@field)

  if original_value.is_a?(Array)
    splits = target.nil? ? event.remove(@field) : original_value
  elsif original_value.is_a?(String)
    # Using -1 for 'limit' on String#split makes ruby not drop trailing empty
    # splits.
    splits = original_value.split(@terminator, -1)
  else
    logger.warn("Only String and Array types are splittable. field:#{@field} is of type = #{original_value.class}")
    event.tag(PARSE_FAILURE_TAG)
    return
  end

  # Skip filtering if splitting this event resulted in only one thing found.
  return if splits.length == 1 && original_value.is_a?(String)
  #or splits[1].empty?

  splits.each do |value|
    next if value.nil? || value.empty?

    event_split = event.clone
    @logger.debug("Split event", :value => value, :field => @field)
    if @target.nil?
      event_split.set(@field, value)
    else
      event_split.set(@target, value)
    end
    filter_matched(event_split)

    # Push this new event onto the stack at the LogStash::FilterWorker
    yield event_split
  end

  # Cancel this event, we'll use the newly generated ones above.
  event.cancel
end

#register ⇒ Object 



62
63
64
 
# File 'lib/logstash/filters/split.rb', line 62

def register
  # Nothing to do
end