Class: Chicago::ETL::DeduplicateRows

Inherits:
Transformation show all
Defined in:
lib/chicago/etl/transformations/deduplicate_rows.rb

Instance Method Summary collapse

Methods inherited from Transformation

#added_fields, added_fields, adds_fields, #applies_to_stream?, #downstream_fields, #initialize, #output_streams, #process, removed_fields, #removed_fields, removes_fields, #required_options, required_options, requires_options, #upstream_fields

Constructor Details

This class inherits a constructor from Chicago::ETL::Transformation

Instance Method Details

#flush ⇒ Object 



16
17
18
 
# File 'lib/chicago/etl/transformations/deduplicate_rows.rb', line 16

def flush
  @working_row.nil? ? [] : [return_row(@working_row)]
end

#process_row(row) ⇒ Object 



4
5
6
7
8
9
10
11
12
13
14
 
# File 'lib/chicago/etl/transformations/deduplicate_rows.rb', line 4

def process_row(row)
  if @working_row.nil?
    @working_row = new_row(row)
    return
  elsif same_row?(row)
    @working_row = merge_rows(row)
    return
  else
    assign_new_row_and_return_old_row(row)
  end
end