Class: Wukong::Processor::Uniq

Inherits:
Accumulator show all
Defined in:
lib/wukong/widget/reducers/uniq.rb

Overview

A processor which emits only unique records from its input. It's intended to work just like uniq.

Examples:

Emit unique elements from the input (like uniq).


$ uniq input
apple
banana
pear
$ cat input | wu-local uniq
apple
banana
pear

Emit unique elements from the input with counts (like uniq -c).


$ uniq -c input
     3 apple
     2 banana
     3 pear
$ cat input | wu-local uniq --count --to=tsv
apple	3
banana	5
pear	8

Constant Summary

Constants inherited from Wukong::Processor

SerializerError

Instance Attribute Summary collapse

Attributes inherited from Accumulator

#group, #key

Instance Method Summary collapse

Methods inherited from Accumulator

#get_key, #process, #start

Methods inherited from Wukong::Processor

configure, consumes, description, #expected_record_type, #expected_serialization, #perform_action, #process, produces, #receive_action, #stop, valid_serializer?, validate_and_set_serialization

Methods included from Logging

included

Methods included from Hanuman::StageClassMethods

#builder, #label, #register, #set_builder

Instance Attribute Details

#size ⇒ Object

The total size of the input recors.



60
61
62
 
# File 'lib/wukong/widget/reducers/uniq.rb', line 60

def size
  @size
end

Instance Method Details

#accumulate(record) ⇒ Object

Accumulate a record by incrmenting the total size.

Parameters:

  • record (Object) 


71
72
73
 
# File 'lib/wukong/widget/reducers/uniq.rb', line 71

def accumulate record
  self.size += 1
end

#finalize {|size| ... } ⇒ Object

Yields the total size.

Yields:

Yield Parameters:

  • size (Integer) 


79
80
81
82
83
84
85
 
# File 'lib/wukong/widget/reducers/uniq.rb', line 79

def finalize
  if count
    yield [key, self.size]
  else
    yield key
  end
end

#setup ⇒ Object

Initializes the count to 0.



63
64
65
66
 
# File 'lib/wukong/widget/reducers/uniq.rb', line 63

def setup
  super()
  @size = 0
end