Module: Enumerable

Defined in:
lib/tb/ex_enumerable.rb,
lib/tb/fileenumerator.rb

Overview

Copyright © 2012 Tanaka Akira <[email protected]>

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above
   copyright notice, this list of conditions and the following
   disclaimer in the documentation and/or other materials provided
   with the distribution.
3. The name of the author may not be used to endorse or promote
   products derived from this software without specific prior
   written permission.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Instance Method Summary collapse

Instance Method Details

#detect_group_by(before_group = nil, after_group = nil, &representative_proc) ⇒ Object

creates an enumerator which yields same as self but given block and procedures are called between each element for grouping.

The block is called for each element to define groups. A group is conecutive elements which the block returns same value.

before_group is called before each group with the first element.

after_group is called after each group with the last element.

before_group and after_group are optional.

The grouping mechanism is called as “control break” in some cluture such as COBOL.

Consecutive even numbers and odd numbers can be grouped as follows.

[1,3,5,4,8].detect_group_by(
  lambda {|v| puts "start" },
  lambda {|v| puts "end" }) {|v| v.even? }.each {|x| p x }
#=> start
#   1
#   3
#   5
#   end
#   start
#   4
#   8
#   end

Note that detect_group_by can be cascaeded but It doesn’t work as nested manner.

(0..9).detect_group_by( 
  lambda {|v| print "[" },
  lambda {|v| print "]" }) {|v|
  v.even?
}.detect_group_by(
  lambda {|v| print "(" },
  lambda {|v| print ")" }) {|v|
  (v/2).even?
}.each {|x| print x }
#=> [(0][1][)(2][3][)(4][5][)(6][7][)(8][9])

Consider detect_nested_group_by for nested groups.



582
583
584
 
# File 'lib/tb/ex_enumerable.rb', line 582

def detect_group_by(before_group=nil, after_group=nil, &representative_proc)
  detect_nested_group_by([[representative_proc, before_group, after_group]])
end

#detect_nested_group_by(group_specs) ⇒ Object

creates an enumerator which yields same as self but nested groups detected by group_specs

group_specs is an array of three procedures arrays as:

[[representative_proc1, before_proc1, after_proc1],
 [representative_proc2, before_proc2, after_proc2],
 ...]

representative_proc1 splits elements as groups. The group is defined as consecutive elements which representative_proc1 returns same value. before_proc1 is called before the each groups. after_proc1 is called after the each groups.

Subsequent procedures, representative_proc2, before_proc2, after_proc2, …, are used to split elements in the above groups.

(0..9).detect_nested_group_by(
  [[lambda {|v| (v/2).even? },
    lambda {|v| print "(" },
    lambda {|v| print ")" }],
   [lambda {|v| v.even? },
    lambda {|v| print "[" },
    lambda {|v| print "]" }]]).each {|x| print x }
#=> ([0][1])([2][3])([4][5])([6][7])([8][9])



612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
 
# File 'lib/tb/ex_enumerable.rb', line 612

def detect_nested_group_by(group_specs)
  Enumerator.new {|y|
    first = true
    prev_reps = nil
    prev = nil
    self.each {|*curr|
      reps = group_specs.map {|representative_proc, _, _|
        representative_proc.call(*curr)
      }
      if first
        first = false
        group_specs.each {|_, before_proc, _|
          before_proc.call(*curr) if before_proc
        }
      else
        different_index = (0...group_specs.length).find {|i| prev_reps[i] != reps[i] }
        if different_index
          (group_specs.length-1).downto(different_index) {|i|
            _, _, after_proc = group_specs[i]
            after_proc.call(*prev) if after_proc
          }
          different_index.upto(group_specs.length-1) {|i|
            _, before_proc, _ = group_specs[i]
            before_proc.call(*curr) if before_proc
          }
        end
      end
      y.yield(*curr)
      prev_reps = reps
      prev = curr
    }
    if !first
      (group_specs.length-1).downto(0) {|i|
        _, _, after_proc = group_specs[i]
        after_proc.call(*prev) if after_proc
      }
    end
  }
end

#each_group_element_by(representative, before_group, body, after_group) ⇒ Object

splits self by representative which is called with a element.

before_group is called before each group with the first element. after_group is called after each group with the last element. body is called for each element.



533
534
535
 
# File 'lib/tb/ex_enumerable.rb', line 533

def each_group_element_by(representative, before_group, body, after_group)
  detect_group_by(before_group, after_group, &representative).each(&body)
end

#extsort_by(opts = {}, &cmpvalue_from) ⇒ Object

:call-seq:

enum.extsort_by(options={}) {|value| cmpvalue }

extsort_by returns an enumerator which yields elements in the receiver in sorted order. The block defines the order which cmpvalue is ascending.

options:

:map : a procedure to convert the element.  It is applied after cmpvalue is obtained.  (default: nil)
:unique : a procedure to merge two values which has same cmpvalue. (default: nil)
:memsize : limit in-memory sorting size in bytes (default: 10000000)

If :unique option is given, it is used to merge elements which have same cmpvalue. The procedure should take two elements and return one. The procedure should be associative. (f(x,f(y,z)) = f(f(x,y),z))



314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
 
# File 'lib/tb/ex_enumerable.rb', line 314

def extsort_by(opts={}, &cmpvalue_from)
  mapfunc = opts[:map]
  opts = opts.dup
  opts[:map] = mapfunc ?
    lambda {|v| Marshal.dump(mapfunc.call(v)) } : 
    lambda {|v| Marshal.dump(v) }
  uniqfunc = opts[:unique]
  if uniqfunc
    opts[:unique] = lambda {|x, y| Marshal.dump(uniqfunc.call(Marshal.load(x), Marshal.load(y))) }
  end
  reducefunc = opts[:unique]
  mapfunc2 = opts[:map] || lambda {|v| v }
  self.lazy_map {|v|
    [cmpvalue_from.call(v), mapfunc2.call(v)]
  }.send(:extsort_internal0, reducefunc, opts).lazy_map {|k, d|
    Marshal.load(d)
  }
end

#extsort_reduce(op, opts = {}, &key_val_proc) ⇒ Object

:call-seq:

enum.extsort_reduce(op, [opts]) {|element| [key, val| }



336
337
338
 
# File 'lib/tb/ex_enumerable.rb', line 336

def extsort_reduce(op, opts={}, &key_val_proc)
  lazy_map(&key_val_proc).send(:extsort_internal0, op, opts)
end

#lazy_map ⇒ Object 



652
653
654
655
656
657
658
 
# File 'lib/tb/ex_enumerable.rb', line 652

def lazy_map
  Enumerator.new {|y|
    self.each {|*vs|
      y.yield(yield(*vs))
    }
  }
end

#tb_categorize(*args, &reduce_proc) ⇒ Object

:call-seq:

enum.tb_categorize(ksel1, ksel2, ..., vsel, [opts])
enum.tb_categorize(ksel1, ksel2, ..., vsel, [opts]) {|ks, vs| ... }

categorizes the elements in enum and returns a hash. This method assumes multiple elements for a category.

tb_categorize takes one or more key selectors, one value selector and an optional option hash. It also takes an optional block.

The selectors specify how to extract a value from an element in enum.

The key selectors, kselN, are used to extract hash keys from an element. If two or more key selectors are specified, the result hash will be nested.

The value selector, vsel, is used for the values of innermost hashes. By default, all values extracted by vsel from the elements which key selectors extracts same value are composed as an array. The array is set to the values of the innermost hashes. This behavior can be customized by the options: :seed, :op and :update.

a = [{:fruit => "banana", :color => "yellow", :taste => "sweet", :price => 100},
     {:fruit => "melon", :color => "green", :taste => "sweet", :price => 300},
     {:fruit => "grapefruit", :color => "yellow", :taste => "tart", :price => 200}]
p a.tb_categorize(:color, :fruit)
#=> {"yellow"=>["banana", "grapefruit"], "green"=>["melon"]}
p a.tb_categorize(:taste, :fruit)
#=> {"sweet"=>["banana", "melon"], "tart"=>["grapefruit"]}
p a.tb_categorize(:taste, :color, :fruit)
#=> {"sweet"=>{"yellow"=>["banana"], "green"=>["melon"]}, "tart"=>{"yellow"=>["grapefruit"]}}
p a.tb_categorize(:taste, :color)
#=> {"sweet"=>["yellow", "green"], "tart"=>["yellow"]}

In the above example, :fruit, :color and :taste is specified as selectors. There are several types of selectors as follows:

  • object with call method (procedure, etc.): extracts a value from the element by calling the procedure with the element as an argument.

  • array of selectors: make an array which contains the values extracted by the selectors.

  • other object: extracts a value from the element using [] method as element[selector].

So the selector :fruit extracts the value from the element :fruit => “banana”, :color => “yellow”, :taste => “sweet”, :price => 100} as {….

p a.tb_categorize(lambda {|elt| elt[:fruit][4] }, :fruit)
#=> {"n"=>["banana", "melon"], "e"=>["grapefruit"]}

When the key selectors returns same key for two or or more elements, corresponding values extracted by the value selector are combined. By default, all values are collected as an array. :seed, :op and :update option in the option hash customizes this behavior. :seed option and :op option is similar to Enumerable#inject. :seed option specifies an initial value. (If :seed option is not given, the first value for each category is treated as an initial value.) :op option specifies a procedure to combine a seed and an element into a next seed. :update option is same as :op option except it takes three arguments instead of two: keys, seed and element. to_proc method is used to convert :op and :update option to a procedure. So a symbol can be used for them.

# count categorized elements.
p a.tb_categorize(:color, lambda {|e| 1 }, :op=>:+)
#=> {"yellow"=>2, "green"=>1}

p a.tb_categorize(:color, :fruit, :seed=>"", :op=>:+)
#=> {"yellow"=>"bananagrapefruit", "green"=>"melon"}

The default behavior, collecting all values as an array, is implemented as follows.

:seed => nil
:update => {|ks, s, v| !s ? [v] : (s << v) }

:op and :update option are disjoint. ArgumentError is raised if both are specified.

The block for tb_categorize method converts combined values to final innermost hash values.

p a.tb_categorize(:color, :fruit) {|ks, vs| vs.join(",") }
#=> {"yellow"=>"banana,grapefruit", "green"=>"melon"}

# calculates the average price for fruits of each color.
p a.tb_categorize(:color, :price) {|ks, vs| vs.inject(0.0, &:+) / vs.length }
#=> {"yellow"=>150.0, "green"=>300.0}



113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
 
# File 'lib/tb/ex_enumerable.rb', line 113

def tb_categorize(*args, &reduce_proc)
  opts = args.last.kind_of?(Hash) ? args.pop : {}
  if args.length < 2
    raise ArgumentError, "needs 2 or more arguments without option hash (but #{args.length})"
  end
  value_selector = tb_cat_selector_proc(args.pop)
  key_selectors = args.map {|a| tb_cat_selector_proc(a) }
  has_seed = opts.has_key? :seed
  seed_value = opts[:seed]
  if opts.has_key?(:update) && opts.has_key?(:op)
    raise ArgumentError, "both :op and :update option specified"
  elsif opts.has_key? :update
    update_proc = opts[:update].to_proc
  elsif opts.has_key? :op
    op_proc = opts[:op].to_proc
    update_proc = lambda {|ks, s, v| op_proc.call(s, v) }
  else
    has_seed = true
    seed_value = nil
    update_proc = lambda {|ks, s, v| !s ? [v] : (s << v) }
  end
  result = {}
  each {|*elts|
    elt = elts.length <= 1 ? elts[0] : elts
    ks = key_selectors.map {|ksel| ksel.call(elt) }
    v = value_selector.call(elt)
    h = result
    0.upto(ks.length-2) {|i|
      k = ks[i]
      h[k] = {} if !h.has_key?(k)
      h = h[k]
    }
    lastk = ks.last
    if !h.has_key?(lastk)
      if has_seed
        h[lastk] = update_proc.call(ks, seed_value, v)
      else
        h[lastk] = v
      end
    else
      h[lastk] = update_proc.call(ks, h[lastk], v)
    end
  }
  if reduce_proc
    tb_cat_reduce(result, [], key_selectors.length-1, reduce_proc)
  end
  result
end

#tb_category_count(*args) ⇒ Object

:call-seq:

enum.tb_category_count(ksel1, ksel2, ...)

counts elements in enum for each category defined by the key selectors.

a = [{:fruit => "banana", :color => "yellow", :taste => "sweet", :price => 100},
     {:fruit => "melon", :color => "green", :taste => "sweet", :price => 300},
     {:fruit => "grapefruit", :color => "yellow", :taste => "tart", :price => 200}]

p a.tb_category_count(:color)
#=> {"yellow"=>2, "green"=>1}

p a.tb_category_count(:taste)
#=> {"sweet"=>2, "tart"=>1}

p a.tb_category_count(:taste, :color)
#=> {"sweet"=>{"yellow"=>1, "green"=>1}, "tart"=>{"yellow"=>1}}

The selectors specify how to extract a value from an element in enum. See Enumerable#tb_categorize for details of selectors.



280
281
282
 
# File 'lib/tb/ex_enumerable.rb', line 280

def tb_category_count(*args)
  tb_categorize(*(args + [lambda {|e| 1 }, {:update => lambda {|ks, s, v| s + v }}]))
end

#tb_unique_categorize(*args, &update_proc) ⇒ Object

:call-seq:

enum.tb_unique_categorize(ksel1, ksel2, ..., vsel, [opts]) -> hash
enum.tb_unique_categorize(ksel1, ksel2, ..., vsel, [opts]) {|s, v| ... } -> hash

categorizes the elements in enum and returns a hash. This method assumes one element for a category by default.

tb_unique_categorize takes one or more key selectors, one value selector and an optional option hash. It also takes an optional block.

The selectors specify how to extract a value from an element in enum. See Enumerable#tb_categorize for details of selectors.

The key selectors, kselN, are used to extract hash keys from an element. If two or more key selectors are specified, the result hash will be nested.

The value selector, vsel, is used for the values of innermost hashes. By default, this method assumes the key selectors categorizes elements in enum uniquely. If the key selectors generates same keys for two or more elements, ArgumentError is raised. This behavior can be customized by :seed option and the block.

a = [{:fruit => "banana", :color => "yellow", :taste => "sweet", :price => 100},
     {:fruit => "melon", :color => "green", :taste => "sweet", :price => 300},
     {:fruit => "grapefruit", :color => "yellow", :taste => "tart", :price => 200}]
p a.tb_unique_categorize(:fruit, :price)
#=> {"banana"=>100, "melon"=>300, "grapefruit"=>200}

p a.tb_unique_categorize(:color, :price)
# ArgumentError

If the block is given, it is used for combining values in a category. The arguments for the block is a seed and the value extracted by vsel. The return value of the block is used as the next seed. :seed option specifies the initial seed. If :seed is not given, the first value for each category is used for the seed.

p a.tb_unique_categorize(:taste, :price) {|s, v| s + v }
#=> {"sweet"=>400, "tart"=>200}

p a.tb_unique_categorize(:color, :price) {|s, v| s + v }
#=> {"yellow"=>300, "green"=>300}



241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
 
# File 'lib/tb/ex_enumerable.rb', line 241

def tb_unique_categorize(*args, &update_proc)
  opts = args.last.kind_of?(Hash) ? args.pop.dup : {}
  if update_proc
    opts[:update] = lambda {|ks, s, v| update_proc.call(s, v) }
  else
    seed = Object.new
    opts[:seed] = seed
    opts[:update] = lambda {|ks, s, v|
      if s.equal? seed
        v
      else
        raise ArgumentError, "ambiguous key: #{ks.map {|k| k.inspect }.join(',')}"
      end
    }
  end
  tb_categorize(*(args + [opts]))
end

#to_fileenumerator ⇒ Object

creates a Tb::FileEnumerator object.



32
33
34
35
36
37
38
 
# File 'lib/tb/fileenumerator.rb', line 32

def to_fileenumerator
  Tb::FileEnumerator.new_tempfile {|gen|
    self.each {|*objs|
      gen.call(*objs)
    }
  }
end