Class: TrickBag::Enumerables::BufferedEnumerable

Inherits:
Object
  • Object
show all
Extended by:
Meta::Classes
Includes:
Enumerable
Defined in:
lib/trick_bag/enumerables/buffered_enumerable.rb

Overview

Provides the plumbing for an enumerator that, in order to serve objects in each(), fetches them in chunks.

This class knows nothing about how to fetch anything; that behavior is provided by either subclassing this class, or calling .create_with_callables and passing a callable that knows how to do that.

Also supported is an optional fetch notification, a method or callable that will be called whenever a fetch is done. This can be useful to update counters, provide user feedback (e.g. a progress bar), etc.

This is useful, for example, in network requests, when multiple requests can be sent one immediately after another, and the responses can be collected as a group, for improved performance.

The fetch method and fetcher callable modify the instance’s data array directly, to avoid the need to allow the callable to modify the data array reference, needlessly copying arrays, and to eliminate the need for garbage collecting many array objects (though the latter is rarely important).

Defined Under Namespace

Classes: NoFetcherError

Constant Summary

Constants included from Meta::Classes

Meta::Classes::VALID_ACCESS_MODES

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Meta::Classes

attr_access, class?, private_attr_accessor, private_attr_reader, private_attr_writer, protected_attr_accessor, protected_attr_reader, protected_attr_writer, undef_class

Constructor Details

#initialize(chunk_size) ⇒ BufferedEnumerable

Returns a new instance of BufferedEnumerable.

Parameters:

  • chunk_size —

    the maximum number of objects to be buffered



63
64
65
66
67
68
69
 
# File 'lib/trick_bag/enumerables/buffered_enumerable.rb', line 63

def initialize(chunk_size)
  @chunk_size = chunk_size
  @data = []
  @chunk_count = 0
  @fetch_count = 0
  @yield_count = 0
end

Instance Attribute Details

#chunk_size ⇒ Object (readonly)

Returns the value of attribute chunk_size.



33
34
35
 
# File 'lib/trick_bag/enumerables/buffered_enumerable.rb', line 33

def chunk_size
  @chunk_size
end

#fetch_notifier ⇒ Object

Returns the value of attribute fetch_notifier.



32
33
34
 
# File 'lib/trick_bag/enumerables/buffered_enumerable.rb', line 32

def fetch_notifier
  @fetch_notifier
end

#fetcher ⇒ Object

Returns the value of attribute fetcher.



32
33
34
 
# File 'lib/trick_bag/enumerables/buffered_enumerable.rb', line 32

def fetcher
  @fetcher
end

Class Method Details

.create_with_callables(chunk_size, fetcher, fetch_notifier = nil) ⇒ Object Also known as: create_with_lambdas

Creates an instance with callables for fetch and fetch notify behaviors. Callables are usually lambdas but can be any object responding to the method name call.

Parameters:

  • chunk_size —

    the maximum number of objects to be buffered

  • fetcher —

    callable to be called to fetch to fill the buffer

  • fetch_notifier (defaults to: nil) —

    callable to be called to when a fetch is done in case the caller wants to receive notification, update counters, etc. It’s passed the array of objects just fetched, whose size may be less than chunk size.



49
50
51
52
53
54
 
# File 'lib/trick_bag/enumerables/buffered_enumerable.rb', line 49

def self.create_with_callables(chunk_size, fetcher, fetch_notifier = nil)
  instance = self.new(chunk_size)
  instance.fetcher = fetcher
  instance.fetch_notifier = fetch_notifier
  instance
end

Instance Method Details

#each ⇒ Object

Enumerable.each method. Note that, like all Enumerable.each methods, if you pass it without a block, it will return an enumerator, and any cleanup that would normally be done after each’s loop has completed will not happen.



94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
 
# File 'lib/trick_bag/enumerables/buffered_enumerable.rb', line 94

def each
  return to_enum unless block_given?

  last_chunk = false
  loop do
    if data.empty?
      return if last_chunk
      fetch
      @chunk_count += 1
      return if data.empty?
      self.fetch_count = self.fetch_count + data.size
      last_chunk = true if data.size < chunk_size
      fetch_notify
    end

    self.yield_count = self.yield_count + 1
    object_to_yield = data.shift
    yield(object_to_yield)
  end
end

#fetch ⇒ Object

Unless you use self.create_with_callables to create your instance, you’ll need to override this method in your subclass.



74
75
76
77
78
79
80
 
# File 'lib/trick_bag/enumerables/buffered_enumerable.rb', line 74

def fetch
  if fetcher
    fetcher.(data, chunk_size)
  else
    raise NoFetcherError.new
  end
end

#fetch_notify ⇒ Object

Unless you use self.create_with_callables to create your instance, you’ll need to override this method in your subclass.



85
86
87
 
# File 'lib/trick_bag/enumerables/buffered_enumerable.rb', line 85

def fetch_notify
  fetch_notifier.(data) if fetch_notifier
end