Module: AWS::Core::Collection::Simple

Includes:
AWS::Core::Collection, Enumerable
Included in:
ELB::AvailabilityZoneCollection, ELB::BackendServerPolicyCollection, ELB::InstanceCollection, ELB::ListenerCollection, ELB::LoadBalancerCollection, ELB::LoadBalancerPolicyCollection, IAM::GroupUserCollection
Defined in:
lib/aws/core/collection/simple.rb

Overview

AWS::Core::Collection::Simple is used by collections that always recieve every matching items in a single response.

This means:

  • Paging methods are simulated

  • Next tokens are artificial (guessable numeric offsets)

AWS services generally return all items only for requests with a small maximum number of results.

See AWS::Core::Collection for documentation on the available collection methods.

Instance Method Summary collapse

Methods included from AWS::Core::Collection

#each, #enum, #first, #in_groups_of, #page

Instance Method Details

#each_batch(options = {}) {|batch| ... } ⇒ nil_or_next_token

Note:

If you require fixed size batches, see AWS::Core::Collection#in_groups_of.

Yields items from this collection in batches.

collection.each_batch do |batch|
  batch.each do |item|
    # ...
  end
end

Variable Batch Sizes

Each AWS service has its own rules on how it returns results. Because of this batch size may very based on:

  • Service limits (e.g. S3 limits keys to 1000 per response)

  • The size of response objects (SimpleDB limits responses to 1MB)

  • Time to process the request

Because of these variables, batch sizes may not be consistent for a single collection. Each batch represents all of the items returned in a single resopnse.

Parameters:

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :limit (Integer) — default: nil —

    The maximum number of items to enumerate from this collection.

  • :next_token (next_token) — default: nil —

    Next tokens act as offsets into the collection. Next tokens vary in format from one service to the next, (e.g. may be a number, an opaque string, a hash of values, etc).

    AWS::Core::Collection#each and #each_batch return a :next_token when called with :limit and there were more items matching the request.

    NOTE It is generally better to call AWS::Core::Collection#page if you only want a few items with the ability to request more later.

Yields:

  • (batch)

Returns:

  • (nil_or_next_token) —

    Returns nil if all items were enumerated. If some items were excluded because of a :limit option then a next_token is returned. Calling an enumerable method on the same collection with the next_token acts like an offset.



39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/aws/core/collection/simple.rb', line 39

def each_batch options = {}, &block

  each_opts  = options.dup
  limit      = each_opts.delete(:limit)
  next_token = each_opts.delete(:next_token)
  offset     = next_token ? next_token.to_i - 1 : 0
  total      = 0

  nil_or_next_token = nil

  batch = []
  _each_item(each_opts.dup) do |item|

    total += 1

    # skip until we reach our offset (derived from the "next token")
    next if total <= offset

    if limit

      if batch.size < limit
        batch << item
      else
        # allow _each_item to yield one more item than needed
        # so we can determine if we should return a "next token"
        nil_or_next_token = total
        break
      end

    else
      batch << item
    end

  end

  yield(batch)

  nil_or_next_token

end