Module: AWS::Core::Collection

Includes:
Enumerable
Included in:
Batchable, Limitable, Simple
Defined in:
lib/aws/core/collection.rb,
lib/aws/core/collection/simple.rb,
lib/aws/core/collection/batchable.rb,
lib/aws/core/collection/limitable.rb

Overview

Different Collection Types in AWS

The Collection module acts as a namespace and base implementation for the primary collection types in AWS:

Each AWS service allows provides a method to enumerate resources.

  • Services that return all results in a single response use Simple.

  • Services that truncate large results sets AND allow you to provide a perfered maximum number of results use Limitable.

Defined Under Namespace

Modules: Batchable, Limitable, Simple

Instance Method Summary collapse

Instance Method Details

#each(options = {}, &block) ⇒ nil_or_next_token

Note:

If you want fewer than all items, it is generally better to call ##page than #each with a :limit.

Yields once for every item in this collection.

collection.each {|item| ... }

Parameters:

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :limit (Integer) — default: nil —

    The maximum number of items to enumerate from this collection.

  • :next_token (next_token) — default: nil —

    Next tokens act as offsets into the collection. Next tokens vary in format from one service to the next, (e.g. may be a number, an opaque string, a hash of values, etc).

    #each and #each_batch return a :next_token when called with :limit and there were more items matching the request.

    NOTE It is generally better to call #page if you only want a few items with the ability to request more later.

Returns:

  • (nil_or_next_token) —

    Returns nil if all items were enumerated. If some items were excluded because of a :limit option then a next_token is returned. Calling an enumerable method on the same collection with the next_token acts like an offset.



70
71
72
73
74
 
# File 'lib/aws/core/collection.rb', line 70

def each options = {}, &block
  each_batch(options) do |batch|
    batch.each(&block)
  end
end

#each_batch(options = {}, &block) ⇒ nil_or_next_token

Note:

If you require fixed size batches, see #in_groups_of.

Yields items from this collection in batches.

collection.each_batch do |batch|
  batch.each do |item|
    # ...
  end
end

Variable Batch Sizes

Each AWS service has its own rules on how it returns results. Because of this batch size may very based on:

  • Service limits (e.g. S3 limits keys to 1000 per response)

  • The size of response objects (SimpleDB limits responses to 1MB)

  • Time to process the request

Because of these variables, batch sizes may not be consistent for a single collection. Each batch represents all of the items returned in a single resopnse.

Parameters:

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :limit (Integer) — default: nil —

    The maximum number of items to enumerate from this collection.

  • :next_token (next_token) — default: nil —

    Next tokens act as offsets into the collection. Next tokens vary in format from one service to the next, (e.g. may be a number, an opaque string, a hash of values, etc).

    #each and #each_batch return a :next_token when called with :limit and there were more items matching the request.

    NOTE It is generally better to call #page if you only want a few items with the ability to request more later.

Returns:

  • (nil_or_next_token) —

    Returns nil if all items were enumerated. If some items were excluded because of a :limit option then a next_token is returned. Calling an enumerable method on the same collection with the next_token acts like an offset.

Raises:

  • (NotImplementedError) 


107
108
109
 
# File 'lib/aws/core/collection.rb', line 107

def each_batch options = {}, &block
  raise NotImplementedError
end

#enum(options = {}) ⇒ Enumerable::Enumerator Also known as: enumerator

Use this method when you want to call a method provided by Enumerable, but you need to pass options:

# raises an error because collect does not accept arguments
collection.collect(:limit => 10) {|i| i.name }

# not an issue with the enum method
collection.enum(:limit => 10).collect(&:name)

Parameters:

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :limit (Integer) — default: nil —

    The maximum number of items to enumerate from this collection.

  • :next_token (next_token) — default: nil —

    Next tokens act as offsets into the collection. Next tokens vary in format from one service to the next, (e.g. may be a number, an opaque string, a hash of values, etc).

    #each and #each_batch return a :next_token when called with :limit and there were more items matching the request.

    NOTE It is generally better to call #page if you only want a few items with the ability to request more later.

Returns:

  • (Enumerable::Enumerator) —

    Returns an enumerator for this collection.



127
128
129
 
# File 'lib/aws/core/collection.rb', line 127

def enum options = {}
  Enumerator.new(self, :each, options) 
end

#first(options = {}) ⇒ item_or_nil

Returns the first item from this collection.

Returns:

  • (item_or_nil) —

    Returns the first item from this collection or nil if the collection is empty.



137
138
139
 
# File 'lib/aws/core/collection.rb', line 137

def first options = {}
  enum(options.merge(:limit => 1)).first
end

#in_groups_of(size, options = {}) {|group| ... } ⇒ nil_or_next_token

Yields items from this collection in groups of an exact size (except for perhaps the last group). 

collection.in_groups_of (10, :limit => 30) do |group|

  # each group should be exactly 10 items unless
  # fewer than 30 items are returned by the service
  group.each do |item|
    #...
  end

end

Parameters:

  • size (Integer) —

    Size each each group of objects should be yielded in.

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :limit (Integer) — default: nil —

    The maximum number of items to enumerate from this collection.

  • :next_token (next_token) — default: nil —

    Next tokens act as offsets into the collection. Next tokens vary in format from one service to the next, (e.g. may be a number, an opaque string, a hash of values, etc).

    #each and #each_batch return a :next_token when called with :limit and there were more items matching the request.

    NOTE It is generally better to call #page if you only want a few items with the ability to request more later.

Yields:

  • (group)

Returns:

  • (nil_or_next_token) —

    Returns nil if all items were enumerated. If some items were excluded because of a :limit option then a next_token is returned. Calling an enumerable method on the same collection with the next_token acts like an offset.



159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
 
# File 'lib/aws/core/collection.rb', line 159

def in_groups_of size, options = {}, &block

  group = []

  nil_or_next_token = each_batch(options) do |batch|
    batch.each do |item|
      group << item
      if group.size == size
        yield(group)
        group = []
      end
    end
  end

  yield(group) unless group.empty?

  nil_or_next_token

end

#page(options = {}) ⇒ Object

Note:

This method does not accept a :page option, which means you can only start at the begining of the collection and request the next page of results. You can not choose an offset or know how many pages of results there will be.

Returns a single page of results in a kind-of array (PageResult).

items = collection.page(:per_page => 10) # defaults to 10 items
items.is_a?(Array) # => true
items.size         # => 8
items.per_page     # => 10
items.last_page?   # => true

If you need to display a “next page” link in a web view you can use the #more? method. Just make sure the generated link contains the next_token.

<% if items.more? %>
  <%= link_to('Next Page', params.merge(:next_token => items.next_token) %>
<% end %>

Then in your controller you can find the next page of results:

items = collection.page(:next_token => params[:next_token])

Given a PageResult you can also get more results directly:

more_items = items.next_page

Parameters:

  • options (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (options):

  • :per_page (Integer) — default: 10 —

    The number of results to return for each page.

  • :next_token (String) — default: nil —

    A token that indicates an offset to use when paging items. Next tokens are returned by PageResult#next_token.

    Next tokens should only be consumed by the same collection that created them.



221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
 
# File 'lib/aws/core/collection.rb', line 221

def page options = {}
  
  each_opts = options.dup

  per_page = each_opts.delete(:per_page)
  per_page = [nil,''].include?(per_page) ? 10 : per_page.to_i

  each_opts[:limit] = per_page

  items = []
  next_token = each(each_opts) do |item|
    items << item
  end

  Core::PageResult.new(self, items, per_page, next_token)

end