Module: FulfilApi::Relation::Batchable

Included in:
FulfilApi::Relation
Defined in:
lib/fulfil_api/relation/batchable.rb

Overview

The Batchable module includes a set of

helper/query methods that queries resources in Fulfil in batches.

Instance Method Summary collapse

Instance Method Details

#find_each(batch_size: 500) {|FulfilApi::Resource| ... } ⇒ FulfilApi::Relation

The #find_each is a shorthand for iterating over individual API resources

in a memory effective way.

Under the hood, it uses the #in_batches to find API resources in batches

and process them efficiently.

Examples:

find all resources


FulfilApi::Resource.set(model_name: "sale.sale").find_each do |sales_order|
  process_sales_order(sales_order)
end

Parameters:

  • batch_size (Integer) (defaults to: 500) —

    The default batch forwarded to the #in_batches method.

Yields:

Returns:



23
24
25
26
27
 
# File 'lib/fulfil_api/relation/batchable.rb', line 23

def find_each(batch_size: 500, &block)
  in_batches(of: batch_size) do |batch|
    batch.each(&block)
  end
end

#in_batches(of: 500) {|FulfilApi::Relation| ... } ⇒ FulfilApi::Relation

Note:

the #in_batches automatically retries when it encounters a 429 (TooManyRequests) HTTP error to ensure the lookup can be completed.

Finds API resources in batches. Defaults to the maximum number of resources

Fulfil's API endpoints will return (500 resources).

Examples:

find resources in batches of 10.


FulfilApi::Resource.set(model_name: "sale.sale").in_batches(of: 10) do |batch|
  batch.each do |sales_order|
    process_sales_order(sales_order)
  end
end

Parameters:

  • of (Integer) (defaults to: 500) —

    The maximum number of resources in a batch.

Yields:

  • (FulfilApi::Relation) —

    Yields FulfilApi::Relation objects to work with a batch of records.

Returns:



47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/fulfil_api/relation/batchable.rb', line 47

def in_batches(of: 500) # rubocop:disable Metrics/MethodLength
  current_offset = request_offset.presence || 0
  batch_size = of

  loop do
    batch_relation = dup.offset(current_offset * batch_size).limit(batch_size)
    batch_relation.load

    yield(batch_relation)

    break unless batch_relation.size == batch_size

    current_offset += 1
  rescue FulfilApi::Error => e
    retry if e.details[:response_status] == 429
  end

  self
end