Module: OccamsRecord::Batches

Included in:
Query, RawQuery
Defined in:
lib/occams-record/batches.rb

Overview

Methods for building batch finding methods. It expects “model” and “scope” methods to be present.

Instance Method Summary collapse

Instance Method Details

#find_each(batch_size: 1000, use_transaction: true, append_order_by: nil) {|OccamsRecord::Results::Row| ... } ⇒ Enumerator

Load records in batches of N and yield each record to a block if given. If no block is given, returns an Enumerator.

NOTE Unlike ActiveRecord’s find_each, ORDER BY is respected. The primary key will be appended to the ORDER BY clause to help ensure consistent batches. Additionally, it will be run inside of a transaction.

Parameters:

  • batch_size (Integer) (defaults to: 1000)
  • use_transaction (Boolean) (defaults to: true) —

    Ensure it runs inside of a database transaction

  • append_order_by (String) (defaults to: nil) —

    Append this column to ORDER BY to ensure consistent results. Defaults to the primary key. Pass false to disable.

Yields:

Returns:

  • (Enumerator) —

    will yield each record



20
21
22
23
24
25
26
27
28
29
30
31
 
# File 'lib/occams-record/batches.rb', line 20

def find_each(batch_size: 1000, use_transaction: true, append_order_by: nil)
  enum = Enumerator.new { |y|
    batches(of: batch_size, use_transaction: use_transaction, append_order_by: append_order_by).each { |batch|
      batch.each { |record| y.yield record }
    }
  }
  if block_given?
    enum.each { |record| yield record }
  else
    enum
  end
end

#find_in_batches(batch_size: 1000, use_transaction: true, append_order_by: nil) {|OccamsRecord::Results::Row| ... } ⇒ Enumerator

Load records in batches of N and yield each batch to a block if given. If no block is given, returns an Enumerator.

NOTE Unlike ActiveRecord’s find_each, ORDER BY is respected. The primary key will be appended to the ORDER BY clause to help ensure consistent batches. Additionally, it will be run inside of a transaction.

Parameters:

  • batch_size (Integer) (defaults to: 1000)
  • use_transaction (Boolean) (defaults to: true) —

    Ensure it runs inside of a database transaction

  • append_order_by (String) (defaults to: nil) —

    Append this column to ORDER BY to ensure consistent results. Defaults to the primary key. Pass false to disable.

Yields:

Returns:

  • (Enumerator) —

    will yield each batch



47
48
49
50
51
52
53
54
 
# File 'lib/occams-record/batches.rb', line 47

def find_in_batches(batch_size: 1000, use_transaction: true, append_order_by: nil)
  enum = batches(of: batch_size, use_transaction: use_transaction, append_order_by: append_order_by)
  if block_given?
    enum.each { |batch| yield batch }
  else
    enum
  end
end