Class: JobIteration::CsvEnumerator

Inherits:
Object
  • Object
show all
Defined in:
lib/job-iteration/csv_enumerator.rb

Overview

CsvEnumerator makes it possible to write an Iteration job that uses CSV file as a collection to Iterate.

Examples:

def build_enumerator(cursor:)
  csv = CSV.open('tmp/files', { converters: :integer, headers: true })
  JobIteration::CsvEnumerator.new(csv).rows(cursor: cursor)
end

def each_iteration(row)
  ...
end

Instance Method Summary collapse

Constructor Details

#initialize(csv) ⇒ JobIteration::CsvEnumerator

Constructs CsvEnumerator instance based on a CSV file.

Examples:

csv = CSV.open('tmp/files', { converters: :integer, headers: true })
JobIteration::CsvEnumerator.new(csv).rows(cursor: cursor)

Parameters:

  • csv (CSV) —

    An instance of CSV object



22
23
24
25
26
27
28
 
# File 'lib/job-iteration/csv_enumerator.rb', line 22

def initialize(csv)
  unless defined?(CSV) && csv.instance_of?(CSV)
    raise ArgumentError, "CsvEnumerator.new takes CSV object"
  end

  @csv = csv
end

Instance Method Details

#batches(batch_size:, cursor:) ⇒ Enumerator

Constructs a enumerator on batches of CSV rows

Returns:

  • (Enumerator) —

    Enumerator instance



41
42
43
44
45
46
47
 
# File 'lib/job-iteration/csv_enumerator.rb', line 41

def batches(batch_size:, cursor:)
  @csv.lazy
    .each_slice(batch_size)
    .with_index
    .drop(count_of_processed_rows(cursor))
    .to_enum { (count_of_rows_in_file.to_f / batch_size).ceil }
end

#rows(cursor:) ⇒ Enumerator

Constructs a enumerator on CSV rows

Returns:

  • (Enumerator) —

    Enumerator instance



32
33
34
35
36
37
 
# File 'lib/job-iteration/csv_enumerator.rb', line 32

def rows(cursor:)
  @csv.lazy
    .each_with_index
    .drop(count_of_processed_rows(cursor))
    .to_enum { count_of_rows_in_file }
end