Class: Importu::Sources::JSON

Inherits:
Object
  • Object
show all
Defined in:
lib/importu/sources/json.rb

Overview

Note:

The entire JSON file is loaded into memory. For very large files, consider using CSV or a streaming JSON parser.

Parses JSON files as import source data.

The JSON must have an array as the root element. Each array element becomes a row. The entire file is loaded into memory.

Examples:

Basic usage

source = Importu::Sources::JSON.new("data.json")
source.rows.each { |row| puts row["name"] }

Expected JSON format

# data.json
[
  { "name": "Alice", "email": "[email protected]" },
  { "name": "Bob", "email": "[email protected]" }
]

From a string

json_data = '[{"name": "Alice"}, {"name": "Bob"}]'
source = Importu::Sources::JSON.new(StringIO.new(json_data))

Instance Method Summary collapse

Constructor Details

#initialize(infile) ⇒ JSON

Creates a new JSON source.

Parameters:

  • infile (String, IO) —

    file path or IO object to read from

Raises:



37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
 
# File 'lib/importu/sources/json.rb', line 37

def initialize(infile, **)
  owns_handle = !infile.respond_to?(:readline)
  @infile = owns_handle ? File.open(infile, "rb") : infile

  begin
    @infile.rewind
    @reader = ::JSON.parse(@infile.read)
  rescue ::JSON::ParserError => e
    raise Importu::InvalidInput, e.message
  ensure
    # JSON loads entire content into memory, so we can close immediately
    @infile.close if owns_handle && @infile && !@infile.closed?
  end

  if @reader.nil?
    raise Importu::InvalidInput, "Empty document"
  end
end

Instance Method Details

#close ⇒ void

This method returns an undefined value.

Closes any resources held by this source.

For JSON sources, the file is already closed after initialization since the entire content is loaded into memory. This method is provided for API consistency with other sources.



63
64
65
 
# File 'lib/importu/sources/json.rb', line 63

def close
  # JSON source closes file immediately after reading into memory
end

#rows ⇒ Enumerator<Hash>

Returns an enumerator that yields each element as a hash.

Returns:

  • (Enumerator<Hash>) —

    rows from the JSON array



70
71
72
73
74
 
# File 'lib/importu/sources/json.rb', line 70

def rows
  Enumerator.new do |yielder|
    @reader.each {|row| yielder.yield(row) }
  end
end

#write_errors(summary, only_errors: false) ⇒ Tempfile?

Generates a JSON file with error information appended.

Creates a copy of the original data with an “_errors” key containing any validation errors for each row.

Parameters:

  • summary (Importu::Summary) —

    the import summary containing errors

  • only_errors (Boolean) (defaults to: false) —

    if true, only include rows that had errors

Returns:

  • (Tempfile, nil) —

    temp file with error data, or nil if no errors



84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
 
# File 'lib/importu/sources/json.rb', line 84

def write_errors(summary, only_errors: false)
  return unless summary.itemized_errors.any?

  itemized_errors = summary.itemized_errors
  updated_rows = rows.each.with_index.with_object([]) do |(row, index), acc|
    if itemized_errors.key?(index)
      acc << row.merge("_errors" => itemized_errors[index].join(", "))
    elsif only_errors
      # Requested to only include rows with new errors, row has none
    elsif row.key?("_errors")
      acc << row.dup.tap {|r| r.delete("_errors") }
    else
      acc << row
    end
  end

  Tempfile.new("import").tap do |file|
    file.write(JSON.pretty_generate(updated_rows))
    file.rewind
  end
end