Class: NdrImport::Table

Inherits:
Object
  • Object
show all
Includes:
Mapper
Defined in:
lib/ndr_import/table.rb

Overview

This class maintains the state of a table mapping and encapsulates the logic required to transform a table of data into “records”. Particular attention has been made to use enumerables throughout to help with the transformation of large quantities of data. rubocop:disable Metrics/ClassLength

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Table

Returns a new instance of Table.



26
27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/ndr_import/table.rb', line 26

def initialize(options = {})
  options.stringify_keys! if options.is_a?(Hash)
  validate_options(options)

  all_valid_options.each do |key|
    # This pattern is used to only set attributes if option specified,
    # which makes for more concise YAML serialization.
    options[key] && instance_variable_set("@#{key}", options[key])
  end

  @row_index = 0
end

Instance Attribute Details

#notifier ⇒ Object

Returns the value of attribute notifier.



24
25
26
 
# File 'lib/ndr_import/table.rb', line 24

def notifier
  @notifier
end

#table_metadata ⇒ Object

Returns the value of attribute table_metadata.



24
25
26
 
# File 'lib/ndr_import/table.rb', line 24

def table_metadata
  @table_metadata
end

Class Method Details

.all_valid_options ⇒ Object 



13
14
15
16
17
 
# File 'lib/ndr_import/table.rb', line 13

def self.all_valid_options
  %w[canonical_name delimiter liberal_parsing filename_pattern file_password last_data_column
     tablename_pattern header_lines footer_lines format klass columns slurp row_identifier
     significant_mapped_fields]
end

Instance Method Details

#all_valid_options ⇒ Object 



19
20
21
 
# File 'lib/ndr_import/table.rb', line 19

def all_valid_options
  self.class.all_valid_options
end

#encode_with(coder) ⇒ Object

For readability, we should serialise the columns last



117
118
119
120
121
122
123
124
 
# File 'lib/ndr_import/table.rb', line 117

def encode_with(coder)
  options = self.class.all_valid_options - ['columns']
  options.each do |option|
    value = send(option)
    coder[option] = value if value
  end
  coder['columns'] = @columns
end

#header_valid? ⇒ Boolean

Returns:

  • (Boolean) 


112
113
114
 
# File 'lib/ndr_import/table.rb', line 112

def header_valid?
  @header_valid == true
end

#match(filename, tablename) ⇒ Object 



39
40
41
42
 
# File 'lib/ndr_import/table.rb', line 39

def match(filename, tablename)
  ::File.basename(filename) =~ (filename_pattern || /\A.*\z/) &&
    (tablename.nil? || tablename =~ (tablename_pattern || /\A.*\z/))
end

#mutate_regexp_columns(line) ⇒ Object

Update ‘column’ values expressed as a regular expression



84
85
86
87
88
89
90
 
# File 'lib/ndr_import/table.rb', line 84

def mutate_regexp_columns(line)
  @columns.each_with_index do |column, index|
    next unless column['column'].is_a? Regexp

    column['column'] = line[index] if line[index].match? column['column']
  end
end

#process_line(line, &block) ⇒ Object

This method process a line of data, If it is a header line it validates it, otherwise transforms it. It also increments the row index and notifies the amount of lines processed.



65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/ndr_import/table.rb', line 65

def process_line(line, &block)
  return enum_for(:process_line, line) unless block

  if @row_index < header_lines
    mutate_regexp_columns(line)
    consume_header_line(line, @columns)
  else
    transform_line(line, @row_index, &block)
  end

  @row_index += 1

  # We've now seen enough lines to have consumed a valid header; is this the case?
  fail_unless_header_complete(@columns) if @row_index == header_lines

  @notifier.try(:processed, @row_index)
end

#transform(lines, &block) ⇒ Object

This method transforms a table of data, given a line array/enumerator and yields klass, fields and index (input row number) for each record that it would create as a result of the transformation process.



47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
 
# File 'lib/ndr_import/table.rb', line 47

def transform(lines, &block)
  return enum_for(:transform, lines) unless block

  @row_index = 0
  @header_valid = false
  @header_best_guess = nil
  @notifier.try(:started)

  last_col = last_column_to_transform
  skip_footer_lines(lines, footer_lines).each do |line|
    line.is_a?(Array) ? process_line(line[0..last_col], &block) : process_line(line, &block)
  end

  @notifier.try(:finished)
end

#transform_line(line, index) ⇒ Object

This method transforms an incoming line of data by applying each of the klass masked mappings to the line and yielding the klass and fields for each mapped klass.



94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/ndr_import/table.rb', line 94

def transform_line(line, index)
  return enum_for(:transform_line, line, index) unless block_given?

  identifier = case @row_identifier.to_s
               when 'index'
                 index
               when 'uuid'
                 SecureRandom.uuid
               end

  masked_mappings.each do |klass, klass_mappings|
    fields = mapped_line(line, klass_mappings)
    fields['row_identifier'] = identifier unless identifier.nil?
    next if fields[:skip].to_s == 'true'
    yield(klass, fields, index)
  end
end