Class: Fech::Filing

Inherits:
Object
  • Object
show all
Defined in:
lib/fech/filing.rb

Overview

Fech::Filing downloads an Electronic Filing given its ID, and will search rows by row type. Using a child Translator object, the data in each row is automatically mapped at runtime into a labeled Hash. Additional Translations may be added to change the way that data is mapped and cleaned.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(filing_id, opts = {}) ⇒ Filing

Create a new Filing object, assign the download directory to system’s temp folder by default.

Parameters:

  • download_dir (String) —

    override the directory where files should be downloaded.

  • translate (Symbol, Array) —

    a list of built-in translation sets to use



18
19
20
21
22
 
# File 'lib/fech/filing.rb', line 18

def initialize(filing_id, opts={})
  @filing_id    = filing_id
  @download_dir = opts[:download_dir] || Dir.tmpdir
  @translator   = Fech::Translator.new(:include => opts[:translate])
end

Instance Attribute Details

#download_dir ⇒ Object

Returns the value of attribute download_dir.



11
12
13
 
# File 'lib/fech/filing.rb', line 11

def download_dir
  @download_dir
end

#filing_id ⇒ Object

Returns the value of attribute filing_id.



11
12
13
 
# File 'lib/fech/filing.rb', line 11

def filing_id
  @filing_id
end

#translator ⇒ Object

Returns the value of attribute translator.



11
12
13
 
# File 'lib/fech/filing.rb', line 11

def translator
  @translator
end

Class Method Details

.map_for(row_type, opts = {}) ⇒ Object

Returns the column names for given row type and version in the order they appear in row data.

Parameters:

  • row_type (String, Regexp) —

    representation of the row desired

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :version (String, Regexp) —

    representation of the version desired



151
152
153
 
# File 'lib/fech/filing.rb', line 151

def self.map_for(row_type, opts={})
  Fech::Mappings.for_row(row_type, opts)
end

Instance Method Details

#amendment? ⇒ Boolean

Whether this filing amends a previous filing or not.

Returns:

  • (Boolean) 


166
167
168
 
# File 'lib/fech/filing.rb', line 166

def amendment?
  !amends.nil?
end

#amends ⇒ Object

Returns the filing ID of the past filing this one amends, nil if this is a first-draft filing. :report_id in the HDR line references the amended filing



173
174
175
 
# File 'lib/fech/filing.rb', line 173

def amends
  header[:report_id]
end

#delimiter ⇒ String

Returns the delimiter used in the filing’s version.

Returns:

  • (String) —

    the delimiter used in the filing’s version



246
247
248
 
# File 'lib/fech/filing.rb', line 246

def delimiter
  filing_version.to_f < 6 ? "," : "\034"
end

#download ⇒ Object

Saves the filing data from the FEC website into the default download directory.



26
27
28
29
30
31
 
# File 'lib/fech/filing.rb', line 26

def download
  File.open(file_path, 'w') do |file|
    file << open(filing_url).read
  end
  self
end

#each_row(opts = {}) {|Array| ... } ⇒ Object

Iterates over and yields the Filing’s lines

Parameters:

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :with_index (Boolean) —

    yield both the item and its index

Yields:

  • (Array) —

    a row of the filing, split by the delimiter from #delimiter



225
226
227
228
229
230
231
232
233
234
235
236
237
238
 
# File 'lib/fech/filing.rb', line 225

def each_row(opts={}, &block)
  unless File.exists?(file_path)
    raise "File #{file_path} does not exist. Try invoking the .download method on this Filing object."
  end
  c = 0
  Fech::Csv.foreach(file_path, :col_sep => delimiter, :skip_blanks => true) do |row|
    if opts[:with_index]
      yield [row, c]
      c += 1
    else
      yield row
    end
  end
end

#each_row_with_index(&block) ⇒ Object

Wrapper around .each_row to include indexes



241
242
243
 
# File 'lib/fech/filing.rb', line 241

def each_row_with_index(&block)
  each_row(:with_index => true, &block)
end

#file_name ⇒ Object 



214
215
216
 
# File 'lib/fech/filing.rb', line 214

def file_name
  "#{filing_id}.fec"
end

#file_path ⇒ Object

The location of the Filing on the file system



210
211
212
 
# File 'lib/fech/filing.rb', line 210

def file_path
  File.join(download_dir, file_name)
end

#filing_url ⇒ Object 



218
219
220
 
# File 'lib/fech/filing.rb', line 218

def filing_url
  "http://query.nictusa.com/dcdev/posted/#{filing_id}.fec"
end

#filing_version ⇒ Object

The version of the FEC software used to generate this Filing



188
189
190
 
# File 'lib/fech/filing.rb', line 188

def filing_version
  @filing_version ||= parse_filing_version
end

#hash_zip(keys, values) ⇒ Fech::Mapped, Hash

Combines an array of keys and values into an Fech::Mapped object, a type of Hash.

Parameters:

  • keys (Array) —

    the desired keys for the new hash

  • values (Array) —

    the desired values for the new hash

Returns:



183
184
185
 
# File 'lib/fech/filing.rb', line 183

def hash_zip(keys, values)
  Fech::Mapped.new(self, values.first).merge(Hash[*keys.zip(values).flatten])
end

#header(opts = {}) ⇒ Hash

Access the header (first) line of the filing, containing information about the filing’s version and metadata about the software used to file it.

Returns:

  • (Hash) —

    a hash that assigns labels to the values of the filing’s header row



36
37
38
39
40
 
# File 'lib/fech/filing.rb', line 36

def header(opts={})
  each_row do |row|
    return parse_row?(row)
  end
end

#map(row, opts = {}) ⇒ Object

Maps a raw row to a labeled hash following any rules given in the filing’s Translator based on its version and row type. Finds the correct map for a given row, performs any matching Translations on the individual values, and returns either the entire dataset, or just those fields requested.

Parameters:

  • row (String, Regexp) —

    a partial or complete name of the type of row desired

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :include (Array) —

    list of field names that should be included in the returned hash



102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
 
# File 'lib/fech/filing.rb', line 102

def map(row, opts={})
  data = Fech::Mapped.new(self, row.first)
  full_row_map = map_for(row.first)
  
  # If specific fields were asked for, return only those
  if opts[:include]
    row_map = full_row_map.select { |k| opts[:include].include?(k) }
  else
    row_map = full_row_map
  end
  
  # Inserts the row into data, performing any specified preprocessing
  # on individual cells along the way
  row_map.each_with_index do |field, index|
    value = row[full_row_map.index(field)]
    translator.get_translations(:row => row.first,
        :version => filing_version, :action => :convert,
        :field => field).each do |translation|
      # User's Procs should be given each field's value as context
      value = translation[:proc].call(value)
    end
    data[field] = value
  end
  
  # Performs any specified group preprocessing / combinations
  combinations = translator.get_translations(:row => row.first,
        :version => filing_version, :action => :combine)
  row_hash = hash_zip(row_map, row) if combinations
  combinations.each do |translation|
    # User's Procs should be given the entire row as context
    value = translation[:proc].call(row_hash)
    field = translation[:field].source.gsub(/[\^\$]*/, "").to_sym
    data[field] = value
  end
  
  data
end

#map_for(row_type) ⇒ Object

Returns the column names for given row type and the filing’s version in the order they appear in row data.

Parameters:

  • row_type (String, Regexp) —

    representation of the row desired



143
144
145
 
# File 'lib/fech/filing.rb', line 143

def map_for(row_type)
  mappings.for_row(row_type)
end

#mappings ⇒ Object

Gets or creats the Mappings instance for this filing_version



205
206
207
 
# File 'lib/fech/filing.rb', line 205

def mappings
  @mapping ||= Fech::Mappings.new(filing_version)
end

#parse_filing_version ⇒ Object

Pulls out the version number from the header line. Must parse this line manually, since we don’t know the version yet, and thus the delimiter type is still a mystery.



195
196
197
198
199
200
201
202
 
# File 'lib/fech/filing.rb', line 195

def parse_filing_version
  first = File.open(file_path).first
  if first.index("\034").nil?
    Fech::Csv.parse(first).flatten[2]
  else
    Fech::Csv.parse(first, :col_sep => "\034").flatten[2]
  end
end

#parse_row?(row, opts = {}) ⇒ Boolean

Decides what to do with a given row. If the row’s type matches the desired type, or if no type was specified, it will run the row through #map. If :raw was passed true, a flat, unmapped data array will be returned.

Parameters:

  • row (String, Regexp) —

    a partial or complete name of the type of row desired

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :include (Array) —

    list of field names that should be included in the returned hash

Returns:

  • (Boolean) 


84
85
86
87
88
89
90
91
92
 
# File 'lib/fech/filing.rb', line 84

def parse_row?(row, opts={})
  # Always parse, unless :parse_if is given and does not match row
  if opts[:parse_if].nil? || \
      Fech.regexify(opts[:parse_if]).match(row.first.downcase)
    opts[:raw] ? row : map(row, opts)
  else
    false
  end
end

#rows_like(row_type, opts = {}) {|Hash| ... } ⇒ Array

Access all lines of the filing that match a given row type. Will return an Array of all available lines if called directly, or will yield the mapped rows one by one if a block is passed.

Parameters:

  • row_type (String, Regexp) —

    a partial or complete name of the type of row desired

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :raw (Boolean) —

    should the function return the data as an array that has not been mapped to column names

  • :include (Array) —

    list of field names that should be included in the returned hash

Yields:

  • (Hash) —

    each matched row’s data, as either a mapped hash or raw array

Returns:

  • (Array) —

    the complete set of mapped hashes for matched lines



63
64
65
66
67
68
69
70
71
72
73
74
75
 
# File 'lib/fech/filing.rb', line 63

def rows_like(row_type, opts={}, &block)
  data = []
  each_row do |row|
    value = parse_row?(row, opts.merge(:parse_if => row_type))
    next if value == false
    if block_given?
      yield value
    else
      data << value if value
    end
  end
  block_given? ? nil : data
end

#summary ⇒ Hash

Access the summary (second) line of the filing, containing aggregate and top-level information about the filing.

Returns:

  • (Hash) —

    a hash that assigns labels to the values of the filing’s summary row



45
46
47
48
49
50
 
# File 'lib/fech/filing.rb', line 45

def summary
  each_row_with_index do |row, index|
    next if index == 0
    return parse_row?(row)
  end
end

#translate {|t| ... } ⇒ Object

Yields:

  • (t) —

    returns a reference to the filing’s Translator

Yield Parameters:



157
158
159
160
161
162
163
 
# File 'lib/fech/filing.rb', line 157

def translate(&block)
  if block_given?
    yield translator
  else
    translator
  end
end