Class: Remi::DataSource

Inherits:
DataSubject show all
Includes:
Testing::DataStub
Defined in:
lib/remi/data_subject.rb,
lib/remi/cucumber.rb

Overview

The DataSource is a DataSubject meant to extract data from an external source and convert (parse) it into a dataframe.

Examples:


my_data_source = DataSource.new do
  extractor some_extractor
  parser some_parser
end

my_data_source.df #=> Returns a dataframe that is created by extracting data
                  #   from some_extractor and parsing it using some_parser.

Instance Attribute Summary

Attributes inherited from DataSubject

#context, #name

Instance Method Summary collapse

Methods included from Testing::DataStub

#empty_stub_df, #stub_boolean, #stub_date, #stub_datetime, #stub_decimal, #stub_df, #stub_float, #stub_integer, #stub_json, #stub_row_array, #stub_string, #stub_values

Methods inherited from DataSubject

#df=, #df_type, #dsl_eval, #dsl_eval!, #enforce_types, #field_symbolizer, #fields, #fields=

Constructor Details

#initialize(*args, **kargs, &block) ⇒ DataSource

Returns a new instance of DataSource.



151
152
153
154
155
 
# File 'lib/remi/data_subject.rb', line 151

def initialize(*args, **kargs, &block)
  @parser = Parser::None.new
  @parser.context = self
  super
end

Instance Method Details

#df ⇒ Remi::DataFrame

The dataframe will only be extracted and parsed once, and only if it has not already been set (e.g., using #df=).

Returns:



193
194
195
 
# File 'lib/remi/data_subject.rb', line 193

def df
  @dataframe ||= parsed_as_dataframe
end

#extract ⇒ Array<Object>

Returns all of the data extracted from the extractors (memoized).

Returns:

  • (Array<Object>) —

    all of the data extracted from the extractors (memoized).



208
209
210
 
# File 'lib/remi/data_subject.rb', line 208

def extract
  @extract ||= extract!
end

#extract! ⇒ Array

Extracts data from all of the extractors.

Returns:

  • (Array) —

    the result of each extractor



179
180
181
 
# File 'lib/remi/data_subject.rb', line 179

def extract!
  extractors.map { |e| e.extract }
end

#extractor(obj) ⇒ Array

Returns the full list of extractors.

Parameters:

  • obj (Object) —

    adds an extractor object to the list of extractors

Returns:

  • (Array) —

    the full list of extractors



164
165
166
 
# File 'lib/remi/data_subject.rb', line 164

def extractor(obj)
  extractors << obj unless extractors.include? obj
end

#extractors ⇒ Array

Returns the list of extractors that are defined for this data source.

Returns:

  • (Array) —

    the list of extractors that are defined for this data source



158
159
160
 
# File 'lib/remi/data_subject.rb', line 158

def extractors
  @extractors ||= []
end

#parse ⇒ Remi::DataFrame

Converts all of the extracted data to a dataframe

Returns:



185
186
187
 
# File 'lib/remi/data_subject.rb', line 185

def parse
  parser.parse *extract
end

#parser(obj = nil) ⇒ Object

Returns the parser set for this data source.

Parameters:

  • obj (Object) (defaults to: nil) —

    sets the parser for this data source

Returns:

  • (Object) —

    the parser set for this data source



170
171
172
173
174
175
 
# File 'lib/remi/data_subject.rb', line 170

def parser(obj = nil)
  return @parser unless obj
  obj.context = self

  @parser = obj
end

#reset ⇒ Remi::DataFrame

This clears any previously extracted and parsed results. A subsequent call to #df will redo the extract and parse.

Returns:



201
202
203
204
205
 
# File 'lib/remi/data_subject.rb', line 201

def reset
  @block = nil
  @dataframe = nil
  @extract = nil
end