Class: DaimonSkycrawlers::Processor::Base

Inherits:
Object
  • Object
show all
Includes:
Callbacks, ConfigMixin, Configurable, LoggerMixin
Defined in:
lib/daimon_skycrawlers/processor/base.rb

Overview

The base class of processor

A processor implementation can inherit this class and override #call in the class.

Direct Known Subclasses

Default, Proc, Spider

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Configurable

#configure

Methods included from Callbacks

#after_process, #before_process, #clear_after_process_callbacks, #clear_before_process_callbacks, #run_after_process_callbacks, #run_before_process_callbacks

Constructor Details

#initialize ⇒ Base

Returns a new instance of Base.



25
26
27
28
29
30
 
# File 'lib/daimon_skycrawlers/processor/base.rb', line 25

def initialize
  super
  @skipped = false

  setup_default_filters
end

Instance Attribute Details

#storage ⇒ Object

Retrieve storage instance



69
70
71
 
# File 'lib/daimon_skycrawlers/processor/base.rb', line 69

def storage
  @storage ||= DaimonSkycrawlers::Storage::RDB.new
end

Instance Method Details

#call(message) ⇒ Object

Process message

Override this method in subclass

Parameters:

  • message (Hash) —

    parameters for processor



62
63
64
 
# File 'lib/daimon_skycrawlers/processor/base.rb', line 62

def call(message)
  raise "Implement this method in subclass"
end

#process(message) ⇒ Object

Process processor sequence

  1. Run registered filters
  2. Process HTTP response from message

Parameters:

  • message (Hash) —

    parameters for processor



45
46
47
48
49
50
51
52
53
 
# File 'lib/daimon_skycrawlers/processor/base.rb', line 45

def process(message)
  @skipped = false
  proceeding = run_before_process_callbacks(message)
  unless proceeding
    skip(message[:url])
    return
  end
  call(message)
end