Class: DaimonSkycrawlers::Storage::RDB

Inherits:
Base
  • Object
show all
Defined in:
lib/daimon_skycrawlers/storage/rdb.rb

Overview

Storage for RDBMS

Defined Under Namespace

Classes: Page

Instance Method Summary collapse

Constructor Details

#initialize(config_path = "config/database.yml") ⇒ RDB

Returns a new instance of RDB.



10
11
12
13
14
 
# File 'lib/daimon_skycrawlers/storage/rdb.rb', line 10

def initialize(config_path = "config/database.yml")
  super()
  Base.configurations = YAML.load(ERB.new(::File.read(config_path)).result)
  Base.establish_connection(DaimonSkycrawlers.env.to_sym)
end

Instance Method Details

#read(message = {}) ⇒ Object

Fetch page identified by url

Parameters:

  • message (Hash) (defaults to: {}) —

    this hash can include :url, :key to find page



44
45
46
47
48
49
50
51
52
 
# File 'lib/daimon_skycrawlers/storage/rdb.rb', line 44

def read(message = {})
  url = message[:url]
  key = message[:key]
  if key
    Page.where(key: key).order(updated_at: :desc).limit(1).first
  else
    Page.where(url: url).order(updated_at: :desc).limit(1).first
  end
end

#save(data) ⇒ Object

Save data to RDB

Parameters:

  • data (Hash) —

    data has following keys

    • :url: URL
    • :message: Given message
    • :response: HTTP response


24
25
26
27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/daimon_skycrawlers/storage/rdb.rb', line 24

def save(data)
  url = data[:url]
  message = data[:message]
  key = message[:key] || url
  response = data[:response]
  headers = response.headers
  body = response.body
  Page.create(url: url,
              key: key,
              headers: JSON.generate(headers),
              body: body,
              last_modified_at: headers["last-modified"],
              etag: headers["etag"])
end