Class: Grabbers::GenericHttp

Inherits:
Object
  • Object
show all
Includes:
EM::Protocols
Defined in:
lib/octopus/grabbers/generic_http.rb

Instance Method Summary collapse

Constructor Details

#initialize ⇒ GenericHttp

Adds a periodic timer to the Eventmachine reactor loop and immediately starts grabbing expired resources and checking them.



9
10
11
12
13
14
 
# File 'lib/octopus/grabbers/generic_http.rb', line 9

def initialize
  EM.add_periodic_timer(5) {
    check_expired_resources
  }
  check_expired_resources
end

Instance Method Details

#check_expired_resources ⇒ Object

Gets all of the expired NetResources from the database and sends an HTTP GET requests for each one. Subscribers to a NetResource will be notified if it has changed since the last time it was grabbed.



20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/octopus/grabbers/generic_http.rb', line 20

def check_expired_resources
  net_resources = ::NetResource.expired
  net_resources.each do |resource|
    http = EM::HttpRequest.new(resource.url).get

    http.callback{ |response|
      resource.set_next_update
      if resource_changed?(resource, response)
        resource.body = response.response
        update_changed_resource(resource, response)
        notify_subscribers(resource)
      end
    }
    http.errback {|response|
      # Do something here, maybe setting the resource
      # to be not checked anymore.
    }
  end
end

#notify_subscribers(resource) ⇒ Object

Notifies each of a NetResource’s subscribers that the resource has changed by doing an HTTP POST request to the subscriber’s callback url.

The POST body contains a key called “data” which contains the feed value.



45
46
47
48
49
50
51
52
53
54
55
56
 
# File 'lib/octopus/grabbers/generic_http.rb', line 45

def notify_subscribers(resource)
  resource.subscriptions.each do |subscription|
    http = EM::HttpRequest.new(subscription.url).post(:body => {:data => resource.body})
    http.callback{ |response|
      puts "POSTed updated data for #{resource.url}, #{resource.body.length} characters"
    }
    http.errback {|response|
      # Do something here, maybe setting the resource
      # to be not checked anymore.
    }
  end
end

#resource_changed?(resource, response) ⇒ Boolean

Determines whether a resource has changed by comparing its saved hash value with the hash value of the response content.

Returns:

  • (Boolean) 


61
62
63
64
65
66
67
68
69
70
 
# File 'lib/octopus/grabbers/generic_http.rb', line 61

def resource_changed?(resource, response)
  changed = false
  puts "checking for changes on #{resource.url}"
  puts "response.response.hash: #{response.response.hash}"
  puts "resource.last_modified_hash: #{resource.last_modified_hash}"
  if response.response.hash != resource.last_modified_hash
    puts "changed!!!!\n\n\n\n"
    changed = true
  end
end

#update_changed_resource(resource, response) ⇒ Object

Updates the resource’s fields when the resource has changed. The last_modified_hash is set to the hash value of the response body, the response body itself is saved so that it can be sent to consumers during notifications, and the resource’s last_updated time is set to the current time.



78
79
80
81
82
83
 
# File 'lib/octopus/grabbers/generic_http.rb', line 78

def update_changed_resource(resource, response)
  resource.last_modified_hash = response.response.hash
  resource.last_updated = Time.now
  resource.body = response.response
  resource.save
end