Class: SiteDiff::Fetch

Inherits:
Object
  • Object
show all
Defined in:
lib/sitediff/fetch.rb

Instance Method Summary collapse

Constructor Details

#initialize(cache, paths, interval, concurrency = 3, curl_opts = nil, debug = true, **tags) ⇒ Fetch

Cache is a cache object, see sitediff/cache Paths is a list of sub-paths Tags is a hash of tag names => base URLs.



11
12
13
14
15
16
17
18
19
20
 
# File 'lib/sitediff/fetch.rb', line 11

def initialize(cache, paths, interval, concurrency = 3, curl_opts = nil,
               debug = true, **tags)
  @cache = cache
  @interval = interval
  @paths = paths
  @tags = tags
  @curl_opts = curl_opts || UriWrapper::DEFAULT_CURL_OPTS
  @concurrency = concurrency
  @debug = debug
end

Instance Method Details

#run(&block) ⇒ Object

Fetch all the paths, once per tag. When a path has been fetched for every tag, block will be called with the path, and a hash of tag => UriWrapper::ReadResult objects.



25
26
27
28
29
30
 
# File 'lib/sitediff/fetch.rb', line 25

def run(&block)
  @callback = block
  @hydra = Typhoeus::Hydra.new(max_concurrency: @concurrency)
  @paths.each { |path| queue_path(path) }
  @hydra.run
end