Class: Monet::CaptureMap::PathSpider

Inherits:
PathCollection show all
Defined in:
lib/monet/capture_map.rb

Constant Summary collapse

SKIP_EXT = 
%w(js css png jpg mp4 txt zip ico ogv ogg pdf gz)
SKIP_PATHS = 
[/\?.*/]

Instance Attribute Summary

Attributes inherited from PathCollection

#root_url

Instance Method Summary collapse

Methods inherited from PathCollection

#add, #initialize, #normalized_path

Constructor Details

This class inherits a constructor from Monet::CaptureMap::PathCollection

Instance Method Details

#ignores ⇒ Object 



46
47
48
 
# File 'lib/monet/capture_map.rb', line 46

def ignores
  SKIP_EXT.map {|x| Regexp.new x }.concat SKIP_PATHS
end

#paths ⇒ Object 



39
40
41
42
43
44
 
# File 'lib/monet/capture_map.rb', line 39

def paths
  return @paths unless @paths.empty?

  normalize Spidr.site(@root_url, ignore_links: ignores)
  @paths.uniq!
end