Class: Rspider::UrlStorageCache

Inherits:
UrlStorage show all
Defined in:
lib/rspider/MysqlUrlStorage.rb

Overview

we re-defined the UrlStorage Class and add the urlStored? method

Constant Summary collapse

MAX = 
512
STEP = 
128

Instance Attribute Summary

Attributes inherited from UrlStorage

#urls, #visitedUrls

Instance Method Summary collapse

Methods inherited from UrlStorage

#<<, #error, #initialize, #pop, #visited, #visited?

Constructor Details

This class inherits a constructor from Rspider::UrlStorage

Instance Method Details

#urlStored?(url) ⇒ Boolean

Returns:

  • (Boolean) 


10
11
12
13
14
15
16
17
18
 
# File 'lib/rspider/MysqlUrlStorage.rb', line 10

def urlStored?(url)
	STEP.times do 
		@urls.shift
	end if @urls.length>MAX
	STEP.times do 
		@visitedUrls.shift
	end if @visitedUrls.length>MAX
	@visitedUrls.include?(url) or @urls.include?(url)
end