Class: Wayfarer::Frontiers::Frontier Abstract Private

Inherits:
Object
  • Object
show all
Defined in:
lib/wayfarer/frontiers/frontier.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

This class is abstract.

The common behaviour of all frontiers.

Direct Known Subclasses

MemoryFrontier, RedisFrontier

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(config) ⇒ Frontier

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Frontier.



10
11
12
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 10

def initialize(config)
  @config = config
end

Instance Attribute Details

#config ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



8
9
10
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 8

def config
  @config
end

Instance Method Details

#cache(*_uris) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Caches URIs so they don't get processed again.

Parameters:

  • uris (*Array<URI>, *Array<String>) 


40
41
42
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 40

def cache(*_uris)
  raise "Unimplemented"
end

#cached?(_uri) ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Whether a URI is cached.

Returns:

  • (Boolean) 


45
46
47
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 45

def cached?(_uri)
  raise "Unimplemented"
end

#current_uris ⇒ Array<URI>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Note:

Usually an expensive operation!

Returns URIs to be scraped in the current cycle.

Returns:

  • (Array<URI>) 


17
18
19
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 17

def current_uris
  raise "Unimplemented"
end

#cycle ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

TODO: Documentation



53
54
55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 53

def cycle
  unless config.allow_circulation
    cache(*current_uris) # TODO: Make it a template method
    filter_staged_uris!
  end

  return false if staged_uris.none?

  swap!
  reset_staged_uris!

  true
end

#filter_staged_uris! ⇒ Object (protected)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

TODO: Documentation



70
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 70

def filter_staged_uris!; end

#free ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Frees resources.



50
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 50

def free; end

#reset_staged_uris! ⇒ Object (protected)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

TODO: Documentation



76
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 76

def reset_staged_uris!; end

#stage(*_uris) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Stages URIs for processing in the next cycle.

Parameters:

  • uris (*Array<URI>, *Array<String>) 


29
30
31
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 29

def stage(*_uris)
  raise "Unimplemented"
end

#staged?(_uri) ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Whether a URI is cached.

Returns:

  • (Boolean) 


34
35
36
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 34

def staged?(_uri)
  raise "Unimplemented"
end

#staged_uris ⇒ Array<URI>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns staged URIs.

Returns:

  • (Array<URI>) 


23
24
25
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 23

def staged_uris
  raise "Unimplemented"
end

#swap! ⇒ Object (protected)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

TODO: Documentation



73
 
# File 'lib/wayfarer/frontiers/frontier.rb', line 73

def swap!; end