Class: Scruber::QueueAdapters::AbstractAdapter Abstract

Inherits:
Object
  • Object
show all
Defined in:
lib/scruber/queue_adapters/abstract_adapter.rb

Overview

This class is abstract.

Abstract Queue Adapter

Author:

  • Ivan Goncharov

Direct Known Subclasses

Memory

Defined Under Namespace

Classes: Page

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ AbstractAdapter

Returns a new instance of AbstractAdapter.



204
205
206
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 204

def initialize(options={})
  @options = options
end

Instance Method Details

#add(url, options = {}) ⇒ void

This method returns an undefined value.

Add page to queue

Parameters:

  • url (String) —

    URL of page

  • options (Hash) (defaults to: {}) —

    Other options, see Page

Raises:

  • (NotImplementedError) 


214
215
216
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 214

def add(url, options={})
  raise NotImplementedError
end

#downloaded_count ⇒ Integer

Count of downloaded pages Using to show downloading progress.

Returns:

  • (Integer) —

    count of downloaded pages

Raises:

  • (NotImplementedError) 


267
268
269
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 267

def downloaded_count
  raise NotImplementedError
end

#fetch_downloaded(count = nil) ⇒ Scruber::QueueAdapters::AbstractAdapter::Page|Array<Scruber::QueueAdapters::AbstractAdapter::Page>

Fetch downloaded and not processed pages for feching

Parameters:

  • count (Integer) (defaults to: nil) —

    count of pages to fetch

Returns:

Raises:

  • (NotImplementedError) 


249
250
251
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 249

def fetch_downloaded(count=nil)
  raise NotImplementedError
end

#fetch_error(count = nil) ⇒ Scruber::QueueAdapters::AbstractAdapter::Page|Array<Scruber::QueueAdapters::AbstractAdapter::Page>

Fetch error page

Parameters:

  • count (Integer) (defaults to: nil) —

    count of pages to fetch

Returns:

Raises:

  • (NotImplementedError) 


258
259
260
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 258

def fetch_error(count=nil)
  raise NotImplementedError
end

#fetch_pending(count = nil) ⇒ Scruber::QueueAdapters::AbstractAdapter::Page|Array<Scruber::QueueAdapters::AbstractAdapter::Page>

Fetch pending page for fetching

Parameters:

  • count (Integer) (defaults to: nil) —

    count of pages to fetch

Returns:

Raises:

  • (NotImplementedError) 


240
241
242
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 240

def fetch_pending(count=nil)
  raise NotImplementedError
end

#find(id) ⇒ Page

Search page by id

Parameters:

Returns:

  • (Page) —

    page object

Raises:

  • (NotImplementedError) 


223
224
225
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 223

def find(id)
  raise NotImplementedError
end

#has_work? ⇒ Boolean

Used by Core. It checks for pages that are not downloaded or not parsed yet.

Returns:

  • (Boolean) —

    true if queue still has work for scraper

Raises:

  • (NotImplementedError) 


286
287
288
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 286

def has_work?
  raise NotImplementedError
end

#initialized? ⇒ Boolean

Check if queue was initialized. Using for ‘seed` method. If queue was initialized, then no need to run seed block.

Returns:

  • (Boolean) —

    true if queue already was initialized

Raises:

  • (NotImplementedError) 


277
278
279
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 277

def initialized?
  raise NotImplementedError
end

#size ⇒ Integer

Size of queue

Returns:

  • (Integer) —

    count of pages in queue

Raises:

  • (NotImplementedError) 


231
232
233
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 231

def size
  raise NotImplementedError
end