Class: Scruber::QueueAdapters::AbstractAdapter Abstract

Inherits:
Object
  • Object
show all
Defined in:
lib/scruber/queue_adapters/abstract_adapter.rb

Overview

This class is abstract.

Abstract Queue Adapter

Author:

  • Ivan Goncharov

Direct Known Subclasses

Memory

Defined Under Namespace

Classes: Page

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ AbstractAdapter

Returns a new instance of AbstractAdapter.



194
195
196
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 194

def initialize(options={})
  @options = options
end

Instance Method Details

#add(url, options = {}) ⇒ void

This method returns an undefined value.

Add page to queue

Parameters:

  • url (String) —

    URL of page

  • options (Hash) (defaults to: {}) —

    Other options, see Page

Raises:

  • (NotImplementedError) 


204
205
206
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 204

def add(url, options={})
  raise NotImplementedError
end

#downloaded_count ⇒ Integer

Count of downloaded pages Using to show downloading progress.

Returns:

  • (Integer) —

    count of downloaded pages

Raises:

  • (NotImplementedError) 


257
258
259
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 257

def downloaded_count
  raise NotImplementedError
end

#fetch_downloaded(count = nil) ⇒ Scruber::QueueAdapters::AbstractAdapter::Page|Array<Scruber::QueueAdapters::AbstractAdapter::Page>

Fetch downloaded and not processed pages for feching

Parameters:

  • count (Integer) (defaults to: nil) —

    count of pages to fetch

Returns:

Raises:

  • (NotImplementedError) 


239
240
241
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 239

def fetch_downloaded(count=nil)
  raise NotImplementedError
end

#fetch_error(count = nil) ⇒ Scruber::QueueAdapters::AbstractAdapter::Page|Array<Scruber::QueueAdapters::AbstractAdapter::Page>

Fetch error page

Parameters:

  • count (Integer) (defaults to: nil) —

    count of pages to fetch

Returns:

Raises:

  • (NotImplementedError) 


248
249
250
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 248

def fetch_error(count=nil)
  raise NotImplementedError
end

#fetch_pending(count = nil) ⇒ Scruber::QueueAdapters::AbstractAdapter::Page|Array<Scruber::QueueAdapters::AbstractAdapter::Page>

Fetch pending page for fetching

Parameters:

  • count (Integer) (defaults to: nil) —

    count of pages to fetch

Returns:

Raises:

  • (NotImplementedError) 


230
231
232
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 230

def fetch_pending(count=nil)
  raise NotImplementedError
end

#find(id) ⇒ Page

Search page by id

Parameters:

Returns:

  • (Page) —

    page object

Raises:

  • (NotImplementedError) 


213
214
215
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 213

def find(id)
  raise NotImplementedError
end

#has_work? ⇒ Boolean

Used by Core. It checks for pages that are not downloaded or not parsed yet.

Returns:

  • (Boolean) —

    true if queue still has work for scraper

Raises:

  • (NotImplementedError) 


276
277
278
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 276

def has_work?
  raise NotImplementedError
end

#initialized? ⇒ Boolean

Check if queue was initialized. Using for ‘seed` method. If queue was initialized, then no need to run seed block.

Returns:

  • (Boolean) —

    true if queue already was initialized

Raises:

  • (NotImplementedError) 


267
268
269
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 267

def initialized?
  raise NotImplementedError
end

#size ⇒ Integer

Size of queue

Returns:

  • (Integer) —

    count of pages in queue

Raises:

  • (NotImplementedError) 


221
222
223
 
# File 'lib/scruber/queue_adapters/abstract_adapter.rb', line 221

def size
  raise NotImplementedError
end