Class: RightScale::ScraperBase

Inherits:
Object
  • Object
show all
Defined in:
lib/right_scraper/scraper_base.rb

Overview

Base class for all scrapers. Actual scraper implementation should override scrape_imp and optionally incremental_update?

Direct Known Subclasses

DownloadScraper, GitScraper, SvnScraper

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(root_dir) ⇒ ScraperBase

Set path to directory containing all scraped repos

Parameters

root_dir(String)

Path to scraped repos parent directory



48
49
50
 
# File 'lib/right_scraper/scraper_base.rb', line 48

def initialize(root_dir)
  @root_dir = root_dir
end

Instance Attribute Details

#errors ⇒ Object (readonly)

(Array) Error messages if any



39
40
41
 
# File 'lib/right_scraper/scraper_base.rb', line 39

def errors
  @errors
end

#repo ⇒ Object (readonly)

(RightScale::Repository) Last scraped repository



36
37
38
 
# File 'lib/right_scraper/scraper_base.rb', line 36

def repo
  @repo
end

#repo_dir ⇒ Object (readonly)

(String) Path to local directory where repository was downloaded



42
43
44
 
# File 'lib/right_scraper/scraper_base.rb', line 42

def repo_dir
  @repo_dir
end

#root_dir ⇒ Object

(String) Path to directory containing all scraped repositories



33
34
35
 
# File 'lib/right_scraper/scraper_base.rb', line 33

def root_dir
  @root_dir
end

Instance Method Details

#scrape(repo, &callback) ⇒ Object

Common implementation of scrape method for all repository types. Each scraper implementation should override scrape_imp which is called after this method initializes all the scraper attributes properly. See RightScale::Scraper#scrape



56
57
58
59
60
61
62
63
64
65
66
67
 
# File 'lib/right_scraper/scraper_base.rb', line 56

def scrape(repo, &callback)
  @repo            = repo
  @callback        = callback
  @scrape_dir_name = Digest::MD5.hexdigest(repo.to_s)
  @scrape_dir_path = File.join(root_dir, @scrape_dir_name)
  @repo_dir        = "#{@scrape_dir_path}/repo"
  @incremental     = incremental_update?
  @errors          = []
  FileUtils.rm_rf(@repo_dir) unless @incremental
  scrape_imp
  true
end

#succeeded? ⇒ Boolean

Was last call to scrapesuccessful? Call errors to get error messages if false

Return

succeeded(TrueClass|FalseClass)

true if scrape finished with no error, false otherwise.

Returns:

  • (Boolean) 


74
75
76
 
# File 'lib/right_scraper/scraper_base.rb', line 74

def succeeded?
  succeeded = @errors.nil? || @errors.size == 0
end