Class: AIPP::Downloader

Inherits:
Object show all
Defined in:
lib/aipp/downloader.rb

Overview

AIP downloader infrastructure

The downloader operates in the storage directory where it creates two subdirectories “archive” and “work”. The initializer looks for archive in “archives” and (if found) unzips its contents into “work”. When reading a document, the downloader looks for the document in “work” and (unless found) downloads it from url. HTML documents are parsed to Nokogiri::HTML5::Document, PDF documents are parsed to AIPP::PDF. Finally, the contents of “work” are written back to archive.

Examples:

AIPP::Downloader.new(storage: options[:storage], archive: "2018-11-08") do |downloader|
  html = downloader.read(
    document: 'ENR-5.1',
    url: 'https://www.sia.aviation-civile.gouv.fr/dvd/eAIP_08_NOV_2018/FRANCE/AIRAC-2018-11-08/html/eAIP/FR-ENR-5.1-fr-FR.html'
  )
  pdf = downloader.read(
    document: 'VAC-LFMV',
    url: 'https://www.sia.aviation-civile.gouv.fr/dvd/eAIP_08_NOV_2018/Atlas-VAC/PDF_AIPparSSection/VAC/AD/AD-2.LFMV.pdf'
  )
end

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(storage:, archive:) ⇒ Downloader

Returns a new instance of Downloader.

Parameters:

  • storage (Pathname) —

    directory to operate within

  • archive (String) —

    name of the archive (without extension “.zip”)



37
38
39
40
41
42
43
44
45
46
47
 
# File 'lib/aipp/downloader.rb', line 37

def initialize(storage:, archive:)
  @storage, @archive = storage, archive
  fail(ArgumentError, 'bad storage directory') unless Dir.exist? storage
  @archive_file = archives_path.join("#{@archive}.zip")
  prepare
  unzip if @archive_file.exist?
  yield self
  zip
ensure
  teardown
end

Instance Attribute Details

#archive ⇒ String (readonly)

Returns name of the archive (without extension “.zip”).

Returns:

  • (String) —

    name of the archive (without extension “.zip”)



30
31
32
 
# File 'lib/aipp/downloader.rb', line 30

def archive
  @archive
end

#archive_file ⇒ Pathname (readonly)

Returns full path to the archive.

Returns:

  • (Pathname) —

    full path to the archive



33
34
35
 
# File 'lib/aipp/downloader.rb', line 33

def archive_file
  @archive_file
end

#storage ⇒ Pathname (readonly)

Returns directory to operate within.

Returns:

  • (Pathname) —

    directory to operate within



27
28
29
 
# File 'lib/aipp/downloader.rb', line 27

def storage
  @storage
end

Instance Method Details

#read(document:, url:, type: nil) ⇒ Nokogiri::HTML5::Document, AIPP::PDF

Download and read document

Parameters:

  • document (String) —

    document to read (without extension)

  • url (String) —

    URL to download the document from

  • type (Symbol, nil) (defaults to: nil) —

    document type: nil (default) to derive it from the URL, :html, or :pdf

Returns:



56
57
58
59
60
61
62
63
64
 
# File 'lib/aipp/downloader.rb', line 56

def read(document:, url:, type: nil)
  type ||= Pathname(URI(url).path).extname[1..-1].to_sym
  file = work_path.join([document, type].join('.'))
  unless file.exist?
    verbose_info "Downloading #{document}"
    IO.copy_stream(Kernel.open(url), file)
  end
  convert file
end