Class: BuildpackUtils::DownloadCache

Inherits:
Object
  • Object
show all
Defined in:
lib/buildpack-utils/download_cache.rb

Overview

A cache for downloaded files that is configured to use a filesystem as the backing store. This cache uses standard file locking (File.flock()) in order ensure that mutation of files in the cache is non-concurrent across processes. Reading files (once they’ve been downloaded) happens concurrently so read performance is not impacted.

Direct Known Subclasses

ApplicationCache, GlobalCache

Instance Method Summary collapse

Constructor Details

#initialize(cache_root = Dir.tmpdir) ⇒ DownloadCache

Creates an instance of the cache that is backed by the filesystem rooted at cache_root

Parameters:

  • cache_root (String) (defaults to: Dir.tmpdir) —

    the filesystem root for downloaded files to be cached in



27
28
29
 
# File 'lib/buildpack-utils/download_cache.rb', line 27

def initialize(cache_root = Dir.tmpdir)
  @cache_root = cache_root
end

Instance Method Details

#delete_file(filename) ⇒ Object 



86
87
88
 
# File 'lib/buildpack-utils/download_cache.rb', line 86

def delete_file(filename)
  File.delete filename if File.exists? filename
end

#evict(key) ⇒ void

This method returns an undefined value.

Remove an item from the cache

Parameters:

  • key (String) —

    the key of the item to remove



74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/buildpack-utils/download_cache.rb', line 74

def evict(key)
  filenames = filenames(key)
  File.open(filenames[:lock], File::CREAT) do |lock_file|
    lock_file.flock(File::LOCK_EX)

    delete_file filenames[:cached]
    delete_file filenames[:etag]
    delete_file filenames[:last_modified]
    delete_file filenames[:lock]
  end
end

#get(key, uri) {|file| ... } ⇒ void

This method returns an undefined value.

Retrieves an item from the cache. Retrieval of the item uses the following algorithm:

  1. Obtain an exclusive lock based on the key of the item. This allows concurrency for different items, but not for the same item.

  2. If the the cached item does not exist, download from uri and cache it, its Etag, and its Last-Modified values if they exist

  3. If the cached file does exist, and the original download had an Etag or a Last-Modified value, attempt to download from uri again. If the result is 304 (Not-Modified), then proceed without changing the cached item. If it is anything else, overwrite the cached file and its Etag and Last-Modified values if they exist.

  4. Downgrade the lock to a shared lock as no further mutation of the cache is possible. This allows concurrency for read access of the item.

  5. Yield the cached file (opened read-only) to the passed in block. Once the block is complete, the file is closed and the lock is released.

Parameters:

  • key (String) —

    the key of the item to retrieve

  • uri (String) —

    the uri to download if the item is not already in the cache. Also used in the case where the item is already in the cache, to validate that the item is up to date

Yield Parameters:

  • file (File) —

    the file representing the cached item. In order to ensure that the file is not changed or deleted while it is being used, the cached item can only be accessed as part of a block.



51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/buildpack-utils/download_cache.rb', line 51

def get(key, uri)
  filenames = filenames(key)
  File.open(filenames[:lock], File::CREAT) do |lock_file|
    lock_file.flock(File::LOCK_EX)

    if should_update(filenames)
      update(filenames, uri)
    elsif should_download(filenames)
      download(filenames, uri)
    end

    lock_file.flock(File::LOCK_SH)

    File.open(filenames[:cached], File::RDONLY) do |cached_file|
      yield cached_file
    end
  end
end