Module: Sluice::Storage

Defined in:
lib/sluice/storage/s3/s3.rb,
lib/sluice/storage/storage.rb,
lib/sluice/storage/s3/location.rb,
lib/sluice/storage/s3/manifest.rb,
lib/sluice/storage/s3/contracts.rb

Defined Under Namespace

Modules: S3 Classes: NegativeRegex

Class Method Summary collapse

Class Method Details

.files_between(start_date, end_date, date_format, file_ext = nil) ⇒ Object

Find files within the given date range (inclusive).

Parameters:

start_date

start date

end_date

end date

+date_format

format of date in filenames

+file_ext

extension on files (if any)



30
31
32
33
34
35
36
37
38
 
# File 'lib/sluice/storage/storage.rb', line 30

def self.files_between(start_date, end_date, date_format, file_ext=nil)

  dates = []
  Date.parse(start_date).upto(Date.parse(end_date)) do |day|
    dates << day.strftime(date_format)
  end

  '(' + dates.join('|') + ')[^/]+%s$' % regexify(file_ext)
end

.files_from(start_date, date_format, file_ext = nil) ⇒ Object

Find files starting from the given date.

Parameters:

start_date

start date

+date_format

format of date in filenames

+file_ext

extension on files (if any); include period



81
82
83
84
85
86
87
88
89
90
91
92
 
# File 'lib/sluice/storage/storage.rb', line 81

def self.files_from(start_date, date_format, file_ext=nil)

  # Let's create a white list from the start_date to today
  today = Date.today

  dates = []
  Date.parse(start_date).upto(today) do |day|
    dates << day.strftime(date_format)
  end

  '(' + dates.join('|') + ')[^/]+%s$' % regexify(file_ext)
end

.files_up_to(end_date, date_format, file_ext = nil) ⇒ Object

Find files up to (and including) the given date.

Returns a regex in a NegativeRegex so that the matcher can negate the match.

Parameters:

end_date

end date

+date_format

format of date in filenames

+file_ext

extension on files (if any)



60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/sluice/storage/storage.rb', line 60

def self.files_up_to(end_date, date_format, file_ext=nil)

  # Let's create a black list from the day
  # after the end_date up to today
  day_after = Date.parse(end_date) + 1
  today = Date.today

  dates = []
  day_after.upto(today) do |day|
    dates << day.strftime(date_format) # Black list
  end

  NegativeRegex.new('(' + dates.join('|') + ')[^/]+%s$' % regexify(file_ext))
end

.trail_slash(path) ⇒ Object

Add a trailing slash to a path if missing. Tolerates a nil path.

Parameters:

path

path to add a trailing slash to



45
46
47
48
49
 
# File 'lib/sluice/storage/storage.rb', line 45

def self.trail_slash(path)
  unless path.nil?
    path[-1].chr != '/' ? path << '/' : path
  end
end