Module: Sluice::Storage

Defined in:
lib/sluice/storage/s3/s3.rb,
lib/sluice/storage/storage.rb,
lib/sluice/storage/s3/location.rb,
lib/sluice/storage/s3/manifest.rb,
lib/sluice/storage/s3/contracts.rb

Defined Under Namespace

Modules: S3 Classes: NegativeRegex

Class Method Summary collapse

Class Method Details

.files_between(start_date, end_date, date_format, file_ext = nil) ⇒ Object

Find files within the given date range (inclusive).

Parameters:

start_date

start date

end_date

end date

+date_format

format of date in filenames

+file_ext

extension on files (if any)



30
31
32
33
34
35
36
37
38
 
# File 'lib/sluice/storage/storage.rb', line 30

def files_between(start_date, end_date, date_format, file_ext=nil)

  dates = []
  Date.parse(start_date).upto(Date.parse(end_date)) do |day|
    dates << day.strftime(date_format)
  end

  '(' + dates.join('|') + ')[^/]+%s$' % regexify(file_ext)
end

.files_from(start_date, date_format, file_ext = nil) ⇒ Object

Find files starting from the given date.

Parameters:

start_date

start date

+date_format

format of date in filenames

+file_ext

extension on files (if any); include period



84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'lib/sluice/storage/storage.rb', line 84

def files_from(start_date, date_format, file_ext=nil)

  # Let's create a white list from the start_date to today
  today = Date.today

  dates = []
  Date.parse(start_date).upto(today) do |day|
    dates << day.strftime(date_format)
  end

  '(' + dates.join('|') + ')[^/]+%s$' % regexify(file_ext)
end

.files_up_to(end_date, date_format, file_ext = nil) ⇒ Object

Find files up to (and including) the given date.

Returns a regex in a NegativeRegex so that the matcher can negate the match.

Parameters:

end_date

end date

+date_format

format of date in filenames

+file_ext

extension on files (if any)



62
63
64
65
66
67
68
69
70
71
72
73
74
75
 
# File 'lib/sluice/storage/storage.rb', line 62

def files_up_to(end_date, date_format, file_ext=nil)

  # Let's create a black list from the day
  # after the end_date up to today
  day_after = Date.parse(end_date) + 1
  today = Date.today

  dates = []
  day_after.upto(today) do |day|
    dates << day.strftime(date_format) # Black list
  end

  NegativeRegex.new('(' + dates.join('|') + ')[^/]+%s$' % regexify(file_ext))
end

.regexify(file_ext) ⇒ Object

Make a file extension regular expression friendly, adding a starting period (.) if missing

Parameters:

+file_ext

the file extension to make regexp friendly



105
106
107
 
# File 'lib/sluice/storage/storage.rb', line 105

def regexify(file_ext)
  file_ext.nil? ? nil : file_ext[0].chr != '.' ? '\\.' << file_ext : '\\' << file_ext
end

.trail_slash(path) ⇒ Object

Add a trailing slash to a path if missing. Tolerates a nil path.

Parameters:

path

path to add a trailing slash to



46
47
48
49
50
 
# File 'lib/sluice/storage/storage.rb', line 46

def trail_slash(path)
  unless path.nil?
    path[-1].chr != '/' ? path << '/' : path
  end
end