Class: Moab::StorageObjectVersion

Inherits:
Object
  • Object
show all
Defined in:
lib/moab/storage_object_version.rb

Overview

Note:

Copyright © 2012 by The Board of Trustees of the Leland Stanford Junior University. All rights reserved. See LICENSE for details.

A class to represent a version subdirectory within an object’s home directory in preservation storage

Data Model

  • StorageRepository = represents a digital object repository storage node

    • StorageServices = supports application layer access to the repository’s objects, data, and metadata

    • StorageObject = represents a digital object’s repository storage location and ingest/dissemination methods

      • StorageObjectVersion [1..*] = represents a version subdirectory within an object’s home directory

        • Bagger [1] = utility for creating bagit packages for ingest or dissemination

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(storage_object, version_id) ⇒ StorageObjectVersion

Returns a new instance of StorageObjectVersion.

Parameters:

  • storage_object (StorageObject) —

    The object representing the digital object’s storage location

  • version_id (Integer, String) —

    The ordinal version number or a string like ‘v0003’



32
33
34
35
36
37
38
39
40
41
42
43
44
 
# File 'lib/moab/storage_object_version.rb', line 32

def initialize(storage_object, version_id)
  if version_id.is_a?(Integer)
    @version_id = version_id
  elsif version_id.is_a?(String) && version_id =~ /^v(\d+)$/
    @version_id = version_id.sub(/^v/, '').to_i
  else
    raise(MoabRuntimeError, "version_id (#{version_id}) is not in a recognized format")
  end
  @version_name = StorageObject.version_dirname(@version_id)
  @version_pathname = storage_object.object_pathname.join(@version_name)
  @storage_object = storage_object
  @inventory_cache = {}
end

Instance Attribute Details

#inventory_cache ⇒ Hash<FileInventory>

Returns Cached copies of versionInventory, versionAdditions, or manifestInventory.

Returns:

  • (Hash<FileInventory>) —

    Cached copies of versionInventory, versionAdditions, or manifestInventory



28
29
30
 
# File 'lib/moab/storage_object_version.rb', line 28

def inventory_cache
  @inventory_cache
end

#storage_object ⇒ Pathname

Returns The location of the object’s home directory.

Returns:

  • (Pathname) —

    The location of the object’s home directory



25
26
27
 
# File 'lib/moab/storage_object_version.rb', line 25

def storage_object
  @storage_object
end

#version_id ⇒ Integer

Returns The ordinal version number.

Returns:

  • (Integer) —

    The ordinal version number



16
17
18
 
# File 'lib/moab/storage_object_version.rb', line 16

def version_id
  @version_id
end

#version_name ⇒ String

Returns The “v0001” directory name derived from the version id.

Returns:

  • (String) —

    The “v0001” directory name derived from the version id



19
20
21
 
# File 'lib/moab/storage_object_version.rb', line 19

def version_name
  @version_name
end

#version_pathname ⇒ Pathname

Returns The location of the version inside the home directory.

Returns:

  • (Pathname) —

    The location of the version inside the home directory



22
23
24
 
# File 'lib/moab/storage_object_version.rb', line 22

def version_pathname
  @version_pathname
end

Instance Method Details

#composite_key ⇒ String

Returns The unique identifier concatenating digital object id with version id.

Returns:

  • (String) —

    The unique identifier concatenating digital object id with version id



47
48
49
 
# File 'lib/moab/storage_object_version.rb', line 47

def composite_key
  @storage_object.digital_object_id + '-' + StorageObject.version_dirname(@version_id)
end

#deactivate(timestamp) ⇒ null

Returns Deactivate this object version by moving it to another directory. (Used by restore operation).

Parameters:

  • timestamp (Time) —

    The time at which the deactivation was initiated. Used to name the inactive directory

Returns:

  • (null) —

    Deactivate this object version by moving it to another directory. (Used by restore operation)



328
329
330
331
332
333
334
335
 
# File 'lib/moab/storage_object_version.rb', line 328

def deactivate(timestamp)
  return unless @version_pathname.exist?

  timestamp_pathname = @version_pathname.parent.join(timestamp.utc.iso8601.gsub(/[-:]/, ''))
  timestamp_pathname.mkpath
  demote_pathame = timestamp_pathname.join(@version_pathname.basename)
  @version_pathname.rename(demote_pathame)
end

#exist? ⇒ Boolean

Returns true if the object version directory exists.

Returns:

  • (Boolean) —

    true if the object version directory exists



52
53
54
 
# File 'lib/moab/storage_object_version.rb', line 52

def exist?
  @version_pathname.exist?
end

#file_category_pathname(file_category) ⇒ Pathname

Returns Pathname object containing this version’s storage home for the specified file category.

Parameters:

  • file_category (String) —

    The category of file (‘content’, ‘metadata’, or ‘s’)

Returns:

  • (Pathname) —

    Pathname object containing this version’s storage home for the specified file category



100
101
102
103
104
105
106
 
# File 'lib/moab/storage_object_version.rb', line 100

def file_category_pathname(file_category)
  if file_category =~ /manifest/
    @version_pathname.join('manifests')
  else
    @version_pathname.join('data', file_category)
  end
end

#file_inventory(type) ⇒ FileInventory

Returns The file inventory of the specified type for this version.

Parameters:

  • type (String) —

    The type of inventory to return (version|additions|manifests)

Returns:

  • (FileInventory) —

    The file inventory of the specified type for this version

See Also:



112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
 
# File 'lib/moab/storage_object_version.rb', line 112

def file_inventory(type)
  if version_id > 0
    return @inventory_cache[type] if @inventory_cache.key?(type)

    @inventory_cache[type] = FileInventory.read_xml_file(@version_pathname.join('manifests'), type)
  else
    groups = %w[content metadata].collect { |id| FileGroup.new(:group_id => id) }
    FileInventory.new(
      :type => 'version',
      :digital_object_id => @storage_object.digital_object_id,
      :version_id => @version_id,
      :groups => groups
    )
  end
end

#file_pathname(file_category, file_id) ⇒ Pathname

Returns Pathname object containing this version’s storage path for the specified file.

Parameters:

  • file_category (String) —

    The category of file (‘content’, ‘metadata’, or ‘s’)

  • file_id (String) —

    The name of the file (path relative to base directory)

Returns:

  • (Pathname) —

    Pathname object containing this version’s storage path for the specified file



94
95
96
 
# File 'lib/moab/storage_object_version.rb', line 94

def file_pathname(file_category, file_id)
  file_category_pathname(file_category).join(file_id)
end

#find_filepath(file_category, file_id) ⇒ Pathname

Returns Pathname object containing the full path for the specified file.

Parameters:

  • file_category (String) —

    The category of file (‘content’, ‘metadata’, or ‘manifest’)

  • file_id (String) —

    The name of the file (path relative to base directory)

Returns:

  • (Pathname) —

    Pathname object containing the full path for the specified file



70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/moab/storage_object_version.rb', line 70

def find_filepath(file_category, file_id)
  this_version_filepath = file_pathname(file_category, file_id)
  return this_version_filepath if this_version_filepath.exist?

  if file_category == 'manifest'
    msg = "manifest file #{file_id} not found for #{@storage_object.digital_object_id} - #{@version_id}"
    raise FileNotFoundException, msg
  end
  file_signature = file_inventory('version').file_signature(file_category, file_id)
  catalog_filepath = signature_catalog.catalog_filepath(file_signature)
  @storage_object.storage_filepath(catalog_filepath)
end

#find_filepath_using_signature(_file_category, file_signature) ⇒ Pathname

Returns Pathname object containing the full path for the specified file.

Parameters:

  • _file_category (unused; kept here for backwards compatibility)
  • file_signature (FileSignature) —

    The signature of the file

Returns:

  • (Pathname) —

    Pathname object containing the full path for the specified file



86
87
88
89
 
# File 'lib/moab/storage_object_version.rb', line 86

def find_filepath_using_signature(_file_category, file_signature)
  catalog_filepath = signature_catalog.catalog_filepath(file_signature)
  @storage_object.storage_filepath(catalog_filepath)
end

#find_signature(file_category, file_id) ⇒ FileSignature

Returns signature of the specified file.

Parameters:

  • file_category (String) —

    The category of file (‘content’, ‘metadata’, or ‘manifest’))

  • file_id (String) —

    The name of the file (path relative to base directory)

Returns:



59
60
61
62
63
64
65
 
# File 'lib/moab/storage_object_version.rb', line 59

def find_signature(file_category, file_id)
  if file_category =~ /manifest/
    file_inventory('manifests').file_signature('manifests', file_id)
  else
    file_inventory('version').file_signature(file_category, file_id)
  end
end

#generate_differences_report(old_inventory, new_inventory) ⇒ void

This method returns an undefined value.

Returns generate a file inventory differences report and save to disk.

Parameters:



196
197
198
199
 
# File 'lib/moab/storage_object_version.rb', line 196

def generate_differences_report(old_inventory, new_inventory)
  differences = FileInventoryDifference.new.compare(old_inventory, new_inventory)
  differences.write_xml_file(@version_pathname.join('manifests'))
end

#generate_manifest_inventory ⇒ void

This method returns an undefined value.

Returns examine the version’s directory and create/serialize a FileInventory containing the manifest files.



203
204
205
206
207
208
209
210
211
212
 
# File 'lib/moab/storage_object_version.rb', line 203

def generate_manifest_inventory
  manifest_inventory = FileInventory.new(
    :type => 'manifests',
    :digital_object_id => @storage_object.digital_object_id,
    :version_id => @version_id
  )
  pathname = @version_pathname.join('manifests')
  manifest_inventory.groups << FileGroup.new(:group_id => 'manifests').group_from_directory(pathname, false)
  manifest_inventory.write_xml_file(pathname)
end

#ingest_bag_data(bag_dir) ⇒ void

This method returns an undefined value.

Returns Create the version subdirectory and move files into it.

Parameters:

  • bag_dir (Pathname, String) —

    The location of the bag to be ingested

Raises:



141
142
143
144
145
146
147
148
149
 
# File 'lib/moab/storage_object_version.rb', line 141

def ingest_bag_data(bag_dir)
  raise(MoabRuntimeError, "Version already exists: #{@version_pathname}") if @version_pathname.exist?

  @version_pathname.join('manifests').mkpath
  bag_dir = Pathname(bag_dir)
  ingest_dir(bag_dir.join('data'), @version_pathname.join('data'))
  ingest_file(bag_dir.join(FileInventory.xml_filename('version')), @version_pathname.join('manifests'))
  ingest_file(bag_dir.join(FileInventory.xml_filename('additions')), @version_pathname.join('manifests'))
end

#ingest_dir(source_dir, target_dir, use_links = true) ⇒ void

This method returns an undefined value.

Returns recursively link or copy the source directory contents to the target directory.

Parameters:

  • source_dir (Pathname) —

    The source location of the directory whose contents are to be ingested

  • target_dir (Pathname) —

    The target location of the directory into which files are ingested

  • use_links (Boolean) (defaults to: true) —

    If true, use hard links; if false, make copies

Raises:



156
157
158
159
160
161
162
163
164
165
166
167
 
# File 'lib/moab/storage_object_version.rb', line 156

def ingest_dir(source_dir, target_dir, use_links = true)
  raise(MoabRuntimeError, "cannot copy - target already exists: #{target_dir.expand_path}") if target_dir.exist?

  target_dir.mkpath
  source_dir.children.each do |child|
    if child.directory?
      ingest_dir(child, target_dir.join(child.basename), use_links)
    else
      ingest_file(child, target_dir, use_links)
    end
  end
end

#ingest_file(source_file, target_dir, use_links = true) ⇒ void

This method returns an undefined value.

Returns link or copy the specified file from source location to the version directory.

Parameters:

  • source_file (Pathname) —

    The source location of the file to be ingested

  • target_dir (Pathname) —

    The location of the directory in which to place the file

  • use_links (Boolean) (defaults to: true) —

    If true, use hard links; if false, make copies



174
175
176
177
178
179
180
 
# File 'lib/moab/storage_object_version.rb', line 174

def ingest_file(source_file, target_dir, use_links = true)
  if use_links
    FileUtils.link(source_file.to_s, target_dir.to_s) #, :force => true)
  else
    FileUtils.copy(source_file.to_s, target_dir.to_s)
  end
end

#signature_catalog ⇒ SignatureCatalog

Returns The signature catalog of the digital object as of this version.

Returns:

  • (SignatureCatalog) —

    The signature catalog of the digital object as of this version



130
131
132
133
134
135
136
 
# File 'lib/moab/storage_object_version.rb', line 130

def signature_catalog
  if version_id > 0
    SignatureCatalog.read_xml_file(@version_pathname.join('manifests'))
  else
    SignatureCatalog.new(:digital_object_id => @storage_object.digital_object_id)
  end
end

#update_catalog(signature_catalog, new_inventory) ⇒ void

This method returns an undefined value.

Returns Updates the catalog to include newly added files, then saves it to disk.

Parameters:

See Also:



187
188
189
190
 
# File 'lib/moab/storage_object_version.rb', line 187

def update_catalog(signature_catalog, new_inventory)
  signature_catalog.update(new_inventory, @version_pathname.join('data'))
  signature_catalog.write_xml_file(@version_pathname.join('manifests'))
end

#verify_manifest_inventory ⇒ VerificationResult

Return true if the manifest inventory matches the actual files

Returns:



225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
 
# File 'lib/moab/storage_object_version.rb', line 225

def verify_manifest_inventory
  # read/parse manifestInventory.xml
  result = VerificationResult.new("manifest_inventory")
  manifest_inventory = file_inventory('manifests')
  result.subentities << VerificationResult.verify_value('composite_key', composite_key, manifest_inventory.composite_key)
  result.subentities << VerificationResult.verify_truth('manifests_group', !manifest_inventory.group_empty?('manifests'))
  # measure the manifest signatures of the files in the directory (excluding manifestInventory.xml)
  directory_inventory = FileInventory.new.inventory_from_directory(@version_pathname.join('manifests'), 'manifests')
  directory_inventory.digital_object_id = storage_object.digital_object_id
  directory_group = directory_inventory.group('manifests')
  directory_group.remove_file_having_path("manifestInventory.xml")
  # compare the measured signatures against the values in manifestInventory.xml
  diff = FileInventoryDifference.new
  diff.compare(manifest_inventory, directory_inventory)
  compare_result = VerificationResult.new('file_differences')
  compare_result.verified = (diff.difference_count == 0)
  compare_result.details = diff.differences_detail
  result.subentities << compare_result
  result.verified = result.subentities.all?(&:verified)
  result
end

#verify_signature_catalog ⇒ VerificationResult

Returns:



248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
 
# File 'lib/moab/storage_object_version.rb', line 248

def verify_signature_catalog
  result = VerificationResult.new("signature_catalog")
  signature_catalog = self.signature_catalog
  result.subentities << VerificationResult.verify_value('signature_key', composite_key, signature_catalog.composite_key)
  found = 0
  missing = []
  object_pathname = storage_object.object_pathname
  signature_catalog.entries.each do |catalog_entry|
    storage_location = object_pathname.join(catalog_entry.storage_path)
    if storage_location.exist?
      found += 1
    else
      missing << storage_location.to_s
    end
  end
  file_result = VerificationResult.new("storage_location")
  file_result.verified = (found == signature_catalog.file_count)
  file_result.details = {
    'expected' => signature_catalog.file_count,
    'found' => found
  }
  file_result.details['missing'] = missing unless missing.empty?
  result.subentities << file_result
  result.verified = result.subentities.all?(&:verified)
  result
end

#verify_version_additions ⇒ Boolean

Returns true if files in data folder match files listed in version addtions inventory

Returns:

  • (Boolean) —

    returns true if files in data folder match files listed in version addtions inventory



310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
 
# File 'lib/moab/storage_object_version.rb', line 310

def verify_version_additions
  result = VerificationResult.new("version_additions")
  version_additions = file_inventory('additions')
  result.subentities << VerificationResult.verify_value('composite_key', composite_key, version_additions.composite_key)
  data_directory = @version_pathname.join('data')
  directory_inventory = FileInventory.new(:type => 'directory').inventory_from_directory(data_directory)
  diff = FileInventoryDifference.new
  diff.compare(version_additions, directory_inventory)
  compare_result = VerificationResult.new('file_differences')
  compare_result.verified = (diff.difference_count == 0)
  compare_result.details = diff.differences_detail
  result.subentities << compare_result
  result.verified = result.subentities.all?(&:verified)
  result
end

#verify_version_inventory ⇒ Boolean

Returns true if files & signatures listed in version inventory can all be found.

Returns:

  • (Boolean) —

    true if files & signatures listed in version inventory can all be found



276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
 
# File 'lib/moab/storage_object_version.rb', line 276

def verify_version_inventory
  result = VerificationResult.new("version_inventory")
  version_inventory = file_inventory('version')
  result.subentities << VerificationResult.verify_value('inventory_key', composite_key, version_inventory.composite_key)
  signature_catalog = self.signature_catalog
  result.subentities << VerificationResult.verify_value('signature_key', composite_key, signature_catalog.composite_key)
  found = 0
  missing = []
  version_inventory.groups.each do |group|
    group.files.each do |file|
      file.instances.each do |instance|
        relative_path = File.join(group.group_id, instance.path)
        catalog_entry = signature_catalog.signature_hash[file.signature]
        if !catalog_entry.nil?
          found += 1
        else
          missing << relative_path.to_s
        end
      end
    end
  end
  file_result = VerificationResult.new("catalog_entry")
  file_result.verified = (found == version_inventory.file_count)
  file_result.details = {
    'expected' => version_inventory.file_count,
    'found' => found
  }
  file_result.details['missing'] = missing unless missing.empty?
  result.subentities << file_result
  result.verified = result.subentities.all?(&:verified)
  result
end

#verify_version_storage ⇒ VerificationResult

Return result of testing correctness of version manifests

Returns:



215
216
217
218
219
220
221
222
 
# File 'lib/moab/storage_object_version.rb', line 215

def verify_version_storage
  result = VerificationResult.new(composite_key)
  result.subentities << verify_manifest_inventory
  result.subentities << verify_version_inventory
  result.subentities << verify_version_additions
  result.verified = result.subentities.all?(&:verified)
  result
end