Class: Moab::StorageRepository

Inherits:
Object
  • Object
show all
Defined in:
lib/moab/storage_repository.rb

Overview

Note:

Copyright © 2012 by The Board of Trustees of the Leland Stanford Junior University. All rights reserved. See LICENSE for details.

A class to represent the SDR repository store

Data Model

  • StorageRepository = represents the digital object repository storage areas

    • StorageServices = supports application layer access to the repository’s objects, data, and metadata

    • StorageObject = represents a digital object’s repository storage location and ingest/dissemination methods

      • StorageObjectVersion [1..*] = represents a version subdirectory within an object’s home directory

        • Bagger [1] = utility for creating bagit packages for ingest or dissemination

Direct Known Subclasses

Stanford::StorageRepository

Instance Method Summary collapse

Instance Method Details

#deposit_branch(object_id) ⇒ Pathname

Returns The branch segment of the object deposit path.

Parameters:

  • object_id (String) —

    The identifier of the digital object

Returns:

  • (Pathname) —

    The branch segment of the object deposit path



63
64
65
66
 
# File 'lib/moab/storage_repository.rb', line 63

def deposit_branch(object_id)
   #todo This method should be customized, or overridden in a subclass
  object_id
end

#deposit_trunk ⇒ String

Returns The trunk segment of the object deposit path.

Returns:

  • (String) —

    The trunk segment of the object deposit path



52
53
54
55
56
57
58
59
 
# File 'lib/moab/storage_repository.rb', line 52

def deposit_trunk
  unless defined?(@deposit_trunk)
    # do not raise error.  this parameter will be ignored if missing
    # raise "Moab::Config.deposit_trunk not found in config file" if Moab::Config.deposit_trunk.nil?
    @deposit_trunk  = Moab::Config.deposit_trunk
  end
  @deposit_trunk
end

#find_storage_object(object_id, include_deposit = false) ⇒ StorageObject

Returns The representation of a digitial object’s storage directory, which might not exist yet.

Parameters:

  • object_id (String) —

    The identifier of the digital object whose version is desired

  • include_deposit (Boolean) (defaults to: false) —

    specifies whether to look in deposit areas for objects in process of initial ingest

Returns:

  • (StorageObject) —

    The representation of a digitial object’s storage directory, which might not exist yet.



97
98
99
100
101
102
103
 
# File 'lib/moab/storage_repository.rb', line 97

def find_storage_object(object_id, include_deposit=false)
  root = find_storage_root(object_id, include_deposit)
  storage_pathname = root.join(storage_trunk, storage_branch(object_id))
  storage_object = StorageObject.new(object_id, storage_pathname)
  storage_object.storage_root = root
  storage_object
end

#find_storage_root(object_id, include_deposit = false) ⇒ Pathname

Returns The location of the desired object’s home directory (default=pairtree).

Parameters:

  • object_id (String) —

    The identifier of the digital object whose version is desired

  • include_deposit (Boolean) (defaults to: false) —

    specifies whether to look in deposit areas for objects in process of initial ingest

Returns:

  • (Pathname) —

    The location of the desired object’s home directory (default=pairtree)



71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
 
# File 'lib/moab/storage_repository.rb', line 71

def find_storage_root(object_id, include_deposit=false)
  # Search for the object's home directory in the storage areas
  branch = storage_branch(object_id)
  storage_roots.each do |root|
    root_trunk = root.join(storage_trunk)
    raise "Storage area not found at #{root_trunk}" unless root_trunk.exist?
    root_trunk_branch = root_trunk.join(branch)
    return root if root_trunk_branch.exist?
  end
  # Search for the object's directory in the deposit areas
  if include_deposit and deposit_trunk
    branch = deposit_branch(object_id)
    storage_roots.each do |root|
      root_trunk = root.join(deposit_trunk)
      raise "Deposit area not found at #{root_trunk}" unless root_trunk.exist?
      root_trunk_branch = root_trunk.join(branch)
      return root if root_trunk_branch.exist?
    end
  end
  # object not found, will store new objects in the newest (most empty) filesystem
  storage_roots.last
end

#object_size(object_id, include_deposit = false) ⇒ Integer

Returns the size occupied on disk by the storage object, in bytes. this is the entire moab (all versions).

Parameters:

  • object_id (String) —

    The identifier of the digital object whose size is desired

  • include_deposit (Boolean) (defaults to: false) —

    specifies whether to look in deposit areas for objects in process of initial ingest

Returns:

  • (Integer) —

    the size occupied on disk by the storage object, in bytes. this is the entire moab (all versions).



108
109
110
111
112
113
114
115
116
117
118
119
 
# File 'lib/moab/storage_repository.rb', line 108

def object_size(object_id, include_deposit=false)
  storage_pathname = find_storage_object(object_id, include_deposit).object_pathname
  size = 0
  Find.find(storage_pathname) do |path|
    if FileTest.directory?(path)
      Find.prune if File.basename(path)[0] == '.'
    else
      size += FileTest.size(path)
    end
  end
  size
end

#storage_branch(object_id) ⇒ String

Returns The branch segment of the object storage path.

Returns:

  • (String) —

    The branch segment of the object storage path



43
44
45
46
47
48
49
 
# File 'lib/moab/storage_repository.rb', line 43

def storage_branch(object_id)
  #todo This method should be customized, or overridden in a subclass
  # split a object ID into 2-character segments, followed by a copy of the object ID
  # for a more sophisticated pairtree implementation see https://github.com/microservices/pairtree
  path_segments = object_id.scan(/..?/) << object_id
  path_segments.join(File::SEPARATOR).gsub(/:/,'_')
end

#storage_object(object_id, create = false) ⇒ StorageObject

Returns The representation of a digitial object’s storage directory, which must exist.

Parameters:

  • object_id (String) —

    The identifier of the digital object whose version is desired

  • create (Boolean) (defaults to: false) —

    If true, the object home directory should be created if it does not exist

Returns:

  • (StorageObject) —

    The representation of a digitial object’s storage directory, which must exist.



124
125
126
127
128
129
130
131
132
133
134
 
# File 'lib/moab/storage_repository.rb', line 124

def storage_object(object_id, create=false)
storage_object = find_storage_object(object_id)
unless storage_object.object_pathname.exist?
  if create
      storage_object.object_pathname.mkpath
    else
      raise Moab::ObjectNotFoundException, "No storage object found for #{object_id}"
  end
end
storage_object
end

#storage_roots ⇒ Array<Pathname>

Returns The list of filesystem root paths in which objects are stored.

Returns:

  • (Array<Pathname>) —

    The list of filesystem root paths in which objects are stored



22
23
24
25
26
27
28
29
30
 
# File 'lib/moab/storage_repository.rb', line 22

def storage_roots
  unless defined?(@storage_roots)
    raise "Moab::Config.storage_roots not found in config file" if Moab::Config.storage_roots.nil?
    @storage_roots = [Moab::Config.storage_roots].flatten.collect{|filesystem| Pathname(filesystem)}
    raise "Moab::Config.storage_roots empty" if  @storage_roots.empty?
    @storage_roots.each{|root| raise "Storage root #{root} not found on system" unless root.exist?}
  end
  @storage_roots
end

#storage_trunk ⇒ String

Returns The trunk segment of the object storage path.

Returns:

  • (String) —

    The trunk segment of the object storage path



33
34
35
36
37
38
39
 
# File 'lib/moab/storage_repository.rb', line 33

def storage_trunk
  unless defined?(@storage_trunk)
    raise "oab::Config.storage_trunk not found in config file" if Moab::Config.storage_trunk.nil?
    @storage_trunk  = Moab::Config.storage_trunk
  end
  @storage_trunk
end

#store_new_object_version(druid, bag_pathname) ⇒ void

This method returns an undefined value.

Returns transfer the object to the preservation repository.

Parameters:

  • druid (String) —

    The object identifier



139
140
141
142
143
 
# File 'lib/moab/storage_repository.rb', line 139

def store_new_object_version(druid, bag_pathname )
  storage_object = self.storage_object(druid, create=true)
  new_version = storage_object.ingest_bag(bag_pathname)
  new_version
end