Class: Moab::FileGroup
- Inherits:
-
Serializer::Serializable
- Object
- Serializer::Serializable
- Moab::FileGroup
- Includes:
- HappyMapper
- Defined in:
- lib/moab/file_group.rb
Overview
Copyright © 2012 by The Board of Trustees of the Leland Stanford Junior University. All rights reserved. See LICENSE for details.
A container for a standard subset of a digital objects FileManifestation objects Used to segregate depositor content from repository metadata files This is a child element of FileInventory, which contains a full example
Data Model
-
FileInventory = container for recording information about a collection of related files
-
FileGroup [1..*] = subset allow segregation of content and metadata files
-
FileManifestation [1..*] = snapshot of a file’s filesystem characteristics
-
FileSignature [1] = file fixity information
-
FileInstance [1..*] = filepath and timestamp of any physical file having that signature
-
-
-
Instance Attribute Summary collapse
-
#base_directory ⇒ Object
Returns the value of attribute base_directory.
-
#block_count ⇒ Integer
The total disk usage (in 1 kB blocks) of all data files (estimating du -k result) (dynamically calculated).
-
#byte_count ⇒ Integer
The total size (in bytes) of all data files (dynamically calculated).
-
#data_source ⇒ String
The directory location or other source of this groups file data.
-
#file_count ⇒ Integer
The total number of data files (dynamically calculated).
-
#files ⇒ Array<FileManifestation>
The set of files comprising the group.
-
#group_id ⇒ String
The name of the file group.
-
#signature_hash ⇒ Hash<FileSignature, FileManifestation>
The actual in-memory store for the collection of FileManifestation objects that are contained in this file group.
Instance Method Summary collapse
-
#add_file(manifestation) ⇒ void
Add a single FileManifestation object to this group.
-
#add_file_instance(signature, instance) ⇒ void
Add a single FileSignature,FileInstance key/value pair to this group.
-
#add_physical_file(pathname, _validated = nil) ⇒ void
Add a single physical file’s data to the array of files in this group.
-
#group_from_bagit_subdir(directory, signatures_from_bag, recursive = true) ⇒ FileGroup
Harvest a directory (using digest hash for fixity data) and add all files to the file group.
-
#group_from_directory(directory, recursive = true) ⇒ FileGroup
Harvest a directory and add all files to the file group.
-
#harvest_directory(path, recursive, validated = nil) ⇒ void
Traverse a directory tree and add all files to the file group Note that unlike Find.find and Dir.glob, Pathname passes through symbolic links.
-
#initialize(opts = {}) ⇒ FileGroup
constructor
A new instance of FileGroup.
-
#is_descendent_of_base?(pathname) ⇒ Boolean
Test whether the given path is contained within the #base_directory.
-
#path_hash ⇒ Hash<String,FileSignature>
An index of file paths, used to test for existence of a filename in this file group.
-
#path_hash_subset(signature_subset) ⇒ Hash<String,FileSignature>
A pathname,signature hash containing a subset of the filenames in this file group.
-
#path_list ⇒ Array<String>
The list of file paths in this group.
-
#remove_file_having_path(path) ⇒ void
for example, the manifest inventory does not contain a file entry for itself.
-
#summary_fields ⇒ Array<String>
The data fields to include in summary reports.
Methods inherited from Serializer::Serializable
#array_to_hash, deep_diff, #diff, #key, #key_name, #summary, #to_hash, #to_json, #to_yaml, #variable_names, #variables
Constructor Details
#initialize(opts = {}) ⇒ FileGroup
Returns a new instance of FileGroup.
22 23 24 25 26 27 |
# File 'lib/moab/file_group.rb', line 22 def initialize(opts = {}) @signature_hash = {} @data_source = "" @signatures_from_bag = nil # prevents later warning: instance variable @signatures_from_bag not initialized super(opts) end |
Instance Attribute Details
#base_directory ⇒ Object
Returns the value of attribute base_directory.
157 158 159 |
# File 'lib/moab/file_group.rb', line 157 def base_directory @base_directory end |
#block_count ⇒ Integer
Returns The total disk usage (in 1 kB blocks) of all data files (estimating du -k result) (dynamically calculated).
55 |
# File 'lib/moab/file_group.rb', line 55 attribute :block_count, Integer, :tag => 'blockCount', :on_save => proc { |i| i.to_s } |
#byte_count ⇒ Integer
Returns The total size (in bytes) of all data files (dynamically calculated).
47 |
# File 'lib/moab/file_group.rb', line 47 attribute :byte_count, Integer, :tag => 'byteCount', :on_save => proc { |i| i.to_s } |
#data_source ⇒ String
Returns The directory location or other source of this groups file data.
35 |
# File 'lib/moab/file_group.rb', line 35 attribute :data_source, String, :tag => 'dataSource' |
#file_count ⇒ Integer
Returns The total number of data files (dynamically calculated).
39 |
# File 'lib/moab/file_group.rb', line 39 attribute :file_count, Integer, :tag => 'fileCount', :on_save => proc { |i| i.to_s } |
#files ⇒ Array<FileManifestation>
Returns The set of files comprising the group.
68 |
# File 'lib/moab/file_group.rb', line 68 has_many :files, FileManifestation, :tag => 'file' |
#group_id ⇒ String
Returns The name of the file group.
31 |
# File 'lib/moab/file_group.rb', line 31 attribute :group_id, String, :tag => 'groupId', :key => true |
#signature_hash ⇒ Hash<FileSignature, FileManifestation>
Returns The actual in-memory store for the collection of Moab::FileManifestation objects that are contained in this file group.
76 77 78 |
# File 'lib/moab/file_group.rb', line 76 def signature_hash @signature_hash end |
Instance Method Details
#add_file(manifestation) ⇒ void
This method returns an undefined value.
Returns Add a single Moab::FileManifestation object to this group.
122 123 124 125 126 |
# File 'lib/moab/file_group.rb', line 122 def add_file(manifestation) manifestation.instances.each do |instance| add_file_instance(manifestation.signature, instance) end end |
#add_file_instance(signature, instance) ⇒ void
This method returns an undefined value.
Returns Add a single Moab::FileSignature,Moab::FileInstance key/value pair to this group. Data is actually stored in the #signature_hash.
133 134 135 136 137 138 139 140 141 142 |
# File 'lib/moab/file_group.rb', line 133 def add_file_instance(signature, instance) if signature_hash.key?(signature) manifestation = signature_hash[signature] else manifestation = FileManifestation.new manifestation.signature = signature signature_hash[signature] = manifestation end manifestation.instances << instance end |
#add_physical_file(pathname, _validated = nil) ⇒ void
This method returns an undefined value.
Returns Add a single physical file’s data to the array of files in this group. If fixity data was supplied in bag manifests, then utilize that data.
220 221 222 223 224 225 226 227 228 229 230 231 232 |
# File 'lib/moab/file_group.rb', line 220 def add_physical_file(pathname, _validated = nil) pathname = Pathname.new(pathname). instance = FileInstance.new.instance_from_file(pathname, @base_directory) if @signatures_from_bag && @signatures_from_bag[pathname] signature = @signatures_from_bag[pathname] unless signature.complete? signature = signature.normalized_signature(pathname) end else signature = FileSignature.new.signature_from_file(pathname) end add_file_instance(signature, instance) end |
#group_from_bagit_subdir(directory, signatures_from_bag, recursive = true) ⇒ FileGroup
Returns Harvest a directory (using digest hash for fixity data) and add all files to the file group.
174 175 176 177 |
# File 'lib/moab/file_group.rb', line 174 def group_from_bagit_subdir(directory, signatures_from_bag, recursive = true) @signatures_from_bag = signatures_from_bag group_from_directory(directory, recursive) end |
#group_from_directory(directory, recursive = true) ⇒ FileGroup
Returns Harvest a directory and add all files to the file group.
183 184 185 186 187 188 189 190 191 |
# File 'lib/moab/file_group.rb', line 183 def group_from_directory(directory, recursive = true) self.base_directory = directory @data_source = @base_directory.to_s harvest_directory(directory, recursive) self rescue Exception # Errno::ENOENT @data_source = directory.to_s self end |
#harvest_directory(path, recursive, validated = nil) ⇒ void
This method returns an undefined value.
Returns Traverse a directory tree and add all files to the file group Note that unlike Find.find and Dir.glob, Pathname passes through symbolic links.
201 202 203 204 205 206 207 208 209 210 211 212 213 |
# File 'lib/moab/file_group.rb', line 201 def harvest_directory(path, recursive, validated = nil) pathname = Pathname.new(path). validated ||= is_descendent_of_base?(pathname) pathname.children.sort.each do |child| next if child.basename.to_s == '.DS_Store' if child.directory? harvest_directory(child, recursive, validated) if recursive else add_physical_file(child, validated) end end nil end |
#is_descendent_of_base?(pathname) ⇒ Boolean
Returns Test whether the given path is contained within the #base_directory.
162 163 164 165 166 167 168 |
# File 'lib/moab/file_group.rb', line 162 def is_descendent_of_base?(pathname) raise("base_directory has not been set") if @base_directory.nil? is_descendent = false pathname..ascend { |ancestor| is_descendent ||= (ancestor == @base_directory) } raise("#{pathname} is not a descendent of #{@base_directory}") unless is_descendent is_descendent end |
#path_hash ⇒ Hash<String,FileSignature>
Returns An index of file paths, used to test for existence of a filename in this file group.
81 82 83 84 85 86 87 88 89 |
# File 'lib/moab/file_group.rb', line 81 def path_hash path_hash = {} signature_hash.each do |signature, manifestation| manifestation.instances.each do |instance| path_hash[instance.path] = signature end end path_hash end |
#path_hash_subset(signature_subset) ⇒ Hash<String,FileSignature>
Returns A pathname,signature hash containing a subset of the filenames in this file group.
99 100 101 102 103 104 105 106 107 108 |
# File 'lib/moab/file_group.rb', line 99 def path_hash_subset(signature_subset) path_hash = {} signature_subset.each do |signature| manifestation = signature_hash[signature] manifestation.instances.each do |instance| path_hash[instance.path] = signature end end path_hash end |
#path_list ⇒ Array<String>
Returns The list of file paths in this group.
92 93 94 |
# File 'lib/moab/file_group.rb', line 92 def path_list files.collect { |file| file.instances.collect(&:path) }.flatten end |
#remove_file_having_path(path) ⇒ void
This method returns an undefined value.
for example, the manifest inventory does not contain a file entry for itself
147 148 149 150 |
# File 'lib/moab/file_group.rb', line 147 def remove_file_having_path(path) signature = path_hash[path] signature_hash.delete(signature) end |
#summary_fields ⇒ Array<String>
Returns The data fields to include in summary reports.
62 63 64 |
# File 'lib/moab/file_group.rb', line 62 def summary_fields %w[group_id file_count byte_count block_count] end |