Class: Dor::SdrIngestService
- Inherits:
-
Object
- Object
- Dor::SdrIngestService
- Defined in:
- lib/dor/services/sdr_ingest_service.rb
Class Method Summary collapse
-
.extract_datastreams(dor_item, workspace) ⇒ Pathname
Pull all the datastreams specified in the configuration file into the workspace’s metadata directory, overwriting existing file if present.
-
.get_content_inventory(metadata_dir, druid, version_id) ⇒ Moab::FileInventory
Parse the contentMetadata and generate a new version inventory object containing a content group.
-
.get_content_metadata(metadata_dir) ⇒ String
Return the contents of the contentMetadata.xml file from the content directory.
-
.get_datastream_content(dor_item, ds_name, required) ⇒ String
Return the xml text of the specified datastream if it exists.
-
.get_metadata_file_group(metadata_dir) ⇒ Moab::FileGroup
Traverse the metadata directory and generate a metadata group.
-
.get_signature_catalog(druid) ⇒ Moab::SignatureCatalog
The catalog of all files previously ingested.
-
.get_version_inventory(metadata_dir, druid, version_id) ⇒ Moab::FileInventory
Generate and return a version inventory for the object.
-
.transfer(dor_item, agreement_id = nil) ⇒ void
Create the moab manifests, export data to a BagIt bag, kick off the SDR ingest workflow.
-
.verify_bag_structure(bag_dir) ⇒ Boolean
True if all required files exist, raises exception if not.
-
.verify_pathname(pathname) ⇒ Boolean
True if file exists, raises exception if not.
- .verify_version_id(pathname, expected, found) ⇒ Object
- .verify_version_metadata(metadata_dir, expected) ⇒ Object
-
.vmfile_version_id(pathname) ⇒ Integer
The versionId found in the last version element, or nil if missing.
Class Method Details
.extract_datastreams(dor_item, workspace) ⇒ Pathname
Returns Pull all the datastreams specified in the configuration file into the workspace’s metadata directory, overwriting existing file if present.
62 63 64 65 66 67 68 69 70 71 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 62 def self.extract_datastreams(dor_item, workspace) = Pathname.new(workspace.path('metadata',create=true)) Config.sdr.datastreams.to_hash.each_pair do |ds_name, required| ds_name = ds_name.to_s = .join("#{ds_name}.xml") = self.get_datastream_content(dor_item, ds_name, required) .open('w') { |f| f << } if end end |
.get_content_inventory(metadata_dir, druid, version_id) ⇒ Moab::FileInventory
Returns Parse the contentMetadata and generate a new version inventory object containing a content group.
130 131 132 133 134 135 136 137 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 130 def self.get_content_inventory(, druid, version_id) = () if Stanford::ContentInventory.new.inventory_from_cm(, druid, subset='preserve', version_id) else FileInventory.new(:type=>"version",:digital_object_id=>druid, :version_id=>version_id) end end |
.get_content_metadata(metadata_dir) ⇒ String
Return the contents of the contentMetadata.xml file from the content directory
141 142 143 144 145 146 147 148 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 141 def self.() = .join('contentMetadata.xml') if .exist? .read else nil end end |
.get_datastream_content(dor_item, ds_name, required) ⇒ String
Return the xml text of the specified datastream if it exists. If not found, return nil unless it is a required datastream in which case raise exception
78 79 80 81 82 83 84 85 86 87 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 78 def self.get_datastream_content(dor_item, ds_name, required) ds = (ds_name == 'relationshipMetadata' ? 'RELS-EXT' : ds_name) if dor_item.datastreams.keys.include?(ds) and not dor_item.datastreams[ds].new? return dor_item.datastreams[ds].content elsif (required == 'optional') return nil else raise "required datastream #{ds_name} not found in DOR" end end |
.get_metadata_file_group(metadata_dir) ⇒ Moab::FileGroup
Returns Traverse the metadata directory and generate a metadata group.
152 153 154 155 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 152 def self.() file_group = FileGroup.new(:group_id=>'metadata').group_from_directory() file_group end |
.get_signature_catalog(druid) ⇒ Moab::SignatureCatalog
Returns the catalog of all files previously ingested.
49 50 51 52 53 54 55 56 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 49 def self.get_signature_catalog(druid) sdr_client = Dor::Config.sdr.rest_client url = "objects/#{druid}/manifest/signatureCatalog.xml" response = sdr_client[url].get Moab::SignatureCatalog.parse(response) rescue Moab::SignatureCatalog.new(:digital_object_id => druid, :version_id => 0) end |
.get_version_inventory(metadata_dir, druid, version_id) ⇒ Moab::FileInventory
Returns Generate and return a version inventory for the object.
119 120 121 122 123 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 119 def self.get_version_inventory(, druid, version_id) version_inventory = get_content_inventory(, druid, version_id) version_inventory.groups << () version_inventory end |
.transfer(dor_item, agreement_id = nil) ⇒ void
This method returns an undefined value.
Returns Create the moab manifests, export data to a BagIt bag, kick off the SDR ingest workflow.
12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 12 def self.transfer(dor_item, agreement_id=nil) druid = dor_item.pid workspace = DruidTools::Druid.new(druid,Dor::Config.sdr.local_workspace_root) signature_catalog = get_signature_catalog(druid) new_version_id = signature_catalog.version_id + 1 = extract_datastreams(dor_item, workspace) (, new_version_id) version_inventory = get_version_inventory(, druid, new_version_id) version_addtions = signature_catalog.version_additions(version_inventory) content_addtions = version_addtions.group('content') if content_addtions.nil? or content_addtions.files.empty? content_dir = nil else new_file_list = content_addtions.path_list content_dir = workspace.find_filelist_parent('content',new_file_list) end content_group = version_inventory.group('content') unless content_group.nil? or content_group.files.empty? signature_catalog.normalize_group_signatures(content_group, content_dir) end # export the bag (in tar format) bag_dir = Pathname(Dor::Config.sdr.local_export_home).join(druid.sub('druid:','')) bagger = Moab::Bagger.new(version_inventory, signature_catalog, bag_dir) bagger.reset_bag bagger.create_bag_inventory(:depositor) bagger.deposit_group('content', content_dir) bagger.deposit_group('metadata', ) bagger.create_tagfiles verify_bag_structure(bag_dir) # Now bootstrap SDR workflow. but do not create the workflows datastream dor_item.initialize_workflow('sdrIngestWF', 'sdr', false) rescue Exception => e raise LyberCore::Exceptions::ItemError.new(druid, "Export failure", e) end |
.verify_bag_structure(bag_dir) ⇒ Boolean
Returns true if all required files exist, raises exception if not.
159 160 161 162 163 164 165 166 167 168 169 170 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 159 def self.verify_bag_structure(bag_dir) verify_pathname(bag_dir) verify_pathname(bag_dir.join('data')) verify_pathname(bag_dir.join('bagit.txt')) verify_pathname(bag_dir.join('bag-info.txt')) verify_pathname(bag_dir.join('manifest-sha256.txt')) verify_pathname(bag_dir.join('tagmanifest-sha256.txt')) verify_pathname(bag_dir.join('versionAdditions.xml')) verify_pathname(bag_dir.join('versionInventory.xml')) verify_pathname(bag_dir.join('data','metadata','versionMetadata.xml')) true end |
.verify_pathname(pathname) ⇒ Boolean
Returns true if file exists, raises exception if not.
174 175 176 177 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 174 def self.verify_pathname(pathname) raise "#{pathname.basename} not found at #{pathname}" unless pathname.exist? true end |
.verify_version_id(pathname, expected, found) ⇒ Object
100 101 102 103 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 100 def self.verify_version_id(pathname, expected, found) raise "Version mismatch in #{pathname}, expected #{expected}, found #{found}" unless (expected == found) true end |
.verify_version_metadata(metadata_dir, expected) ⇒ Object
91 92 93 94 95 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 91 def self.(, expected) vmfile = .join("versionMetadata.xml") verify_version_id(vmfile, expected, vmfile_version_id(vmfile)) true end |
.vmfile_version_id(pathname) ⇒ Integer
Returns the versionId found in the last version element, or nil if missing.
107 108 109 110 111 112 113 |
# File 'lib/dor/services/sdr_ingest_service.rb', line 107 def self.vmfile_version_id(pathname) verify_pathname(pathname) doc = Nokogiri::XML(File.open(pathname.to_s)) nodeset = doc.xpath("/versionMetadata/version") version_id = nodeset.last['versionId'] version_id.nil? ? nil : version_id.to_i end |