Class: Robots::SdrRepo::SdrMigration::MigrationTransfer

Inherits:
Robots::SdrRepo::SdrIngest::TransferObject show all
Defined in:
lib/sdr_migration/migration_transfer.rb

Overview

A robot for copying SDR objects from original storage location to new workspace Rsync is used, which eliminates need for an extra post-transfer validation step serverfault.com/questions/217446/reliable-file-copy-move-process-mostly-unix-linux bagit checksum manifests will be used to check fixity in migration-complete setp

Instance Method Summary collapse

Methods inherited from Robots::SdrRepo::SdrIngest::TransferObject

#cleanup_deposit_files, #get_deposit_home, #initialize, #perform, #tarpipe_command, #verify_accesssion_status, #verify_dor_export, #verify_dor_path, #verify_version_metadata

Methods inherited from Robots::SdrRepo::SdrRobot

#create_workflow_rows, #get_workflow_status, #get_workflow_xml, #initialize, #process_item, #transmit, #update_workflow_error_status, #update_workflow_status

Constructor Details

This class inherits a constructor from Robots::SdrRepo::SdrIngest::TransferObject

Instance Method Details

#date_based_location(druid, old_storage_area) ⇒ Pathname

Returns Construct a druid tree path for the specified object based on the old storage area

Parameters:

  • druid (String) —

    The object identifier

  • old_storage_area (Pathname) —

    The root directory of the storage location previously used for ingest

Returns:

  • (Pathname) —

    Construct a druid tree path for the specified object based on the old storage area

Raises:


56
57
58
59
60
61
62
63
64
65
66
67
68
69
 
# File 'lib/sdr_migration/migration_transfer.rb', line 56

def date_based_location(druid, old_storage_area)
  unless defined? @toc_hash
    # read table of contents file listing date-based paths all older objects
    @toc_hash = Hash.new
    toc_file = old_storage_area.join('deposit-complete.toc')
    toc_file.each_line do |line|
      line.chomp!
      @toc_hash[File.basename(line)] = line
    end
  end
  object_path = @toc_hash[druid]
  return old_storage_area.join(object_path) if object_path
  raise ItemError.new("No line found in deposit-complete.toc for druid #{druid}")
end

#generate_inventory_manifests(druid, deposit_bag_pathname) ⇒ Object 


98
99
100
101
102
103
 
# File 'lib/sdr_migration/migration_transfer.rb', line 98

def generate_inventory_manifests(druid, deposit_bag_pathname)
  version_inventory = get_version_inventory(druid, deposit_bag_pathname)
  version_inventory.write_xml_file(deposit_bag_pathname)
  version_additions = get_version_additions(druid, version_inventory)
  version_additions.write_xml_file(deposit_bag_pathname)
end

#get_data_group(deposit_bag_pathname, group_id) ⇒ Object 


116
117
118
119
120
 
# File 'lib/sdr_migration/migration_transfer.rb', line 116

def get_data_group(deposit_bag_pathname, group_id)
  data_pathname = deposit_bag_pathname.join('data', group_id)
  data_group = Moab::FileGroup.new(:group_id => group_id)
  data_group.group_from_directory(data_pathname)
end

#get_version_additions(druid, version_inventory) ⇒ Object 


135
136
137
138
139
 
# File 'lib/sdr_migration/migration_transfer.rb', line 135

def get_version_additions(druid, version_inventory)
  signature_catalog = Moab::SignatureCatalog.new(:digital_object_id => druid)
  version_additions = signature_catalog.version_additions(version_inventory)
  version_additions
end

#get_version_inventory(druid, deposit_bag_pathname) ⇒ Object 


105
106
107
108
109
110
111
112
113
114
 
# File 'lib/sdr_migration/migration_transfer.rb', line 105

def get_version_inventory(druid, deposit_bag_pathname)
  # Create new version inventory for version 1.
  version_inventory = Moab::FileInventory.new(:type => "version", :digital_object_id => druid, :version_id => 1)
  content_group = get_data_group(deposit_bag_pathname, 'content')
  version_inventory.groups << content_group
  upgrade_content_metadata(deposit_bag_pathname, content_group)
  metadata_group = get_data_group(deposit_bag_pathname, 'metadata')
  version_inventory.groups << metadata_group
  version_inventory
end

#locate_old_bag(druid) ⇒ Pathname

Returns Find the original ingest location of the specified object

Parameters:

  • druid (String) —

    The object identifier

Returns:

  • (Pathname) —

    Find the original ingest location of the specified object

Raises:


33
34
35
36
37
38
39
40
 
# File 'lib/sdr_migration/migration_transfer.rb', line 33

def locate_old_bag(druid)
  old_storage_area = Pathname(Sdr::Config.migration_source)
  tree_based_pathname = tree_based_location(druid, old_storage_area)
  return tree_based_pathname if tree_based_pathname && tree_based_pathname.exist?
  date_based_pathname = date_based_location(druid, old_storage_area)
  return date_based_pathname if date_based_pathname.exist?
  raise ItemError.new("No bag found for druid #{druid}")
end

#remediate_version_metadata(druid, bag_pathname) ⇒ void

This method returns an undefined value.

Returns Add a v1 versionMetadata datastream unless it already exists

Parameters:

  • druid (String) —

    The object identifier

  • bag_pathname (Pathname) —

    The location of the BagIt bag being ingested


88
89
90
91
92
93
94
95
96
 
# File 'lib/sdr_migration/migration_transfer.rb', line 88

def remediate_version_metadata(druid, bag_pathname)
  vm_pathname = bag_pathname.join('data/metadata', "versionMetadata.xml")
  unless vm_pathname.exist?
    template_pathname = Pathname("#{ROBOT_ROOT}/config/versionMetadata-template.xml")
    vm_pathname.open('w') do |vm|
      vm << template_pathname.read.sub(/druid/, druid)
    end
  end
end

#rsync_object(source_pathname, target_pathname) ⇒ void

This method returns an undefined value.

Returns Use rsync to copy the object from original location to workspace location

Parameters:

  • source_pathname (Pathname) —

    The object's original ingest location

  • target_pathname (Pathname) —

    The workspace location to which the object will be copied


74
75
76
77
78
79
80
81
82
83
 
# File 'lib/sdr_migration/migration_transfer.rb', line 74

def rsync_object(source_pathname, target_pathname)
  # for options see http://rsync.samba.org/ftp/rsync/rsync.html
  # and http://www.rsync.net/resources/howto/mac_images.html
  # trailing slash on the source path means "copy the contents of the source dir to the target dir"
  rsync_command = "rsync -qac --inplace #{source_pathname}/ #{target_pathname}/"
  Replication::OperatingSystem.execute(rsync_command)
  LyberCore::Log.debug("#{source_pathname} transferred to #{target_pathname}")
rescue StandardError => e
  raise ItemError.new("Error transferring object")
end

#transfer_object(druid, deposit_bag_pathname) ⇒ void

This method returns an undefined value.

Returns Transfer the object from the DOR export area to the SDR deposit area.

Parameters:

  • druid (String) —

    The object identifier

  • deposit_bag_pathname (Pathname) —

    The location of the BagIt bag being ingested


22
23
24
25
26
27
28
 
# File 'lib/sdr_migration/migration_transfer.rb', line 22

def transfer_object(druid, deposit_bag_pathname)
  LyberCore::Log.debug("( #{__FILE__} : #{__LINE__} ) Enter transfer_object")
  original_bag_pathname = locate_old_bag(druid)
  rsync_object(original_bag_pathname, deposit_bag_pathname)
  remediate_version_metadata(druid, deposit_bag_pathname)
  generate_inventory_manifests(druid, deposit_bag_pathname)
end

#tree_based_location(druid, old_storage_area) ⇒ Pathname

Returns Construct a druid tree path for the specified object based on the old storage area

Parameters:

  • druid (String) —

    The object identifier

  • old_storage_area (Pathname) —

    The root directory of the storage location previously used for ingest

Returns:

  • (Pathname) —

    Construct a druid tree path for the specified object based on the old storage area


45
46
47
48
49
50
51
 
# File 'lib/sdr_migration/migration_transfer.rb', line 45

def tree_based_location(druid, old_storage_area)
  if druid =~ /^(druid):([a-z]{2})(\d{3})([a-z]{2})(\d{4})$/
    old_storage_area.join($1, $2, $3, $4, $5, druid)
  else
    nil
  end
end

#upgrade_content_metadata(deposit_bag_pathname, content_group) ⇒ Object 


122
123
124
125
126
127
128
129
 
# File 'lib/sdr_migration/migration_transfer.rb', line 122

def upgrade_content_metadata(deposit_bag_pathname, content_group)
  content_metadata_pathname = deposit_bag_pathname.join('data/metadata/contentMetadata.xml')
  original_cm = content_metadata_pathname.read
  remediator = Stanford::ContentInventory.new
  remediated_cm = remediator.remediate_content_metadata(original_cm, content_group)
  write_content_metadata(remediated_cm, content_metadata_pathname)
  remediated_cm
end

#verification_files(druid) ⇒ Object 


146
147
148
149
150
151
152
 
# File 'lib/sdr_migration/migration_transfer.rb', line 146

def verification_files(druid)
  deposit_bag_pathname = Replication::SdrObject.new(druid).deposit_bag_pathname
  files = []
  files << deposit_bag_pathname.to_s
  files << deposit_bag_pathname.join("bag-info.txt").to_s
  files
end

#verification_queries(druid) ⇒ Object 


141
142
143
144
 
# File 'lib/sdr_migration/migration_transfer.rb', line 141

def verification_queries(druid)
  queries = []
  queries
end

#write_content_metadata(remediated_cm, content_metadata_pathname) ⇒ Object 


131
132
133
 
# File 'lib/sdr_migration/migration_transfer.rb', line 131

def write_content_metadata(remediated_cm, content_metadata_pathname)
  content_metadata_pathname.open('w') { |f| f << remediated_cm }
end