Class: Robots::SdrRepo::SdrIngest::TransferObject

Inherits:
Robots::SdrRepo::SdrRobot show all
Defined in:
lib/sdr_ingest/transfer_object.rb

Overview

Robot for transferring objects from the DOR export area to the SDR deposit area.

Instance Method Summary collapse

Methods inherited from Robots::SdrRepo::SdrRobot

#create_workflow_rows, #get_workflow_status, #get_workflow_xml, #process_item, #transmit, #update_workflow_error_status, #update_workflow_status

Constructor Details

#initialize(opts = {}) ⇒ TransferObject

set workflow name, step name, log location, log severity level


16
17
18
# File 'lib/sdr_ingest/transfer_object.rb', line 16

def initialize(opts = {})
  super(self.class.workflow_name, self.class.step_name, opts)
end

Instance Method Details

#cleanup_deposit_files(druid, bag_pathname) ⇒ Boolean

Returns Cleanup the temp deposit files, raising an error if cleanup failes after 3 attempts

Parameters:

  • druid (String)

    The object identifier

  • bag_pathname (Object)

    The temp location of the bag containing the object version being deposited

Returns:

  • (Boolean)

    Cleanup the temp deposit files, raising an error if cleanup failes after 3 attempts


102
103
104
105
106
107
108
109
110
111
112
113
# File 'lib/sdr_ingest/transfer_object.rb', line 102

def cleanup_deposit_files(druid, bag_pathname)
  # retry up to 3 times
  tries ||= 3
  bag_pathname.rmtree
  return true
rescue StandardError => e
  if (tries -= 1) > 0
    retry
  else
    raise ItemError.new("Failed cleanup deposit (3 attempts)")
  end
end

#get_deposit_home(druid) ⇒ Object


87
88
89
90
91
92
93
94
95
96
97
# File 'lib/sdr_ingest/transfer_object.rb', line 87

def get_deposit_home(druid)
  deposit_pathname = Replication::SdrObject.new(druid).deposit_bag_pathname
  deposit_home = deposit_pathname.parent
  LyberCore::Log.debug("deposit bag_pathname is : #{deposit_pathname}")
  if deposit_pathname.exist?
    cleanup_deposit_files(druid, deposit_pathname)
  else
    deposit_home.mkpath
  end
  deposit_home
end

#perform(druid) ⇒ void

This method returns an undefined value.

Returns process an object from the queue through this robot See LyberCore::Robot#work

Parameters:

  • druid (String)

    The item to be processed


23
24
25
26
# File 'lib/sdr_ingest/transfer_object.rb', line 23

def perform(druid)
  LyberCore::Log.debug("( #{__FILE__} : #{__LINE__} ) Enter perform")
  transfer_object(druid)
end

#tarpipe_command(druid, deposit_home) ⇒ Object

ssh [email protected] “tar -cf - srcdir | tar -C destdir -xf - SSH authentication is by ssh public/private key pairs (see .ssh/authorized_keys on export host). Note that symbolic links from /dor/export to /dor/workspace get translated into real files by use of –dereference


119
120
121
122
123
124
125
# File 'lib/sdr_ingest/transfer_object.rb', line 119

def tarpipe_command(druid, deposit_home)
  'ssh ' + Sdr::Config.ingest_transfer. +
      ' "tar -C ' + Sdr::Config.ingest_transfer.export_dir +
      ' --dereference -cf - ' + druid.sub('druid:', '') +
      ' " | tar -C ' + deposit_home.to_s +
      ' -xf -'
end

#transfer_object(druid) ⇒ void

This method returns an undefined value.

Returns Transfer and untar the object from the DOR export area to the SDR deposit area. Note: POSIX tar has a limit of 100 chars in a filename

some implementations of gnu TAR work around this by adding a ././@LongLink file containing the full name
See: http://www.delorie.com/gnu/docs/tar/tar_114.html
 http://stackoverflow.com/questions/2078778/what-exactly-is-the-gnu-tar-longlink-trick
 http://www.gnu.org/software/tar/manual/html_section/Portability.html

Also, beware of incompatabilities between BSD tar and other TAR formats

regarding the handling of vendor extended attributes.
See: http://xorl.wordpress.com/2012/05/15/admin-mistakes-gnu-bsd-tar-and-posix-compatibility/

Parameters:

  • druid (String)

    The object identifier

  • deposit_pathname (Pathname)

    The location of the BagIt bag being ingested


39
40
41
42
43
44
45
46
47
48
49
# File 'lib/sdr_ingest/transfer_object.rb', line 39

def transfer_object(druid)
  LyberCore::Log.debug("( #{__FILE__} : #{__LINE__} ) Enter transfer_object")
  verify_accesssion_status(druid)
  verify_dor_export(druid)
  (druid)
  deposit_home = get_deposit_home(druid)
  transfer_cmd = tarpipe_command(druid, deposit_home)
  Archive::OperatingSystem.execute(transfer_cmd)
rescue StandardError => e
  raise ItemError.new("Error transferring object")
end

#verification_files(druid) ⇒ Object


132
133
134
135
136
137
# File 'lib/sdr_ingest/transfer_object.rb', line 132

def verification_files(druid)
  deposit_bag_pathname = Replication::SdrObject.new(druid).deposit_bag_pathname
  files = []
  files << deposit_bag_pathname.to_s
  files
end

#verification_queries(druid) ⇒ Object


127
128
129
130
# File 'lib/sdr_ingest/transfer_object.rb', line 127

def verification_queries(druid)
  queries = []
  queries
end

#verify_accesssion_status(druid) ⇒ Boolean

Returns query the workflow service to ensure that accession workflow has appropriate state

Parameters:

  • druid (String)

    The object identifier

Returns:

  • (Boolean)

    query the workflow service to ensure that accession workflow has appropriate state


53
54
55
56
57
58
59
60
# File 'lib/sdr_ingest/transfer_object.rb', line 53

def verify_accesssion_status(druid)
  accession_status = get_workflow_status('dor', druid, 'accessionWF', 'sdr-ingest-transfer')
  if accession_status == 'completed'
    true
  else
    raise ItemError.new("accessionWF:sdr-ingest-transfer status is #{accession_status}")
  end
end

#verify_dor_export(druid) ⇒ Boolean

Returns query the workflow service to ensure that accession workflow has appropriate state

Parameters:

  • druid (String)

    The object identifier

Returns:

  • (Boolean)

    query the workflow service to ensure that accession workflow has appropriate state


64
65
66
67
# File 'lib/sdr_ingest/transfer_object.rb', line 64

def verify_dor_export(druid)
  vmpath = File.join(Sdr::Config.ingest_transfer.export_dir, druid.sub('druid:', ''))
  verify_dor_path(vmpath)
end

#verify_dor_path(vmpath) ⇒ Object


77
78
79
80
81
82
83
84
85
# File 'lib/sdr_ingest/transfer_object.rb', line 77

def verify_dor_path(vmpath)
  exists_cmd = "if ssh " + Sdr::Config.ingest_transfer. +
      " test -e " + vmpath + ";" + " then echo exists; else echo notfound; fi"
  if (Archive::OperatingSystem.execute(exists_cmd).chomp == 'exists')
    true
  else
    raise "#{vmpath} not found"
  end
end

#verify_version_metadata(druid) ⇒ Boolean

Returns Test existence of versionMetadata file in export. Return true if found, false if not

Parameters:

  • druid (String)

    The object identifier

Returns:

  • (Boolean)

    Test existence of versionMetadata file in export. Return true if found, false if not


71
72
73
74
75
# File 'lib/sdr_ingest/transfer_object.rb', line 71

def (druid)
  vmpath = File.join(Sdr::Config.ingest_transfer.export_dir,
                     druid.sub('druid:', ''), "/data/metadata/versionMetadata.xml")
  verify_dor_path(vmpath)
end