Class: Moab::FileSignature

Inherits:
Serializable
  • Object
show all
Includes:
HappyMapper
Defined in:
lib/moab/file_signature.rb

Overview

Note:

Copyright © 2012 by The Board of Trustees of the Leland Stanford Junior University. All rights reserved. See LICENSE for details.

The fixity properties of a file, used to determine file content equivalence regardless of filename. Placing this data in a class by itself facilitates using file size together with the MD5 and SHA1 checksums as a single key when doing comparisons against other file instances. The Moab design assumes that this file signature is sufficiently unique to act as a comparator for determining file equality and eliminating file redundancy.

The use of signatures for a compare-by-hash mechanism introduces a miniscule (but non-zero) risk that two non-identical files will have the same checksum. While this risk is only about 1 in 1048 when using the SHA1 checksum alone, it can be reduced even further (to about 1 in 1086) if we use the MD5 and SHA1 checksums together. And we gain a bit more comfort by including a comparison of file sizes.

Finally, the “collision” risk is reduced by isolation of each digital object's file pool within an object folder, instead of in a common storage area shared by the whole repository.

Data Model

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(opts = {}) ⇒ FileSignature


50
51
52
# File 'lib/moab/file_signature.rb', line 50

def initialize(opts={})
  super(opts)
end

Instance Attribute Details

#md5String


60
# File 'lib/moab/file_signature.rb', line 60

attribute :md5, String, :on_save => Proc.new { |n| n.nil? ? "" : n.to_s }

#sha1String


64
# File 'lib/moab/file_signature.rb', line 64

attribute :sha1, String, :on_save => Proc.new { |n| n.nil? ? "" : n.to_s }

#sha256String


68
# File 'lib/moab/file_signature.rb', line 68

attribute :sha256, String, :on_save => Proc.new { |n| n.nil? ? "" : n.to_s }

#sizeInteger


56
# File 'lib/moab/file_signature.rb', line 56

attribute :size, Integer, :on_save => Proc.new { |n| n.to_s }

Class Method Details

.checksum_names_for_typeHash<Symbol,String>


180
181
182
183
184
185
186
# File 'lib/moab/file_signature.rb', line 180

def FileSignature.checksum_names_for_type
  names_for_type = Hash.new
  names_for_type[:md5] = ['MD5']
  names_for_type[:sha1] = ['SHA-1', 'SHA1']
  names_for_type[:sha256] = ['SHA-256', 'SHA256']
  names_for_type
end

.checksum_type_for_nameHash<String, Symbol>


189
190
191
192
193
194
195
196
197
# File 'lib/moab/file_signature.rb', line 189

def FileSignature.checksum_type_for_name
  type_for_name = Hash.new
  self.checksum_names_for_type.each do |type, names|
    names.each do |name|
      type_for_name[name] = type
    end
  end
  type_for_name
end

Instance Method Details

#==(other) ⇒ Object

(see #eql?)


128
129
130
# File 'lib/moab/file_signature.rb', line 128

def ==(other)
  eql?(other)
end

#checksumsHash<Symbol,String>


87
88
89
90
91
92
93
94
# File 'lib/moab/file_signature.rb', line 87

def checksums
  checksum_hash = Hash.new
  checksum_hash[:md5] = @md5
  checksum_hash[:sha1] = @sha1
  checksum_hash[:sha256] = @sha256
  checksum_hash.delete_if { |key,value| value.nil? or value.empty?}
  checksum_hash
end

#complete?Boolean


97
98
99
# File 'lib/moab/file_signature.rb', line 97

def complete?
  checksums.size == 3
end

#eql?(other) ⇒ Boolean

Returns true if self and other have comparable fixity data.


113
114
115
116
117
118
119
120
121
122
123
124
# File 'lib/moab/file_signature.rb', line 113

def eql?(other)
  return false unless (other.respond_to?(:size) && other.respond_to?(:checksums))
  return false if self.size.to_i != other.size.to_i
  self_checksums = self.checksums
  other_checksums = other.checksums
  matching_keys = self_checksums.keys & other_checksums.keys
  return false if matching_keys.size == 0
  matching_keys.each do |key|
    return false if self_checksums[key] != other_checksums[key]
  end
  true
end

#fixityHash<Symbol,String>

Returns A hash of fixity data from this signataure object


103
104
105
106
107
108
# File 'lib/moab/file_signature.rb', line 103

def fixity
  fixity_hash = Hash.new
  fixity_hash[:size] = @size.to_s
  fixity_hash.merge!(checksums)
  fixity_hash
end

#hashFixnum

Note:

The hash and eql? methods override the methods inherited from Object. These methods ensure that instances of this class can be used as Hash keys. See

Also overriden is #== so that equality tests in other contexts will also return the expected result.

Returns Compute a hash-code for the fixity value array. Two file instances with the same content will have the same hash code (and will compare using eql?).


140
141
142
# File 'lib/moab/file_signature.rb', line 140

def hash
  @size.to_i
end

#normalized_signature(pathname) ⇒ FileSignature

Returns The full signature derived from the file, unless the fixity is inconsistent with current values


168
169
170
171
172
173
174
175
176
177
# File 'lib/moab/file_signature.rb', line 168

def normalized_signature(pathname)
  sig_from_file = FileSignature.new.signature_from_file(pathname)
  if self.eql?(sig_from_file)
    # The full signature from file is consistent with current values
    return sig_from_file
  else
    # One or more of the fixity values is inconsistent, so raise an exception
    raise "Signature inconsistent between inventory and file for #{pathname}: #{self.diff(sig_from_file).inspect}"
  end
end

#set_checksum(type, value)


73
74
75
76
77
78
79
80
81
82
83
84
# File 'lib/moab/file_signature.rb', line 73

def set_checksum(type,value)
  case type.to_s.downcase.to_sym
    when :md5
      @md5 = value
    when :sha1
      @sha1 = value
    when :sha256
      @sha256 = value
    else
      raise "Unknown checksum type '#{type.to_s}'"
  end
end

#signature_from_file(pathname) ⇒ FileSignature

Returns Generate a FileSignature instance containing size and checksums for a physical file


147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
# File 'lib/moab/file_signature.rb', line 147

def signature_from_file(pathname)
  @size = pathname.size
  md5_digest = Digest::MD5.new
  sha1_digest = Digest::SHA1.new
  sha256_digest = Digest::SHA2.new(256)
  pathname.open("r") do |stream|
    while buffer = stream.read(8192)
      md5_digest.update(buffer)
      sha1_digest.update(buffer)
      sha256_digest.update(buffer)
    end
  end
  @md5 = md5_digest.hexdigest
  @sha1 = sha1_digest.hexdigest
  @sha256 = sha256_digest.hexdigest
  self
end