Class: Gitlab::Ci::Trace::Checksum

Inherits:
Object
  • Object
show all
Includes:
Utils::StrongMemoize
Defined in:
lib/gitlab/ci/trace/checksum.rb

Overview

Trace::Checksum class is responsible for calculating a CRC32 checksum of an entire build trace using partial build trace chunks stored in a database.

CRC32 checksum can be easily calculated by combining partial checksums in a right order.

Then we compare CRC32 checksum provided by a GitLab Runner and expect it to be the same as the CRC32 checksum derived from partial chunks.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(build) ⇒ Checksum

Returns a new instance of Checksum.



22
23
24
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 22

def initialize(build)
  @build = build
end

Instance Attribute Details

#build ⇒ Object (readonly)

Returns the value of attribute build.



20
21
22
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 20

def build
  @build
end

Instance Method Details

#chunks_count ⇒ Object 



92
93
94
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 92

def chunks_count
  trace_chunks.to_a.size
end

#chunks_crc32 ⇒ Object 



40
41
42
43
44
45
46
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 40

def chunks_crc32
  strong_memoize(:chunks_crc32) do
    trace_chunks.reduce(0) do |crc32, chunk|
      Zlib.crc32_combine(crc32, chunk.crc32, chunk_size(chunk))
    end
  end
end

#corrupted? ⇒ Boolean

Returns:

  • (Boolean) 


85
86
87
88
89
90
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 85

def corrupted?
  return false unless has_bytesize?
  return false if valid?

  state_bytesize.to_i != trace_size.to_i
end

#has_bytesize? ⇒ Boolean

Returns:

  • (Boolean) 


96
97
98
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 96

def has_bytesize?
  state_bytesize.present?
end

#last_chunk ⇒ Object 



48
49
50
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 48

def last_chunk
  strong_memoize(:last_chunk) { trace_chunks.max }
end

#state_bytesize ⇒ Object 



73
74
75
76
77
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 73

def state_bytesize
  strong_memoize(:state_bytesize) do
    build.pending_state&.trace_bytesize
  end
end

#state_crc32 ⇒ Object 



32
33
34
35
36
37
38
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 32

def state_crc32
  strong_memoize(:state_crc32) do
    ::Gitlab::Database::Consistency.with_read_consistency do
      build.pending_state&.crc32
    end
  end
end

#trace_chunks ⇒ Object

Trace chunks will be persisted in a database if an object store is not configured - in that case we do not want to load entire raw data of all the chunks into memory.

We ignore ‘raw_data` attribute instead, and rely on internal build trace chunk database adapter to handle `ActiveModel::MissingAttributeError` exception.

Alternative solution would be separating chunk data from chunk metadata on the database level too.



64
65
66
67
68
69
70
71
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 64

def trace_chunks
  strong_memoize(:trace_chunks) do
    ::Ci::BuildTraceChunk.with_read_consistency(build) do
      build.trace_chunks.persisted
        .select(::Ci::BuildTraceChunk.metadata_attributes)
    end
  end
end

#trace_size ⇒ Object 



79
80
81
82
83
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 79

def trace_size
  strong_memoize(:trace_size) do
    trace_chunks.sum { |chunk| chunk_size(chunk) }
  end
end

#valid? ⇒ Boolean

Returns:

  • (Boolean) 


26
27
28
29
30
 
# File 'lib/gitlab/ci/trace/checksum.rb', line 26

def valid?
  return false unless state_crc32.present?

  state_crc32 == chunks_crc32
end