Class: Gitlab::Ci::Trace::Checksum

Inherits:
Object
  • Object
show all
Includes:
Utils::StrongMemoize
Defined in:
lib/gitlab/ci/trace/checksum.rb

Overview

Trace::Checksum class is responsible for calculating a CRC32 checksum of an entire build trace using partial build trace chunks stored in a database.

CRC32 checksum can be easily calculated by combining partial checksums in a right order.

Then we compare CRC32 checksum provided by a GitLab Runner and expect it to be the same as the CRC32 checksum derived from partial chunks.

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Utils::StrongMemoize

#clear_memoization, #strong_memoize, #strong_memoized?

Constructor Details

#initialize(build) ⇒ Checksum

Returns a new instance of Checksum.


22
23
24
# File 'lib/gitlab/ci/trace/checksum.rb', line 22

def initialize(build)
  @build = build
end

Instance Attribute Details

#buildObject (readonly)

Returns the value of attribute build.


20
21
22
# File 'lib/gitlab/ci/trace/checksum.rb', line 20

def build
  @build
end

Instance Method Details

#chunks_countObject


92
93
94
# File 'lib/gitlab/ci/trace/checksum.rb', line 92

def chunks_count
  trace_chunks.to_a.size
end

#chunks_crc32Object


40
41
42
43
44
45
46
# File 'lib/gitlab/ci/trace/checksum.rb', line 40

def chunks_crc32
  strong_memoize(:chunks_crc32) do
    trace_chunks.reduce(0) do |crc32, chunk|
      Zlib.crc32_combine(crc32, chunk.crc32, chunk_size(chunk))
    end
  end
end

#corrupted?Boolean

Returns:

  • (Boolean)

85
86
87
88
89
90
# File 'lib/gitlab/ci/trace/checksum.rb', line 85

def corrupted?
  return false unless has_bytesize?
  return false if valid?

  state_bytesize.to_i != trace_size.to_i
end

#has_bytesize?Boolean

Returns:

  • (Boolean)

96
97
98
# File 'lib/gitlab/ci/trace/checksum.rb', line 96

def has_bytesize?
  state_bytesize.present?
end

#last_chunkObject


48
49
50
# File 'lib/gitlab/ci/trace/checksum.rb', line 48

def last_chunk
  strong_memoize(:last_chunk) { trace_chunks.max }
end

#state_bytesizeObject


73
74
75
76
77
# File 'lib/gitlab/ci/trace/checksum.rb', line 73

def state_bytesize
  strong_memoize(:state_bytesize) do
    build.pending_state&.trace_bytesize
  end
end

#state_crc32Object


32
33
34
35
36
37
38
# File 'lib/gitlab/ci/trace/checksum.rb', line 32

def state_crc32
  strong_memoize(:state_crc32) do
    ::Gitlab::Database::Consistency.with_read_consistency do
      build.pending_state&.crc32
    end
  end
end

#trace_chunksObject

Trace chunks will be persisted in a database if an object store is not configured - in that case we do not want to load entire raw data of all the chunks into memory.

We ignore `raw_data` attribute instead, and rely on internal build trace chunk database adapter to handle `ActiveModel::MissingAttributeError` exception.

Alternative solution would be separating chunk data from chunk metadata on the database level too.


64
65
66
67
68
69
70
71
# File 'lib/gitlab/ci/trace/checksum.rb', line 64

def trace_chunks
  strong_memoize(:trace_chunks) do
    ::Ci::BuildTraceChunk.with_read_consistency(build) do
      build.trace_chunks.persisted
        .select(::Ci::BuildTraceChunk.)
    end
  end
end

#trace_sizeObject


79
80
81
82
83
# File 'lib/gitlab/ci/trace/checksum.rb', line 79

def trace_size
  strong_memoize(:trace_size) do
    trace_chunks.sum { |chunk| chunk_size(chunk) }
  end
end

#valid?Boolean

Returns:

  • (Boolean)

26
27
28
29
30
# File 'lib/gitlab/ci/trace/checksum.rb', line 26

def valid?
  return false unless state_crc32.present?

  state_crc32 == chunks_crc32
end