Class: ActiveStorageEncryption::ResumableGCSUpload::ByteChunker

Inherits:
Object
  • Object
show all
Defined in:
lib/active_storage_encryption/resumable_gcs_upload.rb

Overview

When doing GCP uploads the chunks need to be sized to 256KB increments, and the output that we generate is not guaranteed to be chopped up this way. Also the upload for the last chunk is done slightly different than the preceding chunks. It is convenient to have a way to “chop up” an arbitrary streaming output into evenly sized chunks.

Instance Method Summary collapse

Constructor Details

#initialize(chunk_size: 256 * 1024, &delivery_proc) ⇒ ByteChunker 



31
32
33
34
35
36
37
 
# File 'lib/active_storage_encryption/resumable_gcs_upload.rb', line 31

def initialize(chunk_size: 256 * 1024, &delivery_proc)
  @chunk_size = chunk_size.to_i
  # Use a fixed-capacity String instead of a StringIO since there are some advantages
  # to mutable strings, if a string can be reused this saves memory
  @buf_str = String.new(encoding: Encoding::BINARY, capacity: @chunk_size * 2)
  @delivery_proc = delivery_proc.to_proc
end

Instance Method Details

#<<(bin_str) ⇒ Object

Appends data to the buffer. Once the size of the chunk has been exceeded, a precisely-sized chunk will be passed to the ‘delivery_proc`



44
45
46
47
48
 
# File 'lib/active_storage_encryption/resumable_gcs_upload.rb', line 44

def <<(bin_str)
  @buf_str << bin_str.b
  deliver_buf_in_chunks
  self
end

#finish ⇒ Object

Sends the last chunk to the ‘delivery_proc` even if there is nothing output - the last request will usually be needed to close the file



64
65
66
67
68
 
# File 'lib/active_storage_encryption/resumable_gcs_upload.rb', line 64

def finish
  deliver_buf_in_chunks
  @delivery_proc.call(@buf_str, _is_last_chunk = true)
  nil
end

#write(bin_str) ⇒ Integer

Appends data to the buffer. Once the size of the chunk has been exceeded, a precisely-sized chunk will be passed to the ‘delivery_proc`



55
56
57
58
 
# File 'lib/active_storage_encryption/resumable_gcs_upload.rb', line 55

def write(bin_str)
  self << bin_str
  bin_str.bytesize
end