Class: ZipTricks::Streamer

Inherits:
Object
  • Object
show all
Defined in:
lib/zip_tricks/streamer.rb

Overview

Is used to write streamed ZIP archives into the provided IO-ish object. The output IO is never going to be rewound or seeked, so the output of this object can be coupled directly to, say, a Rack output.

Allows for splicing raw files (for "stored" entries without compression) and splicing of deflated files (for "deflated" storage mode).

For stored entries, you need to know the CRC32 (as a uint) and the filesize upfront, before the writing of the entry body starts.

For compressed entries, you need to know the bytesize of the precompressed entry as well.

Constant Summary collapse

EntryBodySizeMismatch = 
Class.new(StandardError)
InvalidOutput = 
Class.new(ArgumentError)
EFS =

Language encoding flag (EFS) bit (general purpose bit 11)

0b100000000000
DEFAULT_GP_FLAGS =

Default general purpose flags for each entry.

0b00000000000

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(stream) ⇒ Streamer

Creates a new Streamer on top of the given IO-ish object.

Parameters:

  • stream (IO) —

    the destination IO for the ZIP (should respond to tell and <<)

Raises:



38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
 
# File 'lib/zip_tricks/streamer.rb', line 38

def initialize(stream)
  raise InvalidOutput, "The stream should respond to #<<" unless stream.respond_to?(:<<)
  stream = ZipTricks::WriteAndTell.new(stream) unless stream.respond_to?(:tell) && stream.respond_to?(:advance_position_by)
  @output_stream = stream

  @state_monitor = VeryTinyStateMachine.new(:before_entry, callbacks_to=self)
  @state_monitor.permit_state :in_entry_header, :in_entry_body, :in_central_directory, :closed
  @state_monitor.permit_transition :before_entry => :in_entry_header
  @state_monitor.permit_transition :in_entry_header => :in_entry_body
  @state_monitor.permit_transition :in_entry_body => :in_entry_header
  @state_monitor.permit_transition :in_entry_body => :in_central_directory
  @state_monitor.permit_transition :in_central_directory => :closed

  @entry_set = ::Zip::EntrySet.new
end

Class Method Details

.open(stream) {|Streamer| ... } ⇒ Object

Creates a new Streamer on top of the given IO-ish object and yields it. Once the given block returns, the Streamer will have it's close method called, which will write out the central directory of the archive to the output.

Parameters:

  • stream (IO) —

    the destination IO for the ZIP (should respond to tell and <<)

Yields:

  • (Streamer) —

    the streamer that can be written to



29
30
31
32
33
 
# File 'lib/zip_tricks/streamer.rb', line 29

def self.open(stream)
  archive = new(stream)
  yield(archive)
  archive.close
end

Instance Method Details

#<<(binary_data) ⇒ Object

Writes a part of a zip entry body (actual binary data of the entry) into the output stream.

Parameters:

  • binary_data (String) —

    a String in binary encoding

Returns:

  • self



58
59
60
61
62
63
 
# File 'lib/zip_tricks/streamer.rb', line 58

def <<(binary_data)
  @state_monitor.transition_or_maintain! :in_entry_body
  @output_stream << binary_data
  @bytes_written_for_entry += binary_data.bytesize
  self
end

#add_compressed_entry(entry_name, uncompressed_size, crc32, compressed_size) ⇒ Fixnum

Writes out the local header for an entry (file in the ZIP) that is using the deflated storage model (is compressed). Once this method is called, the << method has to be called to write the actual contents of the body.

Note that the deflated body that is going to be written into the output has to be precompressed (pre-deflated) before writing it into the Streamer, because otherwise it is impossible to know it's size upfront.

Parameters:

  • entry_name (String) —

    the name of the file in the entry

  • uncompressed_size (Fixnum) —

    the size of the entry when uncompressed, in bytes

  • crc32 (Fixnum) —

    the CRC32 checksum of the entry when uncompressed

  • compressed_size (Fixnum) —

    the size of the compressed entry that is going to be written into the archive

Returns:

  • (Fixnum) —

    the offset the output IO is at after writing the entry header



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
 
# File 'lib/zip_tricks/streamer.rb', line 100

def add_compressed_entry(entry_name, uncompressed_size, crc32, compressed_size)
  @state_monitor.transition! :in_entry_header

  entry = ::Zip::Entry.new(@file_name, entry_name)
  entry.compression_method = Zip::Entry::DEFLATED
  entry.crc = crc32
  entry.size = uncompressed_size
  entry.compressed_size = compressed_size
  set_gp_flags_for_filename(entry, entry_name)

  @entry_set << entry
  entry.write_local_entry(@output_stream)
  @expected_bytes_for_entry = compressed_size
  @bytes_written_for_entry = 0
  @output_stream.tell
end

#add_stored_entry(entry_name, uncompressed_size, crc32) ⇒ Fixnum

Writes out the local header for an entry (file in the ZIP) that is using the stored storage model (is stored as-is). Once this method is called, the << method has to be called one or more times to write the actual contents of the body.

Parameters:

  • entry_name (String) —

    the name of the file in the entry

  • uncompressed_size (Fixnum) —

    the size of the entry when uncompressed, in bytes

  • crc32 (Fixnum) —

    the CRC32 checksum of the entry when uncompressed

Returns:

  • (Fixnum) —

    the offset the output IO is at after writing the entry header



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
 
# File 'lib/zip_tricks/streamer.rb', line 124

def add_stored_entry(entry_name, uncompressed_size, crc32)
  @state_monitor.transition! :in_entry_header

  entry = ::Zip::Entry.new(@file_name, entry_name)
  entry.compression_method = Zip::Entry::STORED
  entry.crc = crc32
  entry.size = uncompressed_size
  entry.compressed_size = uncompressed_size
  set_gp_flags_for_filename(entry, entry_name)
  @entry_set << entry
  entry.write_local_entry(@output_stream)
  @bytes_written_for_entry = 0
  @expected_bytes_for_entry = uncompressed_size
  @output_stream.tell
end

#close ⇒ Fixnum

Closes the archive. Writes the central directory if it has not yet been written. Switches the Streamer into a state where it can no longer be written to.

Once this method is called, the Streamer should be discarded (the ZIP archive is complete).

Returns:

  • (Fixnum) —

    the offset the output IO is at after closing the archive



160
161
162
163
164
 
# File 'lib/zip_tricks/streamer.rb', line 160

def close
  write_central_directory! unless @state_monitor.in_state?(:in_central_directory)
  @state_monitor.transition! :closed
  @output_stream.tell
end

#simulate_write(num_bytes) ⇒ Numeric

Advances the internal IO pointer to keep the offsets of the ZIP file in check. Use this if you are going to use accelerated writes to the socket (like the sendfile() call) after writing the headers, or if you just need to figure out the size of the archive.

Parameters:

  • num_bytes (Numeric) —

    how many bytes are going to be written bypassing the Streamer

Returns:

  • (Numeric) —

    position in the output stream / ZIP archive



82
83
84
85
86
87
 
# File 'lib/zip_tricks/streamer.rb', line 82

def simulate_write(num_bytes)
  @state_monitor.transition_or_maintain! :in_entry_body
  @output_stream.advance_position_by(num_bytes)
  @bytes_written_for_entry += num_bytes
  @output_stream.tell
end

#write(binary_data) ⇒ Fixnum

Writes a part of a zip entry body (actual binary data of the entry) into the output stream, and returns the number of bytes written. Is implemented to make Streamer usable with IO.copy_stream(from, to).

Parameters:

  • binary_data (String) —

    a String in binary encoding

Returns:

  • (Fixnum) —

    the number of bytes written



71
72
73
74
 
# File 'lib/zip_tricks/streamer.rb', line 71

def write(binary_data)
  self << binary_data
  binary_data.bytesize
end

#write_central_directory! ⇒ Fixnum

Writes out the global footer and the directory entry header and the global directory of the ZIP archive using the information about the entries added using add_stored_entry and add_compressed_entry.

Once this method is called, the Streamer should be discarded (the ZIP archive is complete).

Returns:

  • (Fixnum) —

    the offset the output IO is at after writing the central directory



147
148
149
150
151
152
 
# File 'lib/zip_tricks/streamer.rb', line 147

def write_central_directory!
  @state_monitor.transition! :in_central_directory
  cdir = Zip::CentralDirectory.new(@entry_set, comment = nil)
  cdir.write_to_stream(@output_stream)
  @output_stream.tell
end