Class: XZ::StreamWriter

Inherits:
Stream
  • Object
show all
Defined in:
lib/xz/stream_writer.rb

Overview

An IO-like writer class for XZ-compressed data, allowing you to write uncompressed data to a stream which ends up as compressed data in a wrapped stream such as a file.

A StreamWriter object actually wraps another IO object it writes the XZ-compressed data to. Here’s an ASCII art image to demonstrate way data flows when using StreamWriter to write to a compressed file:

        +----------------+  +------------+
YOUR  =>|StreamWriter's  |=>|Wrapped IO's|=> ACTUAL
DATA  =>|internal buffers|=>|buffers     |=>  FILE
        +----------------+  +------------+

This graphic also illustrates why it is unlikely to see written data directly appear in the file on your harddisk; the data is cached at least twice before it actually gets written out. Regarding file closing that means that before you can be sure any pending data has been written to the file you have to close both the StreamWriter instance and then the wrapped IO object (in exactly that order, otherwise data loss and unexpected exceptions may occur!).

As it might be tedious to always remember the correct closing order, it’s possible to pass a filename to the ::new method. In this case, StreamWriter will open the file internally and also takes care closing it when you call the #close method.

See the io-like gem’s documentation for the IO-writing methods available for this class (although you’re probably familiar with them through Ruby’s own IO class ;-)).

Example

Together with the archive-tar-minitar gem, this library can be used to create XZ-compressed TAR archives (these commonly use a file extension of .tar.xz or rarely .txz).

XZ::StreamWriter.open("foo.tar.xz") do |txz|
  # This automatically closes txz
  Archive::Tar::Minitar.pack("foo", txz)
end

Instance Method Summary collapse

Constructor Details

#initialize(delegate, compression_level = 6, check = :crc64, extreme = false) ⇒ StreamWriter

call-seq: open(delegate, compression_level = 6, check = :crc64, extreme = false) → a_stream_writer new(delegate, compression_level = 6, check = :crc64, extreme = false) → a_stream_writer

Creates a new StreamWriter instance. The block form automatically calls the #close method when the block has finished executing.

Parameters

delegate

An IO object to write the data to or a filename

which will be opened internally. If you pass an IO,
the #close method won’t close the passed IO object;
if you passed a filename, the created internal file
of course gets closed.

The other parameters are identical to what the XZ::compress_stream method expects.

Return value

The newly created instance.

Example

# Wrap it around a file
f = File.open("data.xz")
w = XZ::StreamWriter.new(f)

# Use SHA256 as the checksum and use a higher compression level
# than the default (6)
f = File.open("data.xz")
w = XZ::StreamWriter.new(f, 8, :sha256)

# Instruct liblzma to use ultra-really-high compression
# (may take eternity)
f = File.open("data.xz")
w = XZ::StreamWriter.new(f, 9, :crc64, true)

# Passing a filename
w = XZ::StreamWriter.new("compressed_data.xz")



101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
 
# File 'lib/xz/stream_writer.rb', line 101

def initialize(delegate, compression_level = 6, check = :crc64, extreme = false)
  if delegate.respond_to?(:to_io)
    super(delegate)
  else
    @file = File.open(delegate, "wb")
    super(@file)
  end

  # Initialize the internal LZMA stream for encoding
  res = XZ::LibLZMA.lzma_easy_encoder(@lzma_stream.pointer,
                                compression_level | (extreme ? XZ::LibLZMA::LZMA_PRESET_EXTREME : 0),
                                XZ::LibLZMA::LZMA_CHECK[:"lzma_check_#{check}"])
  XZ::LZMAError.raise_if_necessary(res)

  if block_given?
    begin
      yield(self)
    ensure
      close unless closed?
    end
  end
end

Instance Method Details

#close ⇒ Object

Closes this StreamWriter instance and flushes all internal buffers. Don’t use it afterwards anymore.

Return vaule

The total number of bytes written, i.e. the size of the compressed data.

Example

w.close #=> 424

Remarks

If you passed an IO object to ::new, this method doesn’t close it, you have to do that yourself.



135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
 
# File 'lib/xz/stream_writer.rb', line 135

def close
  super

  #1. Close the current block ("file") (an XZ stream may actually include
  #   multiple compressed files, which however is not supported by
  #   this library). For this we have to tell liblzma that
  #   the next bytes we pass to it are the last bytes (by means of
  #   the FINISH action). Just that we don’t pass any new input ;-)

  output_buffer_p         = FFI::MemoryPointer.new(XZ::CHUNK_SIZE)

  # Get any pending data (LZMA_FINISH causes libzlma to flush its
  # internal buffers) and write it out to our wrapped IO.
  loop do
    @lzma_stream[:next_out]  = output_buffer_p
    @lzma_stream[:avail_out] = output_buffer_p.size

    res = XZ::LibLZMA.lzma_code(@lzma_stream.pointer, XZ::LibLZMA::LZMA_ACTION[:lzma_finish])
    XZ::LZMAError.raise_if_necessary(res)

    @delegate_io.write(output_buffer_p.read_string(output_buffer_p.size - @lzma_stream[:avail_out]))

    break unless @lzma_stream[:avail_out] == 0
  end

  #2. Close the whole XZ stream.
  res = XZ::LibLZMA.lzma_end(@lzma_stream.pointer)
  XZ::LZMAError.raise_if_necessary(res)

  #2b. If we wrapped a file automatically, close it.
  @file.close if @file

  #3. Return the number of bytes written in total.
  @lzma_stream[:total_out]
end

#pos ⇒ Object Also known as: tell

call-seq:

pos()  → an_integer
tell() → an_integer

Total number of input bytes read so far from what you supplied to any writer method.



177
178
179
 
# File 'lib/xz/stream_writer.rb', line 177

def pos
  @lzma_stream[:total_in]
end