Zstdlib - a Zstd (de)compression library binding for Ruby

zstdlib is native Ruby extension for Zstandard data compression library.

zstdlib is currently available for the MRI Ruby platform.

Why bother?

Unlike the other Zstd bindings available for Ruby, Zstdlib utilizes Zstd's native Zlib compatibility layer.

This specifically means that Zstdlib module is and will be (mostly) API-compatible with standard Ruby Zlib module and thus zstdlib can be used as a drop-in replacement for zlib: just replace Zlib with Zstdlib throughout the source code and you're in!

Streams produced by Zstdlib::Deflate can be decompressed with standard zstd utility and hence can be written to .zst or .zstd files.

Use cases

Simple string compression

require 'zstdlib'
data = Zstdlib.deflate('Hello, Zstd!')

Incremental data compression

require 'zstdlib'
zstd = Zstdlib::Deflate.new
data = String.new
data << zstd.deflate('Hello')
data << zstd.deflate('Zstd')
data << zstd.finish

Simple string decompression

require 'zstdlib'
data = Zstdlib.inflate(compressed_data)

Technical notes

zstdlib covers the following parts of zlib:

  • ZStream, Inflate and Deflate classes
  • Various Zlib:: constants (DEFAULT_COMPRESSION etc.)
  • Zlib:: exception classes (Error and its descendants)

Note that zstdlib currently omits Gzip file support with its GzipFile, GzipReader and GzipWriter classes and associated constants although this may be changed in future.

General guidelines

In order to enable Zstd (de)compression capability in existing code, simply replace zlib with zstdlib in require statements and Zlib with Zstdlib module references throughout the code.

The rest of the zlib-aware code should work unchanged.

For further information refer to documentation on zlib .

Notes on compression levels

If unsure do not pass anything to constructor in order to use DEFAULT_COMPRESSION level which works for both zlib ans zstdlib.

Contrary to zlib the NO_COMPRESSION constant in zstdlib is an equivalent to BEST_SPEED as Zstd always compresses data.

The BEST_COMPRESSION constant is adjusted to reflect Zstd-specific compression level range. Anyways this level is recommended against due to abysmal performance not to be expected from the fast state-of-the-art compression algorithm. The Zstd documentation (currently) recommends using 19 as the level for optimal best compression.

The BEST_SPEED constant remains unchanged.

Availability

zstdlib home page on bitbucket.org.

Source code and Windows-specific multi-versioned binary gems can be obtained from rubygems.org.

Release history

For user-visible changes refer to change log.

Caveats

Ruby documentation extraction tools (such as RDoc) produce incorrect documentation for zstdlib which is technically valid but contains all identifies referring to zlib. This may change in future.

Gzip support, although available in Zstd's Zlib compatibility layer, is currently disabled.

Zstd's external compression dictionaries capability is not (yet) implemented.

The end

Cheers,

Oleg A. Khlybov [email protected]