Module: IOStreams

Defined in:
lib/iostreams.rb,
lib/io_streams/s3.rb,
lib/io_streams/pgp.rb,
lib/io_streams/utils.rb,
lib/io_streams/errors.rb,
lib/io_streams/s3/path.rb,
lib/io_streams/tabular.rb,
lib/io_streams/version.rb,
lib/io_streams/base_path.rb,
lib/io_streams/file/path.rb,
lib/io_streams/s3/reader.rb,
lib/io_streams/s3/writer.rb,
lib/io_streams/io_streams.rb,
lib/io_streams/pgp/reader.rb,
lib/io_streams/pgp/writer.rb,
lib/io_streams/row/reader.rb,
lib/io_streams/row/writer.rb,
lib/io_streams/zip/reader.rb,
lib/io_streams/zip/writer.rb,
lib/io_streams/file/reader.rb,
lib/io_streams/file/writer.rb,
lib/io_streams/gzip/reader.rb,
lib/io_streams/gzip/writer.rb,
lib/io_streams/http/reader.rb,
lib/io_streams/line/reader.rb,
lib/io_streams/line/writer.rb,
lib/io_streams/sftp/reader.rb,
lib/io_streams/sftp/writer.rb,
lib/io_streams/xlsx/reader.rb,
lib/io_streams/bzip2/reader.rb,
lib/io_streams/bzip2/writer.rb,
lib/io_streams/encode/reader.rb,
lib/io_streams/encode/writer.rb,
lib/io_streams/record/reader.rb,
lib/io_streams/record/writer.rb,
lib/io_streams/tabular/header.rb,
lib/io_streams/tabular/parser/csv.rb,
lib/io_streams/tabular/parser/psv.rb,
lib/io_streams/tabular/parser/base.rb,
lib/io_streams/tabular/parser/hash.rb,
lib/io_streams/tabular/parser/json.rb,
lib/io_streams/tabular/parser/array.rb,
lib/io_streams/tabular/parser/fixed.rb,
lib/io_streams/tabular/utility/csv_row.rb,
lib/io_streams/symmetric_encryption/reader.rb,
lib/io_streams/symmetric_encryption/writer.rb

Overview

Streaming library for Ruby

Stream types / extensions supported:

.zip       Zip File                                   [ :zip ]
.gz, .gzip GZip File                                  [ :gzip ]
.enc       File Encrypted using symmetric encryption  [ :enc ]
etc...
other      All other extensions will be returned as:  []

When a file is encrypted, it may also be compressed:

.zip.enc  [ :zip, :enc ]
.gz.enc   [ :gz,  :enc ]

Defined Under Namespace

Modules: Bzip2, Encode, Errors, File, Gzip, HTTP, Line, Pgp, Record, Row, S3, SFTP, SymmetricEncryption, Utils, Xlsx, Zip Classes: BasePath, Extension, Scheme, StreamStruct, Tabular

Constant Summary collapse

VERSION = 
'0.20.3'
UTF8_ENCODING = 
Encoding.find('UTF-8').freeze
BINARY_ENCODING = 
Encoding.find('BINARY').freeze

Class Method Summary collapse

Class Method Details

.add_root(root, *elements) ⇒ Object

Add a named root path

Raises:

  • (ArgumentError) 


484
485
486
487
488
 
# File 'lib/io_streams/io_streams.rb', line 484

def self.add_root(root, *elements)
  raise(ArgumentError, "Invalid root name #{root.inspect}") unless root.to_s =~ /\A\w+\Z/

  @roots_paths[root.to_sym] = path(*elements)
end

.compressed?(file_name) ⇒ Boolean

Returns [true|false] whether the file is compressed. Note: Currently only looks at the file name extension

Returns:

  • (Boolean) 


303
304
305
 
# File 'lib/io_streams/io_streams.rb', line 303

def self.compressed?(file_name)
  !(file_name =~ /\.(zip|gz|gzip|xls.|)\z/i).nil?
end

.copy(source_file_name_or_io, target_file_name_or_io, buffer_size: 65536, source_options: {}, target_options: {}) ⇒ Object

Copies the source file/stream to the target file/stream. Returns [Integer] the number of bytes copied

Example: Copy between 2 files

IOStreams.copy('a.csv', 'b.csv')

Example: Read content from a Xlsx file and write it out in CSV form.

IOStreams.copy('a.xlsx', 'b.csv')

Example:

# Read content from a JSON file and write it out in CSV form.
#
# The output header for the CSV file is extracted from the first row in the JSON file.
# If the first JSON row does not contain all the column names then they will be ignored
# for the rest of the file.
IOStreams.copy('a.json', 'b.csv')

Example:

# Read a PSV file and write out a CSV file from it.
IOStreams.copy('a.psv', 'b.csv')

Example:

# Copy between 2 files, encrypting the target file with Symmetric Encryption
# Since the target file_name already includes `.enc` in the filename, it is automatically
# encrypted.
IOStreams.copy('a.csv', 'b.csv.enc')

Example:

# Copy between 2 files, encrypting the target file with Symmetric Encryption
# Since the target file_name does not include `.enc` in the filename, to encrypt it
# the encryption stream is added.
IOStreams.copy('a.csv', 'b', target_options: [:enc])

Example:

# Copy between 2 files, encrypting the target file with Symmetric Encryption
# Since the target file_name does not include `.enc` in the filename, to encrypt it
# the encryption stream is added, along with the optional compression option.
IOStreams.copy('a.csv', 'b', target_options: [enc: { compress: true }])

Example:

# Create a pgp encrypted file.
# For PGP Encryption the recipients email address is required.
IOStreams.copy('a.xlsx', 'b.csv.pgp', target_options: [:csv, pgp: { recipient_email: '[email protected]' }])

Example: Copy between 2 existing streams

IOStreams.reader('a.csv') do |source_stream|
  IOStreams.writer('b.csv.enc') do |target_stream|
    IOStreams.copy(source_stream, target_stream)
  end
end

Example:

# Copy between 2 csv files, reducing the number of columns present and encrypting the
# target file with Symmetric Encryption
output_headers = %w[name address]
IOStreams.copy(
  'a.csv',
  'b.csv.enc',
  target_options: [csv:{headers: output_headers}, enc: {compress: true}]
)

Example:

# Copy a locally encrypted file to AWS S3.
# Decrypts the file, then compresses it with gzip as it is being streamed into S3.
# Useful for when the entire bucket is encrypted on S3.
IOStreams.copy('a.csv.enc', 's3://my_bucket/b.csv.gz')



277
278
279
280
281
282
283
284
285
286
287
288
289
 
# File 'lib/io_streams/io_streams.rb', line 277

def self.copy(source_file_name_or_io, target_file_name_or_io, buffer_size: 65536, source_options: {}, target_options: {})
  bytes = 0
  reader(source_file_name_or_io, **source_options) do |source_stream|
    writer(target_file_name_or_io, **target_options) do |target_stream|
      while data = source_stream.read(buffer_size)
        break if data.size == 0
        bytes += data.size
        target_stream.write(data)
      end
    end
  end
  bytes
end

.deregister_extension(extension) ⇒ Object

De-Register a file extension

Returns [Symbol] the extension removed, or nil if the extension was not registered

Example:

register_extension(:xls)

Raises:

  • (ArgumentError) 


510
511
512
513
 
# File 'lib/io_streams/io_streams.rb', line 510

def self.deregister_extension(extension)
  raise(ArgumentError, "Invalid extension #{extension.inspect}") unless extension.to_s =~ /\A\w+\Z/
  @extensions.delete(extension.to_sym)
end

.each_line(file_name_or_io, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block) ⇒ Object

Iterate over a file / stream returning one line at a time. Embedded lines (within double quotes) will be skipped if

1. The file name contains .csv
2. Or the embedded_within argument is set

Example: Supply custom options

IOStreams.each_line(file_name, embedded_within: '"') do |line|
  puts line
end



78
79
80
81
82
 
# File 'lib/io_streams/io_streams.rb', line 78

def self.each_line(file_name_or_io, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block)
  line_reader(file_name_or_io, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace, **args) do |line_stream|
    line_stream.each(&block)
  end
end

.each_record(file_name_or_io, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block) ⇒ Object

Returns [Hash] of every record in a file or stream with support for headers.

Reading a delimited stream and converting to tabular form.

Each record / line is returned one at a time so that very large files can be read without having to load the entire file into memory.

Embedded lines (within double quotes) will be skipped if

1. The file name contains .csv
2. Or the embedded_within argument is set

Example: Supply custom options

IOStreams.each_record(file_name, embedded_within: '"') do |line|
  puts line
end

Example:

file_name = 'customer_data.csv.pgp'
IOStreams.each_record(file_name) do |hash|
  p hash
end



121
122
123
124
125
 
# File 'lib/io_streams/io_streams.rb', line 121

def self.each_record(file_name_or_io, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block)
  record_reader(file_name_or_io, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace, **args) do |record_stream|
    record_stream.each(&block)
  end
end

.each_row(file_name_or_io, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block) ⇒ Object

Iterate over a file / stream returning one line at a time. Embedded lines (within double quotes) will be skipped if

1. The file name contains .csv
2. Or the embedded_within argument is set

Example: Supply custom options

IOStreams.each_row(file_name, embedded_within: '"') do |line|
  puts line
end



94
95
96
97
98
 
# File 'lib/io_streams/io_streams.rb', line 94

def self.each_row(file_name_or_io, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block)
  row_reader(file_name_or_io, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace, **args) do |row_stream|
    row_stream.each(&block)
  end
end

.encrypted?(file_name) ⇒ Boolean

Returns [true|false] whether the file is encrypted. Note: Currently only looks at the file name extension

Returns:

  • (Boolean) 


309
310
311
 
# File 'lib/io_streams/io_streams.rb', line 309

def self.encrypted?(file_name)
  !(file_name =~ /\.(enc|pgp|gpg)\z/i).nil?
end

.join(*elements, root: :default) ⇒ Object

Join the supplied path elements to a root path.

Example:

IOStreams.add_root(:default, "tmp/export")

IOStreams.join('file.xls')
# => #<IOStreams::File::Path:0x00007fec70391bd8 @path="tmp/export/sample">

IOStreams.join('file.xls').to_s
# => "tmp/export/sample"

IOStreams.join('sample', 'file.xls', root: :ftp)
# => #<IOStreams::File::Path:0x00007fec6ee329b8 @path="tmp/ftp/sample/file.xls">

IOStreams.join('sample', 'file.xls', root: :ftp).to_s
# => "tmp/ftp/sample/file.xls"

Notes:

  • Add the root path first against which this path is permitted to operate.

    `IOStreams.add_root(:default, "/usr/local/var/files")`



474
475
476
 
# File 'lib/io_streams/io_streams.rb', line 474

def self.join(*elements, root: :default)
  root(root).join(*elements)
end

.line_reader(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, embedded_within: nil, **args, &block) ⇒ Object

Iterate over a file / stream returning each record/line one at a time. It will apply the embedded_within argument if the file or input_stream contain .csv in its name.



353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
 
# File 'lib/io_streams/io_streams.rb', line 353

def self.line_reader(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, embedded_within: nil, **args, &block)

  return yield(file_name_or_io) if file_name_or_io.is_a?(IOStreams::Line::Reader) || file_name_or_io.is_a?(Array)

  # TODO: needs to be improved
  if embedded_within.nil? && file_name_or_io.is_a?(String)
    embedded_within = '"' if file_name_or_io.include?('.csv')
  elsif embedded_within.nil? && file_name
    embedded_within = '"' if file_name.include?('.csv')
  end

  reader(file_name_or_io, streams: streams, file_name: file_name, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace) do |io|
    IOStreams::Line::Reader.open(io, embedded_within: embedded_within, **args, &block)
  end
end

.line_writer(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block) ⇒ Object 



183
184
185
186
187
188
189
 
# File 'lib/io_streams/io_streams.rb', line 183

def self.line_writer(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block)
  return yield(file_name_or_io) if file_name_or_io.is_a?(IOStreams::Line::Writer) || file_name_or_io.is_a?(Array)

  writer(file_name_or_io, streams: streams, file_name: file_name, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace) do |io|
    IOStreams::Line::Writer.open(io, **args, &block)
  end
end

.path(*elements) ⇒ Object

Returns [Path] instance for the supplied complete path with optional scheme.

Example:

IOStreams.path("/usr", "local", "sample")
# => #<IOStreams::File::Path:0x00007fec66e59b60 @path="/usr/local/sample">

IOStreams.path("/usr", "local", "sample").to_s
# => "/usr/local/sample"

IOStreams.path("s3://mybucket/path/file.xls")
# => #<IOStreams::S3::Path:0x00007fec66e3a288, @path="s3://mybucket/path/file.xls">

IOStreams.path("s3://mybucket/path/file.xls").to_s
# => "s3://mybucket/path/file.xls"

IOStreams.path("file.xls")
# => #<IOStreams::File::Path:0x00007fec6be6aaf0 @path="file.xls">

IOStreams.path("files", "file.xls").to_s
# => "files/file.xls"



448
449
450
451
452
 
# File 'lib/io_streams/io_streams.rb', line 448

def self.path(*elements)
  path = ::File.join(*elements)
  uri  = URI.parse(path)
  IOStreams.scheme(uri.scheme).path_class.new(path)
end

.reader(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, &block) ⇒ Object

Returns a Reader for reading a file / stream

Parameters

file_name_or_io [String|IO]
  The file_name of the file to write to, or an IO Stream that implements
  #read.

streams [Symbol|Array]
  The formats/streams that be used to convert the data whilst it is
  being read.
  When nil, the file_name will be inspected to try and determine what
  streams should be applied.
  Default: nil

file_name [String]
  When `streams` is not supplied, `file_name` can be used for determining the streams
  to apply to read the file/stream.
  This is particularly useful when `file_name_or_io` is a stream, or a temporary file name.
  Default: nil

Example: Zip

IOStreams.reader('myfile.zip') do |stream|
  puts stream.read
end

Example: Encrypted Zip

IOStreams.reader('myfile.zip.enc') do |stream|
  puts stream.read
end

Example: Explicitly set the streams

IOStreams.reader('myfile.zip.enc', [:zip, :enc]) do |stream|
  puts stream.read
end

Example: Supply custom options

# Encrypt the file and get Symmetric Encryption to also compress it
IOStreams.reader('myfile.csv.enc', streams: enc: {compress: true}) do |stream|
  puts stream.read
end

Note:

  • Passes the file_name_or_io as-is into the block if it is already a reader stream AND no streams are passed in.



64
65
66
 
# File 'lib/io_streams/io_streams.rb', line 64

def self.reader(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, &block)
  stream(:reader, file_name_or_io, streams: streams, file_name: file_name, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace, &block)
end

.reader_stream?(file_name_or_io) ⇒ Boolean

Returns [true|false] whether the supplied file_name_or_io is a reader stream

Returns:

  • (Boolean) 


292
293
294
 
# File 'lib/io_streams/io_streams.rb', line 292

def self.reader_stream?(file_name_or_io)
  file_name_or_io.respond_to?(:read)
end

.record_reader(file_name_or_io, streams: nil, delimiter: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, embedded_within: nil, **args, &block) ⇒ Object

Iterate over a file / stream returning each line as a hash, one at a time.



399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
 
# File 'lib/io_streams/io_streams.rb', line 399

def self.record_reader(file_name_or_io,
  streams: nil,
  delimiter: nil,
  file_name: nil,
  encoding: nil,
  encode_cleaner: nil,
  encode_replace: nil,
  embedded_within: nil,
  **args,
  &block)

  return yield(file_name_or_io) if file_name_or_io.is_a?(IOStreams::Record::Reader)

  line_reader(file_name_or_io,
              streams:         streams,
              delimiter:       delimiter,
              file_name:       file_name,
              encoding:        encoding,
              encode_cleaner:  encode_cleaner,
              encode_replace:  encode_replace,
              embedded_within: embedded_within
  ) do |io|


    file_name = file_name_or_io if file_name.nil? && file_name_or_io.is_a?(String)
    IOStreams::Record::Reader.open(io, file_name: file_name, **args, &block)
  end
end

.record_writer(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block) ⇒ Object 



201
202
203
204
205
206
207
208
209
 
# File 'lib/io_streams/io_streams.rb', line 201

def self.record_writer(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block)
  return yield(file_name_or_io) if file_name_or_io.is_a?(IOStreams::Record::Writer)

  line_writer(file_name_or_io, streams: streams, file_name: file_name, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace) do |io|
    file_name = file_name_or_io if file_name.nil? && file_name_or_io.is_a?(String)

    IOStreams::Record::Writer.open(io, file_name: file_name, **args, &block)
  end
end

.register_extension(extension, reader_class, writer_class) ⇒ Object

Register a file extension and the reader and writer streaming classes

Example:

# MyXls::Reader and MyXls::Writer must implement .open
register_extension(:xls, MyXls::Reader, MyXls::Writer)

Raises:

  • (ArgumentError) 


499
500
501
502
 
# File 'lib/io_streams/io_streams.rb', line 499

def self.register_extension(extension, reader_class, writer_class)
  raise(ArgumentError, "Invalid extension #{extension.inspect}") unless extension.nil? || extension.to_s =~ /\A\w+\Z/
  @extensions[extension.nil? ? nil : extension.to_sym] = Extension.new(reader_class, writer_class)
end

.register_scheme(scheme, reader_class, writer_class, path_class = nil) ⇒ Object

Register a file extension and the reader and writer streaming classes

Example:

# MyXls::Reader and MyXls::Writer must implement .open
register_extension(:xls, MyXls::Reader, MyXls::Writer)

Raises:

  • (ArgumentError) 


520
521
522
523
 
# File 'lib/io_streams/io_streams.rb', line 520

def self.register_scheme(scheme, reader_class, writer_class, path_class = nil)
  raise(ArgumentError, "Invalid scheme #{scheme.inspect}") unless scheme.nil? || scheme.to_s =~ /\A\w+\Z/
  @schemes[scheme.nil? ? nil : scheme.to_sym] = Scheme.new(reader_class, writer_class, path_class)
end

.root(root = :default) ⇒ Object

Return named root path



479
480
481
 
# File 'lib/io_streams/io_streams.rb', line 479

def self.root(root = :default)
  @roots_paths[root.to_sym] || raise(ArgumentError, "Unknown root: #{root.inspect}")
end

.roots ⇒ Object 



490
491
492
 
# File 'lib/io_streams/io_streams.rb', line 490

def self.roots
  @roots_paths.dup
end

.row_reader(file_name_or_io, streams: nil, delimiter: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, embedded_within: nil, **args, &block) ⇒ Object

Iterate over a file / stream returning each line as an array, one at a time.



370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
 
# File 'lib/io_streams/io_streams.rb', line 370

def self.row_reader(file_name_or_io,
  streams: nil,
  delimiter: nil,
  file_name: nil,
  encoding: nil,
  encode_cleaner: nil,
  encode_replace: nil,
  embedded_within: nil,
  **args,
  &block)

  return yield(file_name_or_io) if file_name_or_io.is_a?(IOStreams::Row::Reader)

  line_reader(
    file_name_or_io,
    streams:         streams,
    delimiter:       delimiter,
    file_name:       file_name,
    encoding:        encoding,
    encode_cleaner:  encode_cleaner,
    encode_replace:  encode_replace,
    embedded_within: embedded_within
  ) do |io|
    file_name = file_name_or_io if file_name.nil? && file_name_or_io.is_a?(String)
    IOStreams::Row::Reader.open(io, file_name: file_name, **args, &block)
  end
end

.row_writer(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block) ⇒ Object 



191
192
193
194
195
196
197
198
199
 
# File 'lib/io_streams/io_streams.rb', line 191

def self.row_writer(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, **args, &block)
  return yield(file_name_or_io) if file_name_or_io.is_a?(IOStreams::Row::Writer)

  line_writer(file_name_or_io, streams: streams, file_name: file_name, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace) do |io|
    file_name = file_name_or_io if file_name.nil? && file_name_or_io.is_a?(String)

    IOStreams::Row::Writer.open(io, file_name: file_name, **args, &block)
  end
end

.scheme_for_file_name(file_name) ⇒ Object

Extract URI if any was supplied

Raises:

  • (ArgumentError) 


342
343
344
345
346
347
348
349
 
# File 'lib/io_streams/io_streams.rb', line 342

def self.scheme_for_file_name(file_name)
  raise ArgumentError.new('File name cannot be nil') if file_name.nil?
  raise ArgumentError.new("File name must be a string: #{file_name.inspect}, class: #{file_name.class}") unless file_name.is_a?(String)

  if matches = file_name.match(/\A(\w+):\/\//)
    matches[1].downcase.to_sym
  end
end

.streams_for_file_name(file_name) ⇒ Object

Returns [Array] the formats required to process the file by looking at its extension(s)

Example Zip file:

IOStreams.streams_for_file_name('myfile.zip')
=> [ :zip ]

Example Encrypted Gzip file:

IOStreams.streams_for_file_name('myfile.csv.gz.enc')
=> [ :gz, :enc ]

Example plain text / binary file:

IOStreams.streams_for_file_name('myfile.csv')
=> []

Raises:

  • (ArgumentError) 


327
328
329
330
331
332
333
334
335
336
337
338
339
 
# File 'lib/io_streams/io_streams.rb', line 327

def self.streams_for_file_name(file_name)
  raise ArgumentError.new('File name cannot be nil') if file_name.nil?
  raise ArgumentError.new("File name must be a string: #{file_name.inspect}, class: #{file_name.class}") unless file_name.is_a?(String)

  parts      = ::File.basename(file_name).split('.')
  extensions = []
  while extension = parts.pop
    sym = extension.downcase.to_sym
    break unless @extensions[sym]
    extensions.unshift(sym)
  end
  extensions
end

.writer(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, &block) ⇒ Object

Returns a Writer for writing to a file / stream

Parameters

file_name_or_io [String|IO]
  The file_name of the file to write to, or an IO Stream that implements
  #write.

streams [Symbol|Array]
  The formats/streams that be used to convert the data whilst it is
  being written.
  When nil, the file_name will be inspected to try and determine what
  streams should be applied.
  Default: nil

Stream types / extensions supported:

.zip       Zip File                                   [ :zip ]
.gz, .gzip GZip File                                  [ :gzip ]
.enc       File Encrypted using symmetric encryption  [ :enc ]
other      All other extensions will be returned as:  [ :file ]

When a file is encrypted, it may also be compressed:

.zip.enc  [ :zip, :enc ]
.gz.enc   [ :gz,  :enc ]

Example: Zip

IOStreams.writer('myfile.zip') do |stream|
  stream.write(data)
end

Example: Encrypted Zip

IOStreams.writer('myfile.zip.enc') do |stream|
  stream.write(data)
end

Example: Explicitly set the streams

IOStreams.writer('myfile.zip.enc', [:zip, :enc]) do |stream|
  stream.write(data)
end

Example: Supply custom options

IOStreams.writer('myfile.csv.enc', [enc: { compress: true }]) do |stream|
  stream.write(data)
end

Example: Set internal filename when creating a zip file

IOStreams.writer('myfile.csv.zip', zip: { zip_file_name: 'myfile.csv' }) do |stream|
  stream.write(data)
end

Note:

  • Passes the file_name_or_io as-is into the block if it is already a writer stream AND no streams are passed in.



179
180
181
 
# File 'lib/io_streams/io_streams.rb', line 179

def self.writer(file_name_or_io, streams: nil, file_name: nil, encoding: nil, encode_cleaner: nil, encode_replace: nil, &block)
  stream(:writer, file_name_or_io, streams: streams, file_name: file_name, encoding: encoding, encode_cleaner: encode_cleaner, encode_replace: encode_replace, &block)
end

.writer_stream?(file_name_or_io) ⇒ Boolean

Returns [true|false] whether the supplied file_name_or_io is a reader stream

Returns:

  • (Boolean) 


297
298
299
 
# File 'lib/io_streams/io_streams.rb', line 297

def self.writer_stream?(file_name_or_io)
  file_name_or_io.respond_to?(:write)
end