Class: Down::ChunkedIO
- Inherits:
-
Object
- Object
- Down::ChunkedIO
- Defined in:
- lib/down/chunked_io.rb
Overview
Wraps an enumerator that yields chunks of content into an IO object. It implements some essential IO methods:
-
IO#read
-
IO#readpartial
-
IO#gets
-
IO#size
-
IO#pos
-
IO#eof?
-
IO#rewind
-
IO#close
By default the Down::ChunkedIO caches all read content into a tempfile, allowing it to be rewindable. If rewindability won’t be used, it can be disabled by setting ‘:rewindable` to false, which eliminates any disk I/O.
Any cleanup code (i.e. ensure block) that the given enumerator carries is guaranteed to get executed, either when all content has been retrieved or when Down::ChunkedIO is closed. One can also specify an ‘:on_close` callback that will also get executed in those situations.
Instance Attribute Summary collapse
-
#data ⇒ Object
Returns the value of attribute data.
-
#encoding ⇒ Object
Returns the value of attribute encoding.
-
#size ⇒ Object
Returns the value of attribute size.
Instance Method Summary collapse
-
#close ⇒ Object
Implements IO#close semantics.
-
#closed? ⇒ Boolean
Returns whether the Down::ChunkedIO has been closed.
-
#each_chunk ⇒ Object
Yields elements of the underlying enumerator.
-
#eof? ⇒ Boolean
Implements IO#eof? semantics.
-
#gets(separator_or_limit = $/, limit = nil) ⇒ Object
Implements IO#gets semantics.
-
#initialize(chunks:, size: nil, on_close: nil, data: {}, rewindable: true, encoding: nil) ⇒ ChunkedIO
constructor
A new instance of ChunkedIO.
-
#inspect ⇒ Object
Returns useful information about the Down::ChunkedIO object.
-
#pos ⇒ Object
Implements IO#pos semantics.
-
#read(length = nil, outbuf = nil) ⇒ Object
Implements IO#read semantics.
-
#readpartial(length = nil, outbuf = nil) ⇒ Object
Implements IO#readpartial semantics.
-
#rewind ⇒ Object
Implements IO#rewind semantics.
-
#rewindable? ⇒ Boolean
Returns whether the Down::ChunkedIO was specified as rewindable.
Constructor Details
#initialize(chunks:, size: nil, on_close: nil, data: {}, rewindable: true, encoding: nil) ⇒ ChunkedIO
Returns a new instance of ChunkedIO.
30 31 32 33 34 35 36 37 38 39 40 41 |
# File 'lib/down/chunked_io.rb', line 30 def initialize(chunks:, size: nil, on_close: nil, data: {}, rewindable: true, encoding: nil) @chunks = chunks @size = size @on_close = on_close @data = data @encoding = find_encoding(encoding || "binary") @rewindable = rewindable @buffer = nil @position = 0 retrieve_chunk # fetch first chunk so that we know whether the file is empty end |
Instance Attribute Details
#data ⇒ Object
Returns the value of attribute data.
28 29 30 |
# File 'lib/down/chunked_io.rb', line 28 def data @data end |
#encoding ⇒ Object
Returns the value of attribute encoding.
28 29 30 |
# File 'lib/down/chunked_io.rb', line 28 def encoding @encoding end |
#size ⇒ Object
Returns the value of attribute size.
28 29 30 |
# File 'lib/down/chunked_io.rb', line 28 def size @size end |
Instance Method Details
#close ⇒ Object
Implements IO#close semantics. Closes the Down::ChunkedIO by terminating chunk retrieval and deleting the cached content.
223 224 225 226 227 228 229 230 |
# File 'lib/down/chunked_io.rb', line 223 def close return if @closed chunks_fiber.resume(:terminate) if chunks_fiber.alive? cache.close! if cache @buffer = nil @closed = true end |
#closed? ⇒ Boolean
Returns whether the Down::ChunkedIO has been closed.
233 234 235 |
# File 'lib/down/chunked_io.rb', line 233 def closed? !!@closed end |
#each_chunk ⇒ Object
Yields elements of the underlying enumerator.
44 45 46 47 48 49 50 |
# File 'lib/down/chunked_io.rb', line 44 def each_chunk fail IOError, "closed stream" if closed? return enum_for(__method__) unless block_given? yield retrieve_chunk until chunks_depleted? end |
#eof? ⇒ Boolean
Implements IO#eof? semantics. Returns whether we’ve reached end of file. It returns true if cache is at the end and there is no more content to retrieve. Raises IOError if closed.
203 204 205 206 207 208 |
# File 'lib/down/chunked_io.rb', line 203 def eof? fail IOError, "closed stream" if closed? return false if cache && !cache.eof? @buffer.nil? && chunks_depleted? end |
#gets(separator_or_limit = $/, limit = nil) ⇒ Object
Implements IO#gets semantics. Without arguments it retrieves lines of content separated by newlines.
With ‘separator` argument it does the following:
-
if ‘separator` is a nonempty string returns chunks of content surrounded with that sequence of bytes
-
if ‘separator` is an empty string returns paragraphs of content (content delimited by two newlines)
-
if ‘separator` is nil returns all content
With ‘limit` argument returns maximum of that amount of bytes.
Returns nil if end of file is reached. Raises IOError if closed.
97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
# File 'lib/down/chunked_io.rb', line 97 def gets(separator_or_limit = $/, limit = nil) fail IOError, "closed stream" if closed? if separator_or_limit.is_a?(Integer) separator = $/ limit = separator_or_limit else separator = separator_or_limit end return read if separator.nil? separator = "\n\n" if separator.empty? begin data = readpartial(limit) until data.include?(separator) || data.bytesize == limit || eof? remaining_length = limit - data.bytesize if limit data << readpartial(remaining_length, outbuf ||= String.new) end line, extra = data.split(separator, 2) line << separator if data.include?(separator) if cache cache.pos -= extra.to_s.bytesize else @buffer = @buffer.to_s.prepend(extra.to_s) end rescue EOFError line = nil end line end |
#inspect ⇒ Object
Returns useful information about the Down::ChunkedIO object.
243 244 245 246 247 248 249 250 251 252 253 254 |
# File 'lib/down/chunked_io.rb', line 243 def inspect string = String.new string << "#<#{self.class.name}" string << " chunks=#{@chunks.inspect}" string << " size=#{size.inspect}" string << " encoding=#{encoding.inspect}" string << " data=#{data.inspect}" string << " on_close=#{@on_close.inspect}" string << " rewindable=#{@rewindable.inspect}" string << " (closed)" if closed? string << ">" end |
#pos ⇒ Object
Implements IO#pos semantics. Returns the current position of the Down::ChunkedIO.
196 197 198 |
# File 'lib/down/chunked_io.rb', line 196 def pos @position end |
#read(length = nil, outbuf = nil) ⇒ Object
Implements IO#read semantics. Without arguments it retrieves and returns all content.
With ‘length` argument returns exactly that number of bytes if they’re available.
With ‘outbuf` argument each call will return that same string object, where the value is replaced with retrieved content.
If end of file is reached, returns empty string if called without arguments, or nil if called with arguments. Raises IOError if closed.
63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 |
# File 'lib/down/chunked_io.rb', line 63 def read(length = nil, outbuf = nil) fail IOError, "closed stream" if closed? remaining_length = length begin data = readpartial(remaining_length, outbuf) data = data.dup unless outbuf remaining_length = length - data.bytesize if length rescue EOFError end until remaining_length == 0 || eof? data << readpartial(remaining_length) remaining_length = length - data.bytesize if length end data.to_s unless length && (data.nil? || data.empty?) end |
#readpartial(length = nil, outbuf = nil) ⇒ Object
Implements IO#readpartial semantics. If there is any content readily available reads from it, otherwise fetches and reads from the next chunk. It writes to and reads from the cache when needed.
Without arguments it either returns all content that’s readily available, or the next chunk. This is useful when you don’t care about the size of chunks and you want to minimize string allocations.
With ‘length` argument returns maximum of that amount of bytes.
With ‘outbuf` argument each call will return that same string object, where the value is replaced with retrieved content.
Raises EOFError if end of file is reached. Raises IOError if closed.
148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 |
# File 'lib/down/chunked_io.rb', line 148 def readpartial(length = nil, outbuf = nil) fail IOError, "closed stream" if closed? data = outbuf.clear.force_encoding(@encoding) if outbuf if cache && !cache.eof? data = cache.read(length, outbuf) data.force_encoding(@encoding) end if @buffer.nil? && (data.nil? || data.empty?) fail EOFError, "end of file reached" if chunks_depleted? @buffer = retrieve_chunk end remaining_length = data && length ? length - data.bytesize : length unless @buffer.nil? || remaining_length == 0 buffered_data = if remaining_length && remaining_length < @buffer.bytesize @buffer.byteslice(0, remaining_length) else @buffer end if data data << buffered_data else data = buffered_data end cache.write(buffered_data) if cache if buffered_data.bytesize < @buffer.bytesize @buffer = @buffer.byteslice(buffered_data.bytesize..-1) else @buffer = nil end buffered_data.clear unless buffered_data.equal?(data) end @position += data.bytesize data end |
#rewind ⇒ Object
Implements IO#rewind semantics. Rewinds the Down::ChunkedIO by rewinding the cache and setting the position to the beginning of the file. Raises IOError if closed or not rewindable.
213 214 215 216 217 218 219 |
# File 'lib/down/chunked_io.rb', line 213 def rewind fail IOError, "closed stream" if closed? fail IOError, "this Down::ChunkedIO is not rewindable" if cache.nil? cache.rewind @position = 0 end |
#rewindable? ⇒ Boolean
Returns whether the Down::ChunkedIO was specified as rewindable.
238 239 240 |
# File 'lib/down/chunked_io.rb', line 238 def rewindable? @rewindable end |