Class: Bio::MAF::ThreadedChunkReader

Inherits:
ChunkReader show all
Defined in:
lib/bio/maf/parser.rb

Overview

Variant ChunkReader using a read-ahead thread with internal queue for sequential parsing. Not useful for random-access parsing.

Only beneficial on JRuby.

Instance Attribute Summary

Attributes inherited from ChunkReader

#chunk_size, #f, #pos

Instance Method Summary collapse

Methods inherited from ChunkReader

#check_chunk_size, #read_chunk_at

Constructor Details

#initialize(f, chunk_size, buffer_size = 64) ⇒ ThreadedChunkReader

Returns a new instance of ThreadedChunkReader.



80
81
82
83
84
85
 
# File 'lib/bio/maf/parser.rb', line 80

def initialize(f, chunk_size, buffer_size=64)
  super(f, chunk_size)
  @buffer = SizedQueue.new(buffer_size)
  @eof_reached = false
  start_read_ahead
end

Instance Method Details

#read_ahead ⇒ Object

Read ahead into queue.



93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
 
# File 'lib/bio/maf/parser.rb', line 93

def read_ahead
  # n = 0
  begin
    f_pos = 0
    until f.eof?
      chunk = f.read(@chunk_size)
      @buffer << [f_pos, chunk]
      f_pos += chunk.bytesize
      # n += 1
      # if (n % 100) == 0
      #   $stderr.puts "buffer size: #{@buffer.size}"
      # end
    end
    @eof_reached = true
  rescue Exception
    @read_ahead_ex = $!
    $stderr.puts "read_ahead aborting: #{$!}"
  end
end

#read_chunk ⇒ String

Reads the next chunk of the file.

Returns:



114
115
116
117
118
119
120
121
122
123
 
# File 'lib/bio/maf/parser.rb', line 114

def read_chunk
  raise "readahead failed: #{@read_ahead_ex}" if @read_ahead_ex
  if @eof_reached && @buffer.empty?
    return nil
  else
    c_pos, chunk = @buffer.shift()
    @pos = c_pos
    return chunk
  end
end

#start_read_ahead ⇒ Object

Spawn a read-ahead thread. Called from #initialize.



88
89
90
 
# File 'lib/bio/maf/parser.rb', line 88

def start_read_ahead
  @read_thread = Thread.new { read_ahead }
end