Class: Kaitai::Struct::Stream

Inherits:
Object
  • Object
show all
Defined in:
lib/kaitai/struct/struct.rb

Overview

Kaitai::Struct::Stream is an implementation of Kaitai Struct stream API for Ruby. It’s implemented as a wrapper for generic IO objects.

It provides a wide variety of simple methods to read (parse) binary representations of primitive types, such as integer and floating point numbers, byte arrays and strings, and also provides stream positioning / navigation methods with unified cross-language and cross-toolkit semantics.

Typically, end users won’t access Kaitai Stream class manually, but would describe a binary structure format using .ksy language and then would use Kaitai Struct compiler to generate source code in desired target language. That code, in turn, would use this class and API to do the actual parsing job.

Defined Under Namespace

Classes: UnexpectedDataError

Constant Summary collapse

@@big_endian =

Test endianness of the platform

[0x0102].pack('s') == [0x0102].pack('n')

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(arg) ⇒ Stream

Constructs new Kaitai Stream object.

Parameters:

  • arg (String, IO) —

    if String, it will be used as byte array to read data from; if IO, if will be used literally as source of data



94
95
96
97
98
99
100
101
102
 
# File 'lib/kaitai/struct/struct.rb', line 94

def initialize(arg)
  if arg.is_a?(String)
    @_io = StringIO.new(arg)
  elsif arg.is_a?(IO)
    @_io = arg
  else
    raise TypeError.new('can be initialized with IO or String only')
  end
end

Class Method Details

.open(filename) ⇒ Object

Convenience method to create a Kaitai Stream object, opening a local file with a given filename.

Parameters:

  • filename (String) —

    local file to open



108
109
110
 
# File 'lib/kaitai/struct/struct.rb', line 108

def self.open(filename)
  self.new(File.open(filename, 'rb:ASCII-8BIT'))
end

.process_rotate_left(data, amount, group_size) ⇒ String

Performs a circular left rotation shift for a given buffer by a given amount of bits, using groups of groupSize bytes each time. Right circular rotation should be performed using this procedure with corrected amount.

Parameters:

  • data (String) —

    source data to process

  • amount (Fixnum) —

    number of bits to shift by

  • group_size (Fixnum) —

    number of bytes per group to shift

Returns:

  • (String) —

    copy of source array with requested shift applied

Raises:

  • (NotImplementedError) 


412
413
414
415
416
417
418
419
420
421
422
423
424
425
 
# File 'lib/kaitai/struct/struct.rb', line 412

def self.process_rotate_left(data, amount, group_size)
  raise NotImplementedError.new("unable to rotate group #{group_size} bytes yet") unless group_size == 1

  mask = group_size * 8 - 1
  anti_amount = -amount & mask

  # NB: actually, left bit shift (<<) in Ruby would have required
  # truncation to type_bits size (i.e. something like "& 0xff" for
  # group_size == 8), but we can skip this one, because later these
  # number would be packed with Array#pack, which will do truncation
  # anyway

  data.bytes.map { |x| (x << amount) | (x >> anti_amount) }.pack('C*')
end

.process_xor_many(data, key) ⇒ String

Performs a XOR processing with given data, XORing every byte of input with a key array, repeating key array many times, if necessary (i.e. if data array is longer than key array). Uses pure Ruby implementation suggested by [Thomas Leitner](github.com/gettalong), borrowed from github.com/fny/xorcist/blob/master/bin/benchmark

Parameters:

  • data (String) —

    data to process

  • key (String) —

    array of bytes to XOR with

Returns:

  • (String) —

    processed data



388
389
390
391
392
393
394
395
396
397
398
399
400
401
 
# File 'lib/kaitai/struct/struct.rb', line 388

def self.process_xor_many(data, key)
  out = data.dup
  kl = key.length
  ki = 0
  i = 0
  max = data.length
  while i < max
    out.setbyte(i, data.getbyte(i) ^ key.getbyte(ki))
    ki += 1
    ki = 0 if ki >= kl
    i += 1
  end
  out
end

.process_xor_one(data, key) ⇒ String

Performs a XOR processing with given data, XORing every byte of input with a single given value. Uses pure Ruby implementation suggested by [Thomas Leitner](github.com/gettalong), borrowed from github.com/fny/xorcist/blob/master/bin/benchmark

Parameters:

  • data (String) —

    data to process

  • key (Fixnum) —

    value to XOR with

Returns:

  • (String) —

    processed data



367
368
369
370
371
372
373
374
375
376
 
# File 'lib/kaitai/struct/struct.rb', line 367

def self.process_xor_one(data, key)
  out = data.dup
  i = 0
  max = data.length
  while i < max
    out.setbyte(i, data.getbyte(i) ^ key)
    i += 1
  end
  out
end

.resolve_enum(enum_map, value) ⇒ Object

Resolves value using enum: if the value is not found in the map, we’ll just use literal value per se.



432
433
434
 
# File 'lib/kaitai/struct/struct.rb', line 432

def self.resolve_enum(enum_map, value)
  enum_map[value] || value
end

Instance Method Details

#ensure_fixed_contents(len, expected) ⇒ String

Reads next len bytes from the stream and ensures that they match expected fixed byte array. If they differ, throws a UnexpectedDataError runtime exception.

Parameters:

  • len (Fixnum) —

    number of bytes to read

  • expected (String) —

    contents to be expected

Returns:

  • (String) —

    read bytes as byte array, which are guaranteed to equal to expected

Raises:



314
315
316
317
318
319
320
321
 
# File 'lib/kaitai/struct/struct.rb', line 314

def ensure_fixed_contents(len, expected)
  buf = @_io.read(len)
  actual = buf.bytes
  if actual != expected
    raise UnexpectedDataError.new(actual, expected)
  end
  buf
end

#eof? ⇒ true, false

Check if stream pointer is at the end of stream.

Returns:

  • (true, false) —

    true if we are located at the end of the stream



122
 
# File 'lib/kaitai/struct/struct.rb', line 122

def eof?; @_io.eof?; end

#pos ⇒ Fixnum

Get current position of a stream pointer.

Returns:

  • (Fixnum) —

    pointer position, number of bytes from the beginning of the stream



132
 
# File 'lib/kaitai/struct/struct.rb', line 132

def pos; @_io.pos; end

#read_bytes(n) ⇒ String

Reads designated number of bytes from the stream.

Parameters:

  • n (Fixnum) —

    number of bytes to read

Returns:

  • (String) —

    read bytes as byte array

Raises:

  • (EOFError) —

    if there were less bytes than requested available in the stream



287
288
289
290
291
292
293
294
295
296
 
# File 'lib/kaitai/struct/struct.rb', line 287

def read_bytes(n)
  r = @_io.read(n)
  if r
    rl = r.bytesize
  else
    rl = 0
  end
  raise EOFError.new("attempted to read #{n} bytes, got only #{rl}") if rl < n
  r
end

#read_bytes_full ⇒ String

Reads all the remaining bytes in a stream as byte array.

Returns:

  • (String) —

    all remaining bytes in a stream as byte array



301
302
303
 
# File 'lib/kaitai/struct/struct.rb', line 301

def read_bytes_full
  @_io.read
end

#read_f4be ⇒ Object

Big-endian



257
258
259
 
# File 'lib/kaitai/struct/struct.rb', line 257

def read_f4be
  read_bytes(4).unpack('g')[0]
end

#read_f4le ⇒ Object

Little-endian



269
270
271
 
# File 'lib/kaitai/struct/struct.rb', line 269

def read_f4le
  read_bytes(4).unpack('e')[0]
end

#read_f8be ⇒ Object 



261
262
263
 
# File 'lib/kaitai/struct/struct.rb', line 261

def read_f8be
  read_bytes(8).unpack('G')[0]
end

#read_f8le ⇒ Object 



273
274
275
 
# File 'lib/kaitai/struct/struct.rb', line 273

def read_f8le
  read_bytes(8).unpack('E')[0]
end

#read_s1 ⇒ Object

Signed



147
148
149
 
# File 'lib/kaitai/struct/struct.rb', line 147

def read_s1
  read_bytes(1).unpack('c')[0]
end

#read_s2be ⇒ Object

.….….….….….….….….….….….….….….….….….… Big-endian .….….….….….….….….….….….….….….….….….…



155
156
157
 
# File 'lib/kaitai/struct/struct.rb', line 155

def read_s2be
  to_signed(read_u2be, SIGN_MASK_16)
end

#read_s2le ⇒ Object

.….….….….….….….….….….….….….….….….….… Little-endian .….….….….….….….….….….….….….….….….….…



177
178
179
 
# File 'lib/kaitai/struct/struct.rb', line 177

def read_s2le
  to_signed(read_u2le, SIGN_MASK_16)
end

#read_s4be ⇒ Object 



159
160
161
 
# File 'lib/kaitai/struct/struct.rb', line 159

def read_s4be
  to_signed(read_u4be, SIGN_MASK_32)
end

#read_s4le ⇒ Object 



181
182
183
 
# File 'lib/kaitai/struct/struct.rb', line 181

def read_s4le
  to_signed(read_u4le, SIGN_MASK_32)
end

#read_s8be ⇒ Object 



164
165
166
 
# File 'lib/kaitai/struct/struct.rb', line 164

def read_s8be
  read_bytes(8).unpack('q')[0]
end

#read_s8le ⇒ Object 



186
187
188
 
# File 'lib/kaitai/struct/struct.rb', line 186

def read_s8le
  read_bytes(8).unpack('q')[0]
end

#read_str_byte_limit(byte_size, encoding) ⇒ Object 



331
332
333
 
# File 'lib/kaitai/struct/struct.rb', line 331

def read_str_byte_limit(byte_size, encoding)
  read_bytes(byte_size).force_encoding(encoding)
end

#read_str_eos(encoding) ⇒ Object

Strings



327
328
329
 
# File 'lib/kaitai/struct/struct.rb', line 327

def read_str_eos(encoding)
  read_bytes_full.force_encoding(encoding)
end

#read_strz(encoding, term, include_term, consume_term, eos_error) ⇒ Object 



335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
 
# File 'lib/kaitai/struct/struct.rb', line 335

def read_strz(encoding, term, include_term, consume_term, eos_error)
  r = ''
  loop {
    if @_io.eof?
      if eos_error
        raise EOFError.new("end of stream reached, but no terminator #{term} found")
      else
        return r.force_encoding(encoding)
      end
    end
    c = @_io.getc
    if c.ord == term
      r << c if include_term
      @_io.seek(@_io.pos - 1) unless consume_term
      return r.force_encoding(encoding)
    end
    r << c
  }
end

#read_u1 ⇒ Object

Unsigned



199
200
201
 
# File 'lib/kaitai/struct/struct.rb', line 199

def read_u1
  read_bytes(1).unpack('C')[0]
end

#read_u2be ⇒ Object

.….….….….….….….….….….….….….….….….….… Big-endian .….….….….….….….….….….….….….….….….….…



207
208
209
 
# File 'lib/kaitai/struct/struct.rb', line 207

def read_u2be
  read_bytes(2).unpack('n')[0]
end

#read_u2le ⇒ Object

.….….….….….….….….….….….….….….….….….… Little-endian .….….….….….….….….….….….….….….….….….…



230
231
232
 
# File 'lib/kaitai/struct/struct.rb', line 230

def read_u2le
  read_bytes(2).unpack('v')[0]
end

#read_u4be ⇒ Object 



211
212
213
 
# File 'lib/kaitai/struct/struct.rb', line 211

def read_u4be
  read_bytes(4).unpack('N')[0]
end

#read_u4le ⇒ Object 



234
235
236
 
# File 'lib/kaitai/struct/struct.rb', line 234

def read_u4le
  read_bytes(4).unpack('V')[0]
end

#read_u8be ⇒ Object 



216
217
218
 
# File 'lib/kaitai/struct/struct.rb', line 216

def read_u8be
  read_bytes(8).unpack('Q')[0]
end

#read_u8le ⇒ Object 



239
240
241
 
# File 'lib/kaitai/struct/struct.rb', line 239

def read_u8le
  read_bytes(8).unpack('Q')[0]
end

#seek(x) ⇒ Object

Set stream pointer to designated position.

Parameters:

  • x (Fixnum) —

    new position (offset in bytes from the beginning of the stream)



127
 
# File 'lib/kaitai/struct/struct.rb', line 127

def seek(x); @_io.seek(x); end

#size ⇒ Fixnum

Get total size of the stream in bytes.

Returns:

  • (Fixnum) —

    size of the stream in bytes



137
 
# File 'lib/kaitai/struct/struct.rb', line 137

def size; @_io.size; end