Class: Innodb::Page

Inherits:
Object
  • Object
show all
Defined in:
lib/innodb/page.rb

Overview

A generic class for any type of page, which handles reading the common FIL header and trailer, and can handle (via #parse) dispatching to a more specialized class depending on page type (which comes from the FIL header). A page being handled by Innodb::Page indicates that its type is not currently handled by any more specialized class.

Defined Under Namespace

Classes: Blob, FspHdrXdes, IbufBitmap, Index, Inode, Sys, SysDataDictionaryHeader, SysIbufHeader, SysRsegHeader, TrxSys, UndoLog

Constant Summary collapse

SPECIALIZED_CLASSES =

A hash of page types to specialized classes to handle them. Normally subclasses will register themselves in this list.

{}
PAGE_TYPE =

InnoDB Page Type constants from include/fil0fil.h.

{
  :ALLOCATED => {
    :value => 0,
    :description => "Freshly allocated",
    :usage => "page type field has not been initialized",
  },
  :UNDO_LOG => {
    :value => 2,
    :description => "Undo log",
    :usage => "stores previous values of modified records",
  },
  :INODE => {
    :value => 3,
    :description => "File segment inode",
    :usage => "bookkeeping for file segments",
  },
  :IBUF_FREE_LIST => {
    :value => 4,
    :description => "Insert buffer free list",
    :usage => "bookkeeping for insert buffer free space management",
  },
  :IBUF_BITMAP => {
    :value => 5,
    :description => "Insert buffer bitmap",
    :usage => "bookkeeping for insert buffer writes to be merged",
  },
  :SYS => {
    :value => 6,
    :description => "System internal",
    :usage => "used for various purposes in the system tablespace",
  },
  :TRX_SYS => {
    :value => 7,
    :description => "Transaction system header",
    :usage => "bookkeeping for the transaction system in system tablespace",
  },
  :FSP_HDR => {
    :value => 8,
    :description => "File space header",
    :usage => "header page (page 0) for each tablespace file",
  },
  :XDES => {
    :value => 9,
    :description => "Extent descriptor",
    :usage => "header page for subsequent blocks of 16,384 pages",
  },
  :BLOB => {
    :value => 10,
    :description => "Uncompressed BLOB",
    :usage => "externally-stored uncompressed BLOB column data",
  },
  :ZBLOB => {
    :value => 11,
    :description => "First compressed BLOB",
    :usage => "externally-stored compressed BLOB column data, first page",
  },
  :ZBLOB2 => {
    :value => 12,
    :description => "Subsequent compressed BLOB",
    :usage => "externally-stored compressed BLOB column data, subsequent page",
  },
  :INDEX => {
    :value => 17855,
    :description => "B+Tree index",
    :usage => "table and index data stored in B+Tree structure",
  },
}
PAGE_TYPE_BY_VALUE = 
PAGE_TYPE.inject({}) { |h, (k, v)| h[v[:value]] = k; h }

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(space, buffer, page_number = nil) ⇒ Page

Initialize a page by passing in a buffer containing the raw page contents. The buffer size should match the space’s page size.



42
43
44
45
46
47
48
49
50
51
52
53
54
 
# File 'lib/innodb/page.rb', line 42

def initialize(space, buffer, page_number=nil)
  unless space && buffer
    raise "Page can't be initialized from nil space or buffer (space: #{space}, buffer: #{buffer})"
  end

  unless space.page_size == buffer.size
    raise "Buffer size #{buffer.size} is different than space page size"
  end

  @space  = space
  @buffer = buffer
  @page_number = page_number
end

Instance Attribute Details

#space ⇒ Object (readonly)

Returns the value of attribute space.



56
57
58
 
# File 'lib/innodb/page.rb', line 56

def space
  @space
end

Class Method Details

.handle(page, space, buffer, page_number = nil) ⇒ Object

Allow the specialized class to do something that isn’t ‘new’ with this page.



36
37
38
 
# File 'lib/innodb/page.rb', line 36

def self.handle(page, space, buffer, page_number=nil)
  self.new(space, buffer, page_number)
end

.maybe_undefined(value) ⇒ Object

A helper to convert “undefined” values stored in previous and next pointers in the page header to nil.



213
214
215
 
# File 'lib/innodb/page.rb', line 213

def self.maybe_undefined(value)
  value == 4294967295 ? nil : value
end

.parse(space, buffer, page_number = nil) ⇒ Object

Load a page as a generic page in order to make the “fil” header accessible, and then attempt to hand off the page to a specialized class to be re-parsed if possible. If there is no specialized class for this type of page, return the generic object.

This could be optimized to reach into the page buffer and efficiently extract the page type in order to avoid throwing away a generic Innodb::Page object when parsing every specialized page, but this is a bit cleaner, and we’re not particularly performance sensitive.



22
23
24
25
26
27
28
29
30
31
32
33
 
# File 'lib/innodb/page.rb', line 22

def self.parse(space, buffer, page_number=nil)
  # Create a page object as a generic page.
  page = Innodb::Page.new(space, buffer, page_number)

  # If there is a specialized class available for this page type, re-create
  # the page object using that specialized class.
  if specialized_class = SPECIALIZED_CLASSES[page.type]
    page = specialized_class.handle(page, space, buffer, page_number)
  end

  page
end

Instance Method Details

#checksum ⇒ Object

A helper function to return the checksum from the “fil” header, for easier access.



249
250
251
 
# File 'lib/innodb/page.rb', line 249

def checksum
  fil_header[:checksum]
end

#checksum_crc32 ⇒ Object

Calculate the checksum of the page using the CRC32c algorithm.



356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
 
# File 'lib/innodb/page.rb', line 356

def checksum_crc32
  unless size == 16384
    raise "Checksum calculation is only supported for 16 KiB pages"
  end

  @checksum_crc32 ||= begin
    # Calculate the CRC32c of the page header.
    crc_partial_header = Digest::CRC32c.new
    each_page_header_byte_as_uint8 do |byte|
      crc_partial_header << byte.chr
    end

    # Calculate the CRC32c of the page body.
    crc_page_body = Digest::CRC32c.new
    each_page_body_byte_as_uint8 do |byte|
      crc_page_body << byte.chr
    end

    # Bitwise XOR the two checksums together.
    crc_partial_header.checksum ^ crc_page_body.checksum
  end
end

#checksum_crc32? ⇒ Boolean

Returns:

  • (Boolean) 


379
380
381
 
# File 'lib/innodb/page.rb', line 379

def checksum_crc32?
  checksum == checksum_crc32
end

#checksum_innodb ⇒ Object

Calculate the checksum of the page using InnoDB’s algorithm.



334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
 
# File 'lib/innodb/page.rb', line 334

def checksum_innodb
  unless size == 16384
    raise "Checksum calculation is only supported for 16 KiB pages"
  end

  @checksum_innodb ||= begin
    # Calculate the InnoDB checksum of the page header.
    c_partial_header = Innodb::Checksum.fold_enumerator(each_page_header_byte_as_uint8)

    # Calculate the InnoDB checksum of the page body.
    c_page_body = Innodb::Checksum.fold_enumerator(each_page_body_byte_as_uint8)

    # Add the two checksums together, and mask the result back to 32 bits.
    (c_partial_header + c_page_body) & Innodb::Checksum::MAX
  end
end

#checksum_innodb? ⇒ Boolean

Returns:

  • (Boolean) 


351
352
353
 
# File 'lib/innodb/page.rb', line 351

def checksum_innodb?
  checksum == checksum_innodb
end

#checksum_invalid? ⇒ Boolean

Is the page checksum incorrect?

Returns:

  • (Boolean) 


389
390
391
 
# File 'lib/innodb/page.rb', line 389

def checksum_invalid?
  !checksum_valid?
end

#checksum_trailer ⇒ Object

A helper function to return the checksum from the “fil” trailer, for easier access.



255
256
257
 
# File 'lib/innodb/page.rb', line 255

def checksum_trailer
  fil_trailer[:checksum]
end

#checksum_type ⇒ Object 



393
394
395
396
397
398
399
400
 
# File 'lib/innodb/page.rb', line 393

def checksum_type
  case
  when checksum_crc32?
    :crc32
  when checksum_innodb?
    :innodb
  end
end

#checksum_valid? ⇒ Boolean

Is the page checksum correct?

Returns:

  • (Boolean) 


384
385
386
 
# File 'lib/innodb/page.rb', line 384

def checksum_valid?
  checksum_crc32? || checksum_innodb?
end

#corrupt? ⇒ Boolean

Is the page corrupt, either due to data corruption, tearing, or in the wrong place?

Returns:

  • (Boolean) 


432
433
434
 
# File 'lib/innodb/page.rb', line 432

def corrupt?
  checksum_invalid? || torn? || misplaced?
end

#cursor(buffer_offset) ⇒ Object

If no block is passed, return an BufferCursor object positioned at a specific offset. If a block is passed, create a cursor at the provided offset and yield it to the provided block one time, and then return the return value of the block.



79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/innodb/page.rb', line 79

def cursor(buffer_offset)
  new_cursor = BufferCursor.new(@buffer, buffer_offset)
  new_cursor.push_name("space[#{space.name}]")
  new_cursor.push_name("page[#{name}]")

  if block_given?
    # Call the block once and return its return value.
    yield new_cursor
  else
    # Return the cursor itself.
    new_cursor
  end
end

#dump ⇒ Object

Dump the contents of a page for debugging purposes.



481
482
483
484
485
486
487
488
489
490
491
492
 
# File 'lib/innodb/page.rb', line 481

def dump
  puts "#{self}:"
  puts

  puts "fil header:"
  pp fil_header
  puts

  puts "fil trailer:"
  pp fil_trailer
  puts
end

#each_page_body_byte_as_uint8 ⇒ Object

Iterate each byte of the page body, except for the FIL header and the FIL trailer.



322
323
324
325
326
327
328
329
330
331
 
# File 'lib/innodb/page.rb', line 322

def each_page_body_byte_as_uint8
  unless block_given?
    return enum_for(:each_page_body_byte_as_uint8)
  end

  cursor(pos_page_body).
    each_byte_as_uint8(size_page_body) do |byte|
    yield byte
  end
end

#each_page_header_byte_as_uint8 ⇒ Object

Iterate each byte of the FIL header.



309
310
311
312
313
314
315
316
317
318
 
# File 'lib/innodb/page.rb', line 309

def each_page_header_byte_as_uint8
  unless block_given?
    return enum_for(:each_page_header_byte_as_uint8)
  end

  cursor(pos_partial_page_header).
    each_byte_as_uint8(size_partial_page_header) do |byte|
    yield byte
  end
end

#each_region {|{ :offset => pos_fil_header, :length => size_fil_header, :name => :fil_header, :info => "FIL Header", }| ... } ⇒ Object

Yields:



436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
 
# File 'lib/innodb/page.rb', line 436

def each_region
  unless block_given?
    return enum_for(:each_region)
  end

  yield({
    :offset => pos_fil_header,
    :length => size_fil_header,
    :name   => :fil_header,
    :info   => "FIL Header",
  })

  yield({
    :offset => pos_fil_trailer,
    :length => size_fil_trailer,
    :name   => :fil_trailer,
    :info   => "FIL Trailer",
  })

  nil
end

#fil_header ⇒ Object

Return the “fil” header from the page, which is common for all page types.



218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
 
# File 'lib/innodb/page.rb', line 218

def fil_header
  @fil_header ||= cursor(pos_fil_header).name("fil_header") do |c|
    {
      :checksum   => c.name("checksum") { c.get_uint32 },
      :offset     => c.name("offset") { c.get_uint32 },
      :prev       => c.name("prev") {
        Innodb::Page.maybe_undefined(c.get_uint32)
      },
      :next       => c.name("next") {
        Innodb::Page.maybe_undefined(c.get_uint32)
      },
      :lsn        => c.name("lsn") { c.get_uint64 },
      :type       => c.name("type") { PAGE_TYPE_BY_VALUE[c.get_uint16] },
      :flush_lsn  => c.name("flush_lsn") { c.get_uint64 },
      :space_id   => c.name("space_id") { c.get_uint32 },
    }
  end
end

#fil_trailer ⇒ Object

Return the “fil” trailer from the page, which is common for all page types.



238
239
240
241
242
243
244
245
 
# File 'lib/innodb/page.rb', line 238

def fil_trailer
  @fil_trailer ||= cursor(pos_fil_trailer).name("fil_trailer") do |c|
    {
      :checksum  => c.name("checksum") { c.get_uint32 },
      :lsn_low32 => c.name("lsn_low32") { c.get_uint32 },
    }
  end
end

#in_doublewrite_buffer? ⇒ Boolean

Is the page in the doublewrite buffer?

Returns:

  • (Boolean) 


409
410
411
 
# File 'lib/innodb/page.rb', line 409

def in_doublewrite_buffer?
  space && space.system_space? && space.doublewrite_page?(offset)
end

#inspect ⇒ Object

Implement a custom inspect method to avoid irb printing the contents of the page buffer, since it’s very large and mostly not interesting.



460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
 
# File 'lib/innodb/page.rb', line 460

def inspect
  if fil_header
    "#<%s: size=%i, space_id=%i, offset=%i, type=%s, prev=%s, next=%s, checksum_valid?=%s (%s), torn?=%s, misplaced?=%s>" % [
      self.class,
      size,
      fil_header[:space_id],
      fil_header[:offset],
      fil_header[:type],
      fil_header[:prev] || "nil",
      fil_header[:next] || "nil",
      checksum_valid?,
      checksum_type ? checksum_type : "unknown",
      torn?,
      misplaced?,
    ]
  else
    "#<#{self.class}>"
  end
end

#lsn ⇒ Object

A helper function to return the LSN from the page header, for easier access.



280
281
282
 
# File 'lib/innodb/page.rb', line 280

def lsn
  fil_header[:lsn]
end

#lsn_low32_header ⇒ Object

A helper function to return the low 32 bits of the LSN from the page header for use in comparing to the low 32 bits stored in the trailer.



286
287
288
 
# File 'lib/innodb/page.rb', line 286

def lsn_low32_header
  fil_header[:lsn] & 0xffffffff
end

#lsn_low32_trailer ⇒ Object

A helper function to return the low 32 bits of the LSN as stored in the page trailer.



292
293
294
 
# File 'lib/innodb/page.rb', line 292

def lsn_low32_trailer
  fil_trailer[:lsn_low32]
end

#misplaced? ⇒ Boolean

Is the page misplaced in the wrong file or by offset in the file?

Returns:

  • (Boolean) 


426
427
428
 
# File 'lib/innodb/page.rb', line 426

def misplaced?
  !in_doublewrite_buffer? && (misplaced_space? || misplaced_offset?)
end

#misplaced_offset? ⇒ Boolean

Is the page number stored in the header different from the page number which was supposed to be read?

Returns:

  • (Boolean) 


421
422
423
 
# File 'lib/innodb/page.rb', line 421

def misplaced_offset?
  offset != @page_number
end

#misplaced_space? ⇒ Boolean

Is the space ID stored in the header different from that of the space provided when initializing this page?

Returns:

  • (Boolean) 


415
416
417
 
# File 'lib/innodb/page.rb', line 415

def misplaced_space?
  space && (space_id != space.space_id)
end

#name ⇒ Object

Return a simple string to uniquely identify this page within the space. Be careful not to call anything which would instantiate a BufferCursor so that we can use this method in cursor initialization.



66
67
68
69
70
71
72
73
 
# File 'lib/innodb/page.rb', line 66

def name
  page_offset = BinData::Uint32be.read(@buffer.slice(4, 4))
  page_type = BinData::Uint16be.read(@buffer.slice(24, 2))
  "%i,%s" % [
    page_offset,
    PAGE_TYPE_BY_VALUE[page_type],
  ]
end

#next ⇒ Object

A helper function to return the page number of the logical next page (from the doubly-linked list from page to page) from the “fil” header, for easier access.



275
276
277
 
# File 'lib/innodb/page.rb', line 275

def next
  fil_header[:next]
end

#offset ⇒ Object

A helper function to return the page offset from the “fil” header, for easier access.



261
262
263
 
# File 'lib/innodb/page.rb', line 261

def offset
  fil_header[:offset]
end

#pos_fil_header ⇒ Object

Return the byte offset of the start of the “fil” header, which is at the beginning of the page. Included here primarily for completeness.



95
96
97
 
# File 'lib/innodb/page.rb', line 95

def pos_fil_header
  0
end

#pos_fil_trailer ⇒ Object

Return the byte offset of the start of the “fil” trailer, which is at the end of the page.



120
121
122
 
# File 'lib/innodb/page.rb', line 120

def pos_fil_trailer
  size - size_fil_trailer
end

#pos_page_body ⇒ Object

Return the position of the “body” of the page, which starts after the FIL header.



131
132
133
 
# File 'lib/innodb/page.rb', line 131

def pos_page_body
  pos_fil_header + size_fil_header
end

#pos_partial_page_header ⇒ Object

The start of the checksummed portion of the file header.



105
106
107
 
# File 'lib/innodb/page.rb', line 105

def pos_partial_page_header
  pos_fil_header + 4
end

#prev ⇒ Object

A helper function to return the page number of the logical previous page (from the doubly-linked list from page to page) from the “fil” header, for easier access.



268
269
270
 
# File 'lib/innodb/page.rb', line 268

def prev
  fil_header[:prev]
end

#size ⇒ Object

Return the page size, to eventually be able to deal with non-16kB pages.



59
60
61
 
# File 'lib/innodb/page.rb', line 59

def size
  @size ||= @buffer.size
end

#size_fil_header ⇒ Object

Return the size of the “fil” header, in bytes.



100
101
102
 
# File 'lib/innodb/page.rb', line 100

def size_fil_header
  4 + 4 + 4 + 4 + 8 + 2 + 8 + 4
end

#size_fil_trailer ⇒ Object

Return the size of the “fil” trailer, in bytes.



125
126
127
 
# File 'lib/innodb/page.rb', line 125

def size_fil_trailer
  4 + 4
end

#size_page_body ⇒ Object

Return the size of the page body, excluding the header and trailer.



136
137
138
 
# File 'lib/innodb/page.rb', line 136

def size_page_body
  size - size_fil_trailer - size_fil_header
end

#size_partial_page_header ⇒ Object

The size of the portion of the fil header that is included in the checksum. Exclude the following:

:checksum   (offset 4, size 4)
:flush_lsn  (offset 26, size 8)
:space_id   (offset 34, size 4)



114
115
116
 
# File 'lib/innodb/page.rb', line 114

def size_partial_page_header
  size_fil_header - 4 - 8 - 4
end

#space_id ⇒ Object

A helper function to return the space ID from the “fil” header, for easier access.



304
305
306
 
# File 'lib/innodb/page.rb', line 304

def space_id
  fil_header[:space_id]
end

#torn? ⇒ Boolean

Is the LSN stored in the header different from the one stored in the trailer?

Returns:

  • (Boolean) 


404
405
406
 
# File 'lib/innodb/page.rb', line 404

def torn?
  lsn_low32_header != lsn_low32_trailer
end

#type ⇒ Object

A helper function to return the page type from the “fil” header, for easier access.



298
299
300
 
# File 'lib/innodb/page.rb', line 298

def type
  fil_header[:type]
end