Class: DRbDump

Inherits:
Object
  • Object
show all
Defined in:
lib/drbdump.rb

Overview

drbdump is a tcpdump-like tool for the dRuby protocol.

Usage

The drbdump command-line utility works similarly to tcpdump. This is the easiest way to get started:

sudo drbdump

This captures DRb messages on your loopback and public interface. You can disable name resolution with -n. You can also drop root privileges with the -Z option if you don't want drbdump to run as root after it creates the capture device.

Output

drbdump reassembles TCP streams to create a complete message-send or message-result and displays it to you when complete. Here is an object in a Rinda::TupleSpace being renewed (checked if it is still alive), but broken into two lines:

17:46:27.818412 "druby://kault.local:65172" ???
                  ("druby://kault.local:63874", 70093484759080).renew()
17:46:27.818709 "druby://kault.local:65172" ???
                  "druby://kault.local:63874" success: 180

The first two lines are the message-send. The first field is the timestamp of the packet. The second is the DRb peer the messages was sent from. The rightward arrow indicates this is a message-send. The remainder is the DRb peer and object reference (7009...) the message is being sent-to along with the message (renew). If any arguments were present they would appear in the argument list.

The URIs are quoted to make it easy to copy part of the message into irb if you want to perform further debugging. For example, you can attach to the peer sending the message with:

>> sender = DRb::DRbObject.new_with_uri "druby://kault.local:65172"

You can re-send the message by copying the message from the first open parenthesis to the end of the line:

>> DRb::DRbObject.new_with("druby://kault.local:63874", 70093484759080).
     renew()

For the second two lines are the return value from the message-send. Here they are again:

17:46:27.818709 "druby://kault.local:65172" ???
                  "druby://kault.local:63874" success: 180

The fields are the timestamp, the DRb peer that sent the message and is receiving the result, the DRb peer that received the message, "success" for a non-exception result and the response value.

Unlike tcpdump drbdump always shows the peer that send the message on the left and uses the arrow to indicate the direction of the message.

Note that the message-send and its result may be separated by other messages and results, so you will need to check the port values to connect a message send to its result.

Statistics

To run drbdump in a to only display statistical information, run:

drbdump -n -q -c 10000

This disables name resolution and per-message output, collects 10,000 messages then prints statistics at exit. Depending on the diversity of messages in your application you may need to capture a different amount of packets.

On supporting operating systems (OS X, BSD) you can send a SIGINFO (control-t) to display current statistics for the basic counters at any time:

load: 0.91  cmd: ruby 31579 running 2.48u 8.64s
29664 total packets captured
71 Rinda packets received
892 DRb packets received
446 messages sent
446 results received
0 exceptions raised

These statistics are also printed when you quit drbdump.

At exit, per-message statistics are displayed including message name, the number of argument count (to help distinguish between messages with the same name and different receivers), a statistical summary of allocations required to load the message send and result objects and a statistical summary of total latency (from first packet of the message-send to last packet of the message result:

Messages sent min, avg, max, stddev:
call         (1 args) 12 sent; 3.0, 3.0, 3.0, 0.0 allocations;
                               0.214, 1.335, 6.754, 2.008 ms
each         (1 args)  6 sent; 5.0, 5.0, 5.0, 0.0 allocations;
                               0.744, 1.902, 4.771, 1.918 ms
[]           (1 args)  3 sent; 3.0, 3.0, 3.0, 0.0 allocations;
                               0.607, 1.663, 3.518, 1.612 ms
[]=          (2 args)  3 sent; 5.0, 5.0, 5.0, 0.0 allocations;
                               0.737, 0.791, 0.839, 0.051 ms
add          (1 args)  2 sent; 3.0, 3.0, 3.0, 0.0 allocations;
                               0.609, 0.651, 0.694, 0.060 ms
update       (1 args)  2 sent; 3.0, 3.0, 3.0, 0.0 allocations;
                               0.246, 0.272, 0.298, 0.037 ms
add_observer (1 args)  1 sent; 5.0, 5.0, 5.0, 0.0 allocations;
                               1.689, 1.689, 1.689, 0.000 ms
respond_to?  (2 args)  1 sent; 4.0, 4.0, 4.0, 0.0 allocations;
                               0.597, 0.597, 0.597, 0.000 ms

(The above has been line-wrapped, display output is one line per.)

This helps you determine which message-sends are causing more network traffic or are less performant overall. Some message-sends may be naturally long running (such as an enumerator that performs many message-sends to invoke its block) so a high result latency may not be indicative of a poorly-performing method.

Messages with higher numbers of allocations typically take longer to send and load and create more pressure on the garbage collector. You can change locations that call these messages to use DRb::DRbObject references to help reduce the size of the messages sent.

Switching entirely to sending references may increase latency as the remote end needs to continually ask the sender to invoke methods on its behalf.

To help determine if changes you make are causing too many messages drbdump shows the number of messages sent between peers along with the message latency:

Peers min, avg, max, stddev:
6 messages from "druby://a.example:54167" to "druby://a.example:54157"
           0.609, 1.485, 4.771, 1.621 ms
4 messages from "druby://a.example:54166" to "druby://a.example:54163"
           1.095, 2.848, 6.754, 2.645 ms
3 messages from "druby://a.example:54162" to "druby://a.example:54159"
           0.246, 0.380, 0.597, 0.189 ms
3 messages from "druby://a.example:54169" to "druby://a.example:54163"
           0.214, 0.254, 0.278, 0.035 ms
2 messages from "druby://a.example:54168" to "druby://a.example:54163"
           0.324, 0.366, 0.407, 0.059 ms
2 messages from "druby://a.example:54164" to "druby://a.example:54154"
           0.607, 0.735, 0.863, 0.181 ms
2 messages from "druby://a.example:54160" to "druby://a.example:54154"
           0.798, 2.158, 3.518, 1.923 ms
4 single-message peers 0.225, 0.668, 1.259, 0.435 ms

(The above has been line-wrapped, display output is one line per.)

To save terminal lines (the peers report can be long when many messages are captured) any single-peer results are wrapped up into a one-line aggregate.

An efficient API between peers would send the fewest messages with the fewest allocations.

Replaying packet logs

You can capture and record packets with tcpdump then replay the captured file with drbdump. To record captured packets use tcpdump -w dump_file:

$ tcpdump -i lo0 -w drb.pcap [filter]

To replay the capture with drbdump give the path to the dump file to drbdump -i:

$ drbdump -i drb.pcap

Defined Under Namespace

Classes: Error, Loader, Message, MessageResult, MessageSend, Statistic, Statistics, TestCase

Constant Summary collapse

VERSION =

The version of DRbDump you are using

'1.0'
FIN_OR_RST =

:nodoc:

Capp::TCP_FIN | Capp::TCP_RST
TIMESTAMP_FORMAT =

:nodoc:

'%H:%M:%S.%6N'

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options) ⇒ DRbDump

Creates a new DRbDump for options. The following options are understood:

:devices

An Array of devices to listen on. If the Array is empty then the default device (see Capp::default_device_name) and the loopback device are used.

:resolve_names

When true drbdump will look up address names.

:run_as_user

When set, drop privileges from root to this user after starting packet capture.

:run_as_directory

When set, chroot() to this directory after starting packet capture. Only useful with :run_as_user



381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
 
# File 'lib/drbdump.rb', line 381

def initialize options
  @count                 = options[:count] || Float::INFINITY
  @drb_config            = DRb::DRbServer.make_config
  @incoming_packets      = Queue.new
  @incomplete_streams    = {}
  @incomplete_timestamps = {}
  @loader                = DRbDump::Loader.new @drb_config
  @quiet                 = options[:quiet]
  @resolver              = Resolv if options[:resolve_names]
  @run_as_directory      = options[:run_as_directory]
  @run_as_user           = options[:run_as_user]

  initialize_devices options[:devices]

  @capps       = []
  @drb_streams = {}
  @running     = false
  @statistics  = DRbDump::Statistics.new
end

Instance Attribute Details

#count ⇒ Object

Number of messages to process before stopping



202
203
204
 
# File 'lib/drbdump.rb', line 202

def count
  @count
end

#drb_streams ⇒ Object (readonly)

Tracks if TCP packets contain DRb content or not



207
208
209
 
# File 'lib/drbdump.rb', line 207

def drb_streams
  @drb_streams
end

#incoming_packets ⇒ Object (readonly)

Queue of all incoming packets from Capp.



212
213
214
 
# File 'lib/drbdump.rb', line 212

def incoming_packets
  @incoming_packets
end

#incomplete_streams ⇒ Object (readonly)

Storage for incomplete DRb messages



217
218
219
 
# File 'lib/drbdump.rb', line 217

def incomplete_streams
  @incomplete_streams
end

#incomplete_timestamps ⇒ Object (readonly)

The timestamp for the first packet added to an incomplete stream



222
223
224
 
# File 'lib/drbdump.rb', line 222

def incomplete_timestamps
  @incomplete_timestamps
end

#loader ⇒ Object (readonly)

The DRb protocol loader



227
228
229
 
# File 'lib/drbdump.rb', line 227

def loader
  @loader
end

#quiet ⇒ Object

If true no per-packet information will be shown



237
238
239
 
# File 'lib/drbdump.rb', line 237

def quiet
  @quiet
end

#resolver ⇒ Object

A Resolv-compatible DNS resolver for looking up host names



232
233
234
 
# File 'lib/drbdump.rb', line 232

def resolver
  @resolver
end

#run_as_directory ⇒ Object

Directory to chroot to after starting packet capture devices (which require root privileges)

Note that you will need to either set up a custom resolver that excludes Resolv::Hosts or provide /etc/hosts in the chroot directory when setting the run_as_directory.



247
248
249
 
# File 'lib/drbdump.rb', line 247

def run_as_directory
  @run_as_directory
end

#run_as_user ⇒ Object

User to run as after starting packet capture devices (which require root privileges)



253
254
255
 
# File 'lib/drbdump.rb', line 253

def run_as_user
  @run_as_user
end

#statistics ⇒ Object (readonly)

Collects statistics on packets and messages. See DRbDump::Statistics.



258
259
260
 
# File 'lib/drbdump.rb', line 258

def statistics
  @statistics
end

Class Method Details

.process_args(argv) ⇒ Object

Converts command-line arguments argv into an options Hash



263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
 
# File 'lib/drbdump.rb', line 263

def self.process_args argv
  options = {
    count:            Float::INFINITY,
    devices:          [],
    quiet:            false,
    resolve_names:    true,
    run_as_directory: nil,
    run_as_user:      nil,
  }

  op = OptionParser.new do |opt|
    opt.program_name = File.basename $0
    opt.version = VERSION
    opt.release = nil
    opt.banner = <<-BANNER
Usage: #{opt.program_name} [options]

drbdump dumps DRb traffic from your local network.

drbdump understands TCP traffic and Rinda broadcast queries.

For information on drbdump output and usage see `ri DRbDump`.
    BANNER

    opt.separator nil

    opt.on('-c', '--count MESSAGES', Integer,
           'Capture the given number of message sends',
           'and exit, printing statistics.',
           "\n",
           'Use with -q to analyze a sample of traffic') do |count|
      options[:count] = count
    end

    opt.separator nil

    opt.on('-i', '--interface INTERFACE',
           'The interface to listen on or a tcpdump',
           'packet capture file.  Multiple interfaces',
           'can be specified.',
           "\n",
           'The tcpdump default interface and the',
           'loopback interface are the drbdump',
           'defaults') do |interface|
      options[:devices] << interface
    end

    opt.separator nil

    opt.on('-n', 'Disable name resolution') do |do_not_resolve_names|
      options[:resolve_names] = !do_not_resolve_names
    end

    opt.separator nil

    opt.on('-q', '--quiet',
           'Do not print per-message information.') do |quiet|
      options[:quiet] = quiet
    end

    opt.separator nil

    opt.on(      '--run-as-directory DIRECTORY',
           'chroot to the given directory after',
           'starting packet capture',
           "\n",
           'Note that you must disable name resolution',
           'or provide /etc/hosts in the chroot',
           'directory') do |directory|
      options[:run_as_directory] = directory
    end

    opt.separator nil

    opt.on('-Z', '--run-as-user USER',
           'Drop root privileges and run as the',
           'given user') do |user|
      options[:run_as_user] = user
    end
  end

  op.parse! argv

  options
rescue OptionParser::ParseError => e
  $stderr.puts op
  $stderr.puts
  $stderr.puts e.message

  abort
end

.run(argv = ARGV) ⇒ Object

Starts dumping DRb traffic.



358
359
360
361
362
 
# File 'lib/drbdump.rb', line 358

def self.run argv = ARGV
  options = process_args argv

  new(options).run
end

Instance Method Details

#capture_loop(capp) ⇒ Object

Loop that processes captured packets.



428
429
430
431
432
 
# File 'lib/drbdump.rb', line 428

def capture_loop capp # :nodoc:
  capp.loop do |packet|
    enqueue_packet packet
  end
end

#close_stream(source) ⇒ Object

Removes tracking data for the stream from source.



437
438
439
440
441
 
# File 'lib/drbdump.rb', line 437

def close_stream source # :nodoc:
  @drb_streams.delete source
  @incomplete_streams.delete source
  @incomplete_timestamps.delete source
end

#create_capp(device) ⇒ Object

Creates a new Capp instance that listens on device for DRb and Rinda packets.



447
448
449
450
451
452
453
454
455
456
457
 
# File 'lib/drbdump.rb', line 447

def create_capp device # :nodoc:
  capp = Capp.open device

  capp.filter = <<-FILTER
    (tcp and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)) or
    (tcp[tcpflags] & (tcp-fin|tcp-rst) != 0) or
    (udp port #{Rinda::Ring_PORT})
  FILTER

  capp
end

#display_drb(packet) ⇒ Object

Displays information from the possible DRb packet packet



484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
 
# File 'lib/drbdump.rb', line 484

def display_drb packet
  return unless @running
  return unless stream = packet_stream(packet)

  source = packet.source

  message = DRbDump::Message.from_stream self, packet, stream

  message.display

  stop if @statistics.drb_messages_sent >= @count

  @statistics.drb_packet_count += 1
  @drb_streams[source] = true
  @incomplete_timestamps.delete source
rescue DRbDump::Loader::TooLarge
  display_drb_too_large packet
rescue DRbDump::Loader::Premature, DRbDump::Loader::DataError
  @incomplete_streams[source] = stream.string
  @incomplete_timestamps[source] ||= packet.timestamp
rescue DRbDump::Loader::Error
  @drb_streams[source] = false
end

#display_drb_too_large(packet) ⇒ Object

Writes the start of a DRb stream from a packet that was too large to transmit.



512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
 
# File 'lib/drbdump.rb', line 512

def display_drb_too_large packet # :nodoc:
  return if @quiet

  rest = packet.payload

  source, destination = resolve_addresses packet

  valid, size, rest = valid_in_payload rest

  puts '%s %s to %s packet too large, valid: [%s] too big (%d bytes): %s' % [
    packet.timestamp.strftime(TIMESTAMP_FORMAT),
    source, destination,
    valid.join(', '), size, rest.dump
  ]
end

#display_packets ⇒ Object

Starts a thread that displays each captured packet.



531
532
533
534
535
536
537
538
539
540
541
542
543
 
# File 'lib/drbdump.rb', line 531

def display_packets
  @running = true

  @display_thread = Thread.new do
    while @running and packet = @incoming_packets.deq do
      if packet.udp? then
        display_ring_finger packet
      else
        display_drb packet
      end
    end
  end
end

#display_ring_finger(packet) ⇒ Object

Displays information from Rinda::RingFinger packet packet.

Currently only understands RingFinger broadcast packets.



464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
 
# File 'lib/drbdump.rb', line 464

def display_ring_finger packet
  @statistics.rinda_packet_count += 1

  return if @quiet

  obj = Marshal.load packet.payload

  (_, tell), timeout = obj

  puts '%s find ring on %s for %s timeout: %d' % [
    packet.timestamp.strftime(TIMESTAMP_FORMAT),
    packet.destination(@resolver), tell.__drburi,
    timeout
  ]
rescue
end

#enqueue_packet(packet) ⇒ Object

Enqueues packet unless it is a FIN or RST or the stream is not a DRb stream.



549
550
551
552
553
554
555
556
557
558
559
560
561
 
# File 'lib/drbdump.rb', line 549

def enqueue_packet packet # :nodoc:
  @statistics.total_packet_count += 1

  if packet.tcp? and 0 != packet.tcp_header.flags & FIN_OR_RST then
    close_stream packet.source

    return
  end

  return if @drb_streams[packet.source] == false

  @incoming_packets.enq packet
end

#initialize_devices(devices) ⇒ Object

:nodoc:



401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
 
# File 'lib/drbdump.rb', line 401

def initialize_devices devices # :nodoc:
  @devices = devices

  if @devices.empty? then
    devices = Capp.devices

    abort "you must run #{$0} with root permissions, try sudo" if
      devices.empty?

    loopback = devices.find do |device|
      device.addresses.any? do |address|
        %w[127.0.0.1 ::1].include? address.address
      end
    end

    @devices = [
      Capp.default_device_name,
      (loopback.name rescue nil),
    ].compact
  end

  @devices.uniq!
end

#load_marshal_data(object) ⇒ Object

Loads Marshal data in object if possible, or returns a DRb::DRbUnknown if there was some error.



567
568
569
570
571
 
# File 'lib/drbdump.rb', line 567

def load_marshal_data object # :nodoc:
  object.load
rescue NameError, ArgumentError => e
  DRb::DRbUnknown.new e, object.stream
end

#packet_stream(packet) ⇒ Object

Returns a StringIO created from packets that are part of the TCP connection in stream.

Returns nil if the stream is not a DRb message stream or the packet is empty.



580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
 
# File 'lib/drbdump.rb', line 580

def packet_stream packet # :nodoc:
  payload = packet.payload

  return if payload.empty?

  source = packet.source

  if previous = @incomplete_streams.delete(source) then
    payload = previous << payload
  elsif /\A....\x04\x08/m !~ payload then
    @drb_streams[source] = false
    return
  end

  stream = StringIO.new payload
  stream.set_encoding Encoding::BINARY, Encoding::BINARY
  stream
end

#resolve_addresses(packet) ⇒ Object

Resolves source and destination addresses in packet for use in DRb URIs.



602
603
604
605
606
607
608
609
610
 
# File 'lib/drbdump.rb', line 602

def resolve_addresses packet # :nodoc:
  source = packet.source @resolver
  source = "\"druby://#{source.sub(/\.(\d+)$/, ':\1')}\""

  destination = packet.destination @resolver
  destination = "\"druby://#{destination.sub(/\.(\d+)$/, ':\1')}\""

  return source, destination
end

#run ⇒ Object

Captures packets and displays them on the screen.



615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
 
# File 'lib/drbdump.rb', line 615

def run
  capps = @devices.map { |device| create_capp device }

  Capp.drop_privileges @run_as_user, @run_as_directory

  start_capture capps

  trap_info

  display_packets.join
rescue Interrupt
  untrap_info

  stop

  @display_thread.join

  puts # clear ^C

  exit
ensure
  @statistics.show
end

#start_capture(capps) ⇒ Object

Captures DRb packets and feeds them to the incoming_packets queue



642
643
644
645
646
647
648
649
650
 
# File 'lib/drbdump.rb', line 642

def start_capture capps
  @capps.concat capps

  capps.map do |capp|
    Thread.new do
      capture_loop capp
    end
  end
end

#stop ⇒ Object

Stops the message capture and packet display. If root privileges were dropped message capture cannot be restarted.



656
657
658
659
660
661
662
663
664
 
# File 'lib/drbdump.rb', line 656

def stop
  @running = false

  @capps.each do |capp|
    capp.stop
  end

  @incoming_packets.enq nil
end

#trap_info ⇒ Object

Adds a SIGINFO handler if the OS supports it



669
670
671
672
673
674
675
 
# File 'lib/drbdump.rb', line 669

def trap_info
  return unless Signal.list['INFO']

  trap 'INFO' do
    @statistics.show_basic
  end
end

#untrap_info ⇒ Object

Sets the SIGINFO handler to the DEFAULT handler



680
681
682
683
684
 
# File 'lib/drbdump.rb', line 680

def untrap_info
  return unless Signal.list['INFO']

  trap 'INFO', 'DEFAULT'
end

#valid_in_payload(too_large) ⇒ Object

Returns the valid parts, the size and content of the invalid part in large_packet



690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
 
# File 'lib/drbdump.rb', line 690

def valid_in_payload too_large # :nodoc:
  load_limit = @drb_config[:load_limit]

  size  = nil
  valid = []

  loop do
    size, too_large = too_large.unpack 'Na*'

    break if load_limit < size

    valid << Marshal.load(too_large.slice!(0, size)).inspect
  end

  return valid, size, too_large
end