Class: Arachni::RPC::Server::Instance

Inherits:
Object
  • Object
show all
Includes:
UI::Output, Utilities
Defined in:
lib/arachni/rpc/server/instance.rb

Overview

Note:

Ignore:

  • Inherited methods and attributes – only public methods of this class are

    accessible over RPC.

  • ‘block` parameters, they are an RPC implementation detail for methods which

    perform asynchronous operations.
Note:

Methods which expect ‘Symbol` type parameters will also accept `String` types as well.

For example, the following:

instance.service.scan url: 'http://testfire.net'

Is the same as:

instance.service.scan 'url' => 'http://testfire.net'

Represents an Arachni instance (or multiple instances when running a multi-Instance scan) and serves as a central point of access and control.

# Methods

Provides methods for:

(A nice simple example can be found in the RPC command-line client interface.)

Examples:

A minimalistic example – assumes Arachni is installed and available.

require 'arachni'
require 'arachni/rpc/client'

instance = Arachni::RPC::Client::Instance.new( Arachni::Options.instance,
                                               'localhost:1111', 's3cr3t' )

instance.service.scan url: 'http://testfire.net',
                      audit:  {
                          elements: [:links, :forms]
                      },
                      # load all XSS checks
                      checks: 'xss*'

print 'Running.'
while instance.service.busy?
    print '.'
    sleep 1
end

# Grab the report
report = instance.service.report

# Kill the instance and its process, no zombies please...
instance.service.shutdown

puts
puts
puts 'Logged issues:'
report['issues'].each do |issue|
   puts "  * #{issue['name']} in '#{issue['vector']['type']}' input '#{issue['vector']['affected_input_name']}' at '#{issue['vector']['action']}'."
end

Author:

Instance Method Summary collapse

Methods included from Utilities

#available_port, #caller_name, #caller_path, #cookie_decode, #cookie_encode, #cookies_from_document, #cookies_from_file, #cookies_from_response, #exception_jail, #exclude_path?, #follow_protocol?, #form_decode, #form_encode, #forms_from_document, #forms_from_response, #generate_token, #get_path, #hms_to_seconds, #html_decode, #html_encode, #include_path?, #links_from_document, #links_from_response, #normalize_url, #page_from_response, #page_from_url, #parse_set_cookie, #path_in_domain?, #path_too_deep?, #port_available?, #rand_port, #random_seed, #redundant_path?, #regexp_array_match, #remove_constants, #request_parse_body, #seconds_to_hms, #skip_page?, #skip_path?, #skip_resource?, #skip_response?, #to_absolute, #uri_decode, #uri_encode, #uri_parse, #uri_parse_query, #uri_parser, #uri_rewrite

Methods included from UI::Output

#debug?, #debug_off, #debug_on, #disable_only_positives, #included, #mute, #muted?, #only_positives, #only_positives?, #print_bad, #print_debug, #print_debug_backtrace, #print_debug_level_1, #print_debug_level_2, #print_debug_level_3, #print_error, #print_error_backtrace, #print_exception, #print_info, #print_line, #print_ok, #print_status, #print_verbose, #reroute_to_file, #reroute_to_file?, reset_output_options, #unmute, #verbose?, #verbose_on

Constructor Details

#initialize(options, token) ⇒ Instance

Initializes the RPC interface and the framework.

Parameters:



118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/arachni/rpc/server/instance.rb', line 118

def initialize( options, token )
    @options = options
    @token   = token

    @options.snapshot.save_path ||= @options.paths.snapshots

    @framework      = Server::Framework.new( Options.instance )
    @active_options = Server::ActiveOptions.new( @framework )

    @server = Base.new( @options, token )
    @server.logger.level = @options.datastore.log_level if @options.datastore.log_level

    @options.datastore.token = token

    if @options.output.reroute_to_logfile
        reroute_to_file "#{@options.paths.logs}/Instance - #{Process.pid}" <<
                            "-#{@options.rpc.server_port}.log"
    else
        reroute_to_file false
    end

    set_error_logfile "#{@options.paths.logs}/Instance - #{Process.pid}" <<
                          "-#{@options.rpc.server_port}.error.log"

    set_handlers( @server )

    # trap interrupts and exit cleanly when required
    %w(QUIT INT).each do |signal|
        next if !Signal.list.has_key?( signal )
        trap( signal ){ shutdown if !@options.datastore.do_not_trap }
    end

    @consumed_pids = []

    Reactor.global.run do
        run
    end
end

Instance Method Details

#abort_and_report(&block) ⇒ Hash

Note:

Don’t forget to #shutdown the instance once you get the report.

Cleans up and returns the report.

Returns:

See Also:



294
295
296
 
# File 'lib/arachni/rpc/server/instance.rb', line 294

def abort_and_report( &block )
    @framework.clean_up { block.call report.to_h }
end

#abort_and_report_as(name, &block) ⇒ Object

Note:

Don’t forget to #shutdown the instance once you get the report.

Cleans up and delegates to #report_as.

Parameters:

  • name (String) —

    Name of the report component to run, as presented by #list_reporters‘s `:shortname` key.

See Also:



315
316
317
 
# File 'lib/arachni/rpc/server/instance.rb', line 315

def abort_and_report_as( name, &block )
    @framework.clean_up { block.call report_as( name ) }
end

#alive? ⇒ true

Returns:

  • (true) 


200
201
202
 
# File 'lib/arachni/rpc/server/instance.rb', line 200

def alive?
    @server.alive?
end

#busy?(&block) ⇒ Bool

Returns ‘true` if the scan is initializing or running, `false` otherwise.

Returns:

  • (Bool) —

    ‘true` if the scan is initializing or running, `false` otherwise.



206
207
208
209
210
211
212
213
 
# File 'lib/arachni/rpc/server/instance.rb', line 206

def busy?( &block )
    if @scan_initializing
        block.call( true ) if block_given?
        return true
    end

    @framework.busy?( &block )
end

#clear_cookies ⇒ Object

For testing.



695
696
697
698
699
 
# File 'lib/arachni/rpc/server/instance.rb', line 695

def clear_cookies
    Arachni::Options.reset
    Arachni::HTTP::Client.cookie_jar.clear
    true
end

#consumed_pids(&block) ⇒ Object 



665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
 
# File 'lib/arachni/rpc/server/instance.rb', line 665

def consumed_pids( &block )
    pids  = ([Process.pid] | @consumed_pids)
    pids |= browser_cluster.consumed_pids if browser_cluster

    if @consumed_pids.empty?
        return block.call pids
    end

    foreach = proc do |instance, iter|
        instance.service.consumed_pids do |slave_pids|
            iter.return( !slave_pids.rpc_exception? ? slave_pids : [] )
        end
    end
    after = proc do |results|
        block.call pids | results.flatten
    end

    @framework.map_slaves( foreach, after )

    true
end

#cookies ⇒ Object

For testing.



689
690
691
 
# File 'lib/arachni/rpc/server/instance.rb', line 689

def cookies
    Arachni::HTTP::Client.cookies.map(&:to_rpc_data)
end

#error_test(str, &block) ⇒ Object 



660
661
662
 
# File 'lib/arachni/rpc/server/instance.rb', line 660

def error_test( str, &block )
    @framework.error_test( str, &block )
end

#errors(starting_line = 0, &block) ⇒ Array<String>

Parameters:

  • starting_line (Integer) (defaults to: 0) —

    Sets the starting line for the range of errors to return.

Returns:



217
218
219
 
# File 'lib/arachni/rpc/server/instance.rb', line 217

def errors( starting_line = 0, &block )
    @framework.errors( starting_line, &block )
end

#list_checks ⇒ Object 



233
234
235
 
# File 'lib/arachni/rpc/server/instance.rb', line 233

def list_checks
    @framework.list_checks
end

#list_platforms ⇒ Object 



228
229
230
 
# File 'lib/arachni/rpc/server/instance.rb', line 228

def list_platforms
    @framework.list_platforms
end

#list_plugins ⇒ Object 



238
239
240
 
# File 'lib/arachni/rpc/server/instance.rb', line 238

def list_plugins
    @framework.list_plugins
end

#list_reporters ⇒ Object 



243
244
245
 
# File 'lib/arachni/rpc/server/instance.rb', line 243

def list_reporters
    @framework.list_reporters
end

#native_abort_and_report(&block) ⇒ Object

Like #abort_and_report but returns a Arachni::RPC::Serializer#dump representation of Arachni::Report.



302
303
304
 
# File 'lib/arachni/rpc/server/instance.rb', line 302

def native_abort_and_report( &block )
    @framework.clean_up { native_report( &block ) }
end

#native_progress(options = {}, &block) ⇒ Object

Like #progress but returns MessagePack representation of native objects instead of simplified hashes.



437
438
439
 
# File 'lib/arachni/rpc/server/instance.rb', line 437

def native_progress( options = {}, &block )
    progress_handler( options.merge( as_hash: false ), &block )
end

#native_report(&block) ⇒ Object 



321
322
323
 
# File 'lib/arachni/rpc/server/instance.rb', line 321

def native_report( &block )
    @framework.report( &block )
end

#pause(&block) ⇒ Object

Pauses the running scan on a best effort basis.



253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
 
# File 'lib/arachni/rpc/server/instance.rb', line 253

def pause( &block )
    if @rpc_pause_request
        block.call( true )
        return
    end

    # Send the pause request but don't block.
    r = @framework.pause( false )
    @rpc_pause_request ||= r

    if !@framework.has_slaves?
        block.call( true )
        return
    end

    each = proc { |instance, iter| instance.service.pause { iter.next } }
    each_slave( each, proc { block.call true } )
end

#paused? ⇒ Boolean

Returns:

  • (Boolean) 


248
249
250
 
# File 'lib/arachni/rpc/server/instance.rb', line 248

def paused?
    @framework.paused?
end

#progress(options = {}, &block) ⇒ Hash

# Recommended usage

Please request from the method only the things you are going to actually
use, otherwise you'll just be wasting bandwidth.
In addition, ask to **not** be served data you already have, like issues
or error messages.

To be kept completely up to date on the progress of a scan (i.e. receive
new issues and error messages asap) in an efficient manner, you will need
to keep track of the issues and error messages you already have and
explicitly tell the method to not send the same data back to you on
subsequent calls.

## Retrieving errors (‘:errors` option) without duplicate data

This is done by telling the method how many error messages you already
have and you will be served the errors from the error-log that are past
that line.
So, if you were to use a loop to get fresh progress data it would look
like so:

  error_cnt = 0
  i = 0
  while sleep 1
      # Test method, triggers an error log...
      instance.service.error_test "BOOM! #{i+=1}"

      # Only request errors we don't already have
      errors = instance.service.progress( with: { errors: error_cnt } )[:errors]
      error_cnt += errors.size

      # You will only see new errors
      puts errors.join("\n")
  end

## Retrieving issues without duplicate data

In order to be served only new issues you will need to let the method
know which issues you already have. This is done by providing a list
of {Issue#digest digests} for the issues you already know about.

  issue_digests = []
  while sleep 1
      issues = instance.service.progress(
                   with: :issues,
                   # Only request issues we don't already have
                   without: { issues: issue_digests  }
               )[:issues]

      issue_digests |= issues.map { |issue| issue['digest'] }

      # You will only see new issues
      issues.each do |issue|
          puts "  * #{issue['name']} in '#{issue['vector']['type']}' input '#{issue['vector']['affected_input_name']}' at '#{issue['vector']['action']}'."
      end
  end

Parameters:

  • options (Hash) (defaults to: {}) —

    Options about what progress data to retrieve and return.

Options Hash (options):

  • :with (Array<Symbol, Hash>) —

    Specify data to include:

    • :issues – Discovered issues as hashes.

    • :instances – Statistics and info for slave instances.

    • :errors – Errors and the line offset to use for #errors. Pass as a hash, like: ‘{ errors: 10 }`

  • :without (Array<Symbol, Hash>) —

    Specify data to exclude:

    • :statistics – Don’t include runtime statistics.

    • :issues – Don’t include issues with the given digests. Pass as a hash, like: ‘{ issues: […] }`

Returns:

  • (Hash) —

    • ‘statistics` – General runtime statistics (merged when part of Grid)

      (enabled by default)

    • ‘status` – #status

    • ‘busy` – #busy?

    • ‘issues` – Discovered issues as hashes.

      (disabled by default)

    • ‘instances` – Raw `statistics` for each running instance (only when part

      of Grid) (disabled by default)

    • ‘errors` – #errors (disabled by default)

    • ‘sitemap` – #sitemap (disabled by default)



429
430
431
 
# File 'lib/arachni/rpc/server/instance.rb', line 429

def progress( options = {}, &block )
    progress_handler( options.merge( as_hash: true ), &block )
end

#report ⇒ Hash

Returns Arachni::Report#to_h.

Returns:



327
328
329
 
# File 'lib/arachni/rpc/server/instance.rb', line 327

def report
    @framework.report.to_h
end

#report_as(name) ⇒ Object

Parameters:

  • name (String) —

    Name of the report component to run, as presented by #list_reporters‘s `:shortname` key.



336
337
338
 
# File 'lib/arachni/rpc/server/instance.rb', line 336

def report_as( name )
    @framework.report_as( name )
end

#restore(snapshot) ⇒ Object

See Also:



187
188
189
190
191
 
# File 'lib/arachni/rpc/server/instance.rb', line 187

def restore( snapshot )
    @framework.restore snapshot
    @framework.run
    true
end

#resume(&block) ⇒ Object

Resumes a paused scan right away.



273
274
275
276
277
278
279
280
281
282
283
284
285
 
# File 'lib/arachni/rpc/server/instance.rb', line 273

def resume( &block )
    return block.call( false ) if !@rpc_pause_request

    @framework.resume( @rpc_pause_request )

    if !@framework.has_slaves?
        block.call true
        return
    end

    each = proc { |instance, iter| instance.service.resume { iter.next } }
    each_slave( each, proc { block.call true } )
end

#scan(opts = {}, &block) ⇒ Object

Note:

Options marked with an asterisk are required.

Note:

Options which expect patterns will interpret their arguments as regular expressions regardless of their type.

Configures and runs a scan.

Parameters:

  • opts (Hash) (defaults to: {}) —

    Scan options to be passed to Options#update (along with some extra ones to keep configuration in one place).

    _The options presented here are the most commonly used ones, in actuality, you can use anything supported by Options#update._

Options Hash (opts):

  • *:url (String) —

    Target URL to audit.

  • :authorized_by (String) — default: nil —

    The e-mail address of the person who authorized the scan.

    john.doe@bigscanners.com
  • :audit (Hash) —

    Audit options.

  • :scope (Hash) —

    Scope options.

  • :http (Hash) —

    HTTP options.

  • :login (Hash) —

    Session options.

  • :checks (String, Array<String>) — default: [] —

    Checks to load, by name.

    # To load all checks use the wildcard on its own
'*'

# To load all XSS and SQLi checks:
[ 'xss*', 'sql_injection*' ]
  • :plugins (Hash<Hash>) — default: {} —

    Plugins to load, by name, along with their options.

    {
    'proxy'      => {}, # empty options
    'autologin'  => {
        'url'         => 'http://demo.testfire.net/bank/login.aspx',
        'parameters' => 'uid=jsmith&passw=Demo1234',
        'check'       => 'MY ACCOUNT'
    },
}
  • :platforms (String, Symbol, Array<String, Symbol>) — default: [] —

    Initialize the fingerprinter with the given platforms.

    The fingerprinter cannot identify database servers so specifying the remote DB backend will greatly enhance performance and reduce bandwidth consumption.

  • :no_fingerprinting (Bool) — default: false —

    Disable platform fingerprinting and include all payloads in the audit.

    Use this option in addition to the ‘:platforms` one to restrict the audit payloads to explicitly specified platforms.

  • :grid (Bool) — default: false —

    Use the Dispatcher Grid to load-balance scans across the available nodes.

    If set to ‘true`, it serves as a shorthand for:

    grid_mode: :balance
  • :grid_mode (String, Symbol) — default: nil —

    Grid mode to use, available modes are:

    • ‘nil` – No grid.

    • ‘:balance` – Slave Instances will be provided by the least burdened

      grid members to keep the overall Grid workload even across all Dispatchers.

    • ‘:aggregate` – Used to perform a multi-Instance scan and will only

      request Instances from Grid members with different Pipe-IDs, resulting
in application-level bandwidth aggregation.
  • :spawns (Integer) — default: 0 —

    The amount of slaves to spawn. The behavior of this option changes depending on the ‘grid_mode` setting:

    • ‘nil` – All slave Instances will be spawned by this Instance directly,

      and thus reside in the same machine.

    • ‘:balance` – Slaves will be provided by the least burdened Grid Dispatchers.

    • ‘:aggregate` – Slaves will be provided by Grid Dispatchers with unique

      Pipe-IDs and the value of this option will be treated as a possible
maximum rather than a hard setting. Actual spawn count will be determined
by Dispatcher availability at the time.



527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
 
# File 'lib/arachni/rpc/server/instance.rb', line 527

def scan( opts = {}, &block )
    # If the instance isn't clean bail out now.
    if busy? || @called
        block.call false
        return false
    end

    # Normalize this sucker to have symbols as keys.
    opts = opts.my_symbolize_keys( false )

    slaves      = opts.delete(:slaves) || []
    spawn_count = opts[:spawns]
    spawn_count = spawn_count.to_i

    if (platforms = opts.delete(:platforms))
        begin
            Platform::Manager.new( [platforms].flatten.compact )
        rescue => e
            fail ArgumentError, e.to_s
        end
    end

    opts[:dispatcher] ||= {}
    opts[:scope]      ||= {}

    if opts[:grid] || opts[:grid_mode]
        if spawn_count <= 0
            fail ArgumentError,
                 'Option \'spawns\' must be greater than 1 for Grid scans.'
        end

        if [opts[:scope]['restrict_paths']].flatten.compact.any?
            fail ArgumentError,
                 'Scope option \'restrict_paths\' is not supported when in' <<
                     ' multi-Instance mode.'
        end
    end

    # There may be follow-up/retry calls by the client in cases of network
    # errors (after the request has reached us) so we need to keep minimal
    # track of state in order to bail out on subsequent calls.
    @called = @scan_initializing = true

    # Plugins option needs to be a hash...
    if opts[:plugins] && opts[:plugins].is_a?( Array )
        opts[:plugins] = opts[:plugins].inject( {} ) { |h, n| h[n] = {}; h }
    end

    if opts.include?( :grid )
        @framework.options.dispatcher.grid = opts.delete(:grid)
    end

    if opts.include?( :grid_mode )
        @framework.options.dispatcher.grid_mode = opts.delete(:grid_mode)
    end

    @active_options.set( opts )

    if @framework.options.url.to_s.empty?
        fail ArgumentError, 'Option \'url\' is mandatory.'
    end

    @framework.checks.load opts[:checks] if opts[:checks]
    @framework.plugins.load opts[:plugins] if opts[:plugins]

    # Starts the scan after all necessary options have been set.
    after = proc { block.call @framework.run; @scan_initializing = false }

    if @framework.options.dispatcher.grid?
        # If a Grid scan has been selected then just set us as the master,
        # the Framework will sort out the rest.
        @framework.set_as_master

        # Rock n' roll!
        after.call
    else
        # Handles each spawn, enslaving it for a multi-Instance scan.
        each = proc do |slave, iter|
            @framework.enslave( slave ){ iter.next }
        end

        spawn( spawn_count ) do |spawns|
            # Add our spawns to the slaves list which was passed as an option.
            slaves |= spawns

            # Process the Instances.
            Reactor.global.create_iterator( slaves, slaves.empty? ? 1 : slaves.size ).
                each( each, after )
        end
    end

    true
end

#shutdown(&block) ⇒ Object

Makes the server go bye-bye…Lights out!



622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
 
# File 'lib/arachni/rpc/server/instance.rb', line 622

def shutdown( &block )
    if @shutdown
        block.call if block_given?
        return
    end
    @shutdown = true

    print_status 'Shutting down...'

    # We're shutting down services so we need to use a concurrent way but
    # without going through the Reactor.
    Thread.new do
        t = []

        if browser_cluster
            # We can't block until the browser cluster shuts down cleanly
            # (i.e. wait for any running jobs) but we don't need to anyways.
            t << Thread.new { browser_cluster.shutdown false }
        end

        @framework.instance_eval do
            next if !has_slaves?

            @slaves.each do |instance|
                t << Thread.new { connect_to_instance( instance ).service.shutdown }
            end
        end

        t.each(&:join)
        @server.shutdown

        block.call true if block_given?
    end

    true
end

#sitemap(index = 0) ⇒ Object 



223
224
225
 
# File 'lib/arachni/rpc/server/instance.rb', line 223

def sitemap( index = 0 )
    @framework.sitemap_entries( index )
end

#snapshot_path ⇒ String?

Returns Path to the snapshot of the suspended scan, ‘nil` if not #suspended?.

Returns:

See Also:



163
164
165
166
 
# File 'lib/arachni/rpc/server/instance.rb', line 163

def snapshot_path
    return if !suspended?
    @framework.snapshot_path
end

#status ⇒ Object 



341
342
343
 
# File 'lib/arachni/rpc/server/instance.rb', line 341

def status
    @framework.status
end

#suspend ⇒ Object

Note:

The path to the snapshot can be retrieved via #snapshot_path.

Writes a Snapshot to disk and aborts the scan.

See Also:



173
174
175
176
177
178
179
180
 
# File 'lib/arachni/rpc/server/instance.rb', line 173

def suspend
    if !@framework.solo?
        fail State::Framework::Error::StateNotSuspendable,
             'Cannot suspend a multi-Instance scan.'
    end

    @framework.suspend false
end

#suspended? ⇒ Boolean

Returns:

  • (Boolean)

See Also:



195
196
197
 
# File 'lib/arachni/rpc/server/instance.rb', line 195

def suspended?
    @framework.suspended?
end