Class: RFuzz::HttpClient

Inherits:
Object
  • Object
show all
Includes:
HttpEncoding
Defined in:
lib/rfuzz/client.rb

Overview

The actual HttpClient that does the work with the thinnest layer between you and the protocol. All exceptions and leaks are allowed to pass through since those are important when testing. It doesn’t pretend to be a full client, but instead is just enough client to track cookies, form proper HTTP requests, and return HttpResponse hashes with the results.

It’s designed so that you create one client, and then you work it with a minimum of parameters as you need. The initialize method lets you pass in defaults for most of the parameters you’ll need, and you can simple call the method you want and it’ll be translated to an HTTP method (client.get => GET, client.foobar = FOOBAR).

Here’s a few examples:

client = HttpClient.new(:head => {"X-DefaultHeader" => "ONE"})
resp = client.post("/test")
resp = client.post("/test", :head => {"X-TestSend" => "Status"}, :body => "TEST BODY")
resp = client.put("/testput", :query => {"q" => "test"}, :body => "SOME JUNK")
client.reset

The HttpClient.reset call clears cookies that are maintained.

It uses method_missing to do the translation of .put to “PUT /testput HTTP/1.1” so you can get into trouble if you’re calling unknown methods on it. By default the methods are PUT, GET, POST, DELETE, HEAD. You can change the allowed methods by passing :allowed_methods => [:put, :get, ..] to the initialize for the object.

Notifications

You can register a “notifier” with the client that will get called when different events happen. Right now the Notifier class just has a few functions for the common parts of an HTTP request that each take a symbol and some extra parameters. See RFuzz::Notifier for more information.

Parameters

:head => {K => V}  or {K => [V1,V2]}
:query => {K => V} or {K => [V1,V2]}
:body => "some body" (you must encode for now)
:cookies => {K => V} or {K => [V1, V2]}
:allowed_methods => [:put, :get, :post, :delete, :head]
:notifier => Notifier.new
:redirect => false (give it a number and it'll follow redirects for that count)

Constant Summary collapse

TRANSFER_ENCODING = 
"TRANSFER_ENCODING"
CONTENT_LENGTH = 
"CONTENT_LENGTH"

"SET_COOKIE"
LOCATION = 
"LOCATION"
HOST = 
"HOST"
HTTP_REQUEST_HEADER = 
"%s %s HTTP/1.1\r\n"
REQ_CONTENT_LENGTH = 
"Content-Length"
REQ_HOST = 
"Host"
CHUNK_SIZE = 
1024 * 16
CRLF = 
"\r\n"

Constants included from HttpEncoding

RFuzz::HttpEncoding::COOKIE, RFuzz::HttpEncoding::FIELD_ENCODING

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from HttpEncoding

#encode_cookies, #encode_field, #encode_headers, #encode_host, #encode_param, #encode_query, #escape, #query_parse, #unescape

Constructor Details

#initialize(host, port, options = {}) ⇒ HttpClient

Doesn’t make the connect until you actually call a .put,.get, etc.



230
231
232
233
234
235
236
237
238
239
 
# File 'lib/rfuzz/client.rb', line 230

def initialize(host, port, options = {})
  @options = options
  @host = host
  @port = port
  @cookies = options[:cookies] || {}
  @allowed_methods = options[:allowed_methods] || [:put, :get, :post, :delete, :head]
  @notifier = options[:notifier]
  @redirect = options[:redirect] || false
  @parser = HttpClientParser.new
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(symbol, *args) ⇒ Object

Translates unknown function calls into PUT, GET, POST, DELETE, HEAD methods. The allowed HTTP methods allowed are restricted by the during construction with :allowed_methods => [:put, :get, …]



405
406
407
408
409
410
411
412
413
414
415
 
# File 'lib/rfuzz/client.rb', line 405

def method_missing(symbol, *args)
  if @allowed_methods.include? symbol
    method = symbol.to_s.upcase
    resp = send_request(method, args[0], args[1] || {})
    resp = redirect(symbol, resp) if @redirect

    return resp
  else
    raise HttpClientError.new("Invalid method: #{symbol}")
  end
end

Instance Attribute Details

#allowed_methods ⇒ Object

Access to the host, port, default options, and cookies currently in play



227
228
229
 
# File 'lib/rfuzz/client.rb', line 227

def allowed_methods
  @allowed_methods
end

#cookies ⇒ Object

Access to the host, port, default options, and cookies currently in play



227
228
229
 
# File 'lib/rfuzz/client.rb', line 227

def cookies
  @cookies
end

#host ⇒ Object

Access to the host, port, default options, and cookies currently in play



227
228
229
 
# File 'lib/rfuzz/client.rb', line 227

def host
  @host
end

#notifier ⇒ Object

Access to the host, port, default options, and cookies currently in play



227
228
229
 
# File 'lib/rfuzz/client.rb', line 227

def notifier
  @notifier
end

#options ⇒ Object

Access to the host, port, default options, and cookies currently in play



227
228
229
 
# File 'lib/rfuzz/client.rb', line 227

def options
  @options
end

#port ⇒ Object

Access to the host, port, default options, and cookies currently in play



227
228
229
 
# File 'lib/rfuzz/client.rb', line 227

def port
  @port
end

#sock ⇒ Object

Access to the host, port, default options, and cookies currently in play



227
228
229
 
# File 'lib/rfuzz/client.rb', line 227

def sock
  @sock
end

Instance Method Details

#build_request(out, method, uri, req) ⇒ Object

Builds a full request from the method, uri, req, and @cookies using the default @options and writes it to out (should be an IO).

It returns the body that the caller should use (based on defaults resolution).



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
 
# File 'lib/rfuzz/client.rb', line 247

def build_request(out, method, uri, req)
  ops = @options.merge(req)
  query = ops[:query]

  # merge head differently since that's typically what they mean
  head = req[:head] || {}
  head = ops[:head].merge(head) if ops[:head]

  # setup basic headers we always need
  head[REQ_HOST] = encode_host(@host,@port)
  head[REQ_CONTENT_LENGTH] = ops[:body] ? ops[:body].length : 0

  # blast it out
  out.write(HTTP_REQUEST_HEADER % [method, encode_query(uri,query)])
  out.write(encode_headers(head))
  out.write(encode_cookies(@cookies.merge(req[:cookies] || {})))
  out.write(CRLF)
  ops[:body] || ""
end

#notify(event) ⇒ Object

Sends the notifications to the registered notifier, taking a block that it runs doing the :begins, :ends states around it.

It also catches errors transparently in order to call the notifier when an attempt fails.



451
452
453
454
455
456
457
458
459
460
461
 
# File 'lib/rfuzz/client.rb', line 451

def notify(event)
  @notifier.send(event, :begins) if @notifier

  begin
    yield
    @notifier.send(event, :ends) if @notifier
  rescue Object
    @notifier.send(event, :error) if @notifier
    raise $!
  end
end

#read_chunked_body(header) ⇒ Object

Collects up a chunked body both collecting the body together and collecting the chunks into HttpResponse.raw_chunks[] for alternative analysis.



306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
 
# File 'lib/rfuzz/client.rb', line 306

def read_chunked_body(header)
  @sock.push(header.http_body)
  header.http_body = ""
  header.raw_chunks = []

  while true
    @notifier.read_chunk(:begins) if @notifier
    chunk = read_chunked_header
    header.raw_chunks << chunk
    if !chunk.last_chunk?
      header.http_body << chunk.http_body
      @notifier.read_chunk(:end) if @notifier
    else
      @notifier.read_chunk(:end) if @notifier
      break # last chunk, done
    end
  end

  header
end

#read_chunked_header ⇒ Object

Used to process chunked headers and then read up their bodies.



286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
 
# File 'lib/rfuzz/client.rb', line 286

def read_chunked_header
  resp = read_parsed_header
  @sock.push(resp.http_body)

  if !resp.last_chunk?
    resp.http_body = @sock.read(resp.chunk_size)

    trail = @sock.read(2)
    if trail != CRLF
      raise HttpClientParserError.new("Chunk ended in #{trail.inspect} not #{CRLF.inspect}")
    end
  end

  return resp
end

#read_parsed_header ⇒ Object

Does the read operations needed to parse a header with the @parser. A “header” in this case is either an HTTP header or a Chunked encoding header (since the @parser handles both).



270
271
272
273
274
275
276
277
278
279
280
281
282
 
# File 'lib/rfuzz/client.rb', line 270

def read_parsed_header
  @parser.reset
  resp = HttpResponse.new
  data = @sock.read(CHUNK_SIZE, partial=true)
  nread = @parser.execute(resp, data, 0)

  while !@parser.finished?
    data << @sock.read(CHUNK_SIZE, partial=true)
    nread = @parser.execute(resp, data, nread)
  end

  return resp
end

#read_response ⇒ Object

Reads an HTTP response from the given socket. It uses readpartial which only appeared in Ruby 1.8.4. The result is a fully formed HttpResponse object for you to play with.

As with other methods in this class it doesn’t stop any exceptions from reaching your code. It’s for experts who want these exceptions so either write a wrapper, use net/http, or deal with it on your end.



344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
 
# File 'lib/rfuzz/client.rb', line 344

def read_response
  resp = HttpResponse.new

  notify :read_header do
    resp = read_parsed_header
  end

  notify :read_body do
    if resp.chunked_encoding?
      read_chunked_body(resp)
    elsif resp[CONTENT_LENGTH]
      needs = resp[CONTENT_LENGTH].to_i - resp.http_body.length
      # Some requests can actually give a content length, and then not have content
      # so we ignore HttpClientError exceptions and pray that's good enough
      resp.http_body += @sock.read(needs) if needs > 0 rescue HttpClientError
    else
      while true
        begin
          resp.http_body += @sock.read(CHUNK_SIZE, partial=true)
        rescue HttpClientError
          break # this is fine, they closed the socket then
        end
      end
    end
  end

  store_cookies(resp)
  return resp
end

#redirect(method, resp, *args) ⇒ Object

Keeps doing requests until it doesn’t receive a 3XX request.



418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
 
# File 'lib/rfuzz/client.rb', line 418

def redirect(method, resp, *args)
  @redirect.times do
    break if resp.http_status.index("3") != 0

    host = encode_host(@host,@port)
    location = resp[LOCATION]

    if location.index(host) == 0
      # begins with the host so strip that off
      location = location[host.length .. -1]
    end

    @notifier.redirect(:begins) if @notifier
    resp = self.send(method, location, *args)
    @notifier.redirect(:ends) if @notifier
  end

  return resp
end

#reset ⇒ Object

Clears out the cookies in use so far in order to get a clean slate.



440
441
442
 
# File 'lib/rfuzz/client.rb', line 440

def reset
  @cookies.clear
end

#send_request(method, uri, req) ⇒ Object

Does the socket connect and then build_request, read_response calls finally returning the result.



376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
 
# File 'lib/rfuzz/client.rb', line 376

def send_request(method, uri, req)
  begin
    notify :connect do
      @sock = PushBackIO.new(TCPSocket.new(@host, @port))
    end

    out = StringIO.new
    body = build_request(out, method, uri, req)

    notify :send_request do
      @sock.write(out.string + body)
      @sock.flush
    end

    return read_response
  rescue Object
    raise $!
  ensure
    if @sock
      notify(:close) { @sock.close }
    end
  end
end

#store_cookies(resp) ⇒ Object

Reads the SET_COOKIE string out of resp and translates it into the @cookies store for this HttpClient.



329
330
331
332
333
334
335
 
# File 'lib/rfuzz/client.rb', line 329

def store_cookies(resp)
  if resp[SET_COOKIE]
    cookies = query_parse(resp[SET_COOKIE], ';')
    @cookies.merge! cookies
    @cookies.delete "path"
  end
end