Class: RFuzz::HttpClient

Inherits:
Object
  • Object
show all
Includes:
HttpEncoding
Defined in:
lib/rfuzz/client.rb

Overview

The actual HttpClient that does the work with the thinnest layer between you and the protocol. All exceptions and leaks are allowed to pass through since those are important when testing. It doesn’t pretend to be a full client, but instead is just enough client to track cookies, form proper HTTP requests, and return HttpResponse hashes with the results.

It’s designed so that you create one client, and then you work it with a minimum of parameters as you need. The initialize method lets you pass in defaults for most of the parameters you’ll need, and you can simple call the method you want and it’ll be translated to an HTTP method (client.get => GET, client.foobar = FOOBAR).

Here’s a few examples:

client = HttpClient.new(:head => {"X-DefaultHeader" => "ONE"})
resp = client.post("/test")
resp = client.post("/test", :head => {"X-TestSend" => "Status"}, :body => "TEST BODY")
resp = client.put("/testput", :query => {"q" => "test"}, :body => "SOME JUNK")
client.reset

The HttpClient.reset call clears cookies that are maintained.

It uses method_missing to do the translation of .put to “PUT /testput HTTP/1.1” so you can get into trouble if you’re calling unknown methods on it. By default the methods are PUT, GET, POST, DELETE, HEAD. You can change the allowed methods by passing :allowed_methods => [:put, :get, ..] to the initialize for the object.

Notifications

You can register a “notifier” with the client that will get called when different events happen. Right now the Notifier class just has a few functions for the common parts of an HTTP request that each take a symbol and some extra parameters. See RFuzz::Notifier for more information.

Parameters

:head => {K => V}  or {K => [V1,V2]}
:query => {K => V} or {K => [V1,V2]}
:body => "some body" (you must encode for now)
:cookies => {K => V} or {K => [V1, V2]}
:allowed_methods => [:put, :get, :post, :delete, :head]
:notifier => Notifier.new
:redirect => false (give it a number and it'll follow redirects for that count)

Constant Summary collapse

TRANSFER_ENCODING = 
"TRANSFER_ENCODING"
CONTENT_LENGTH = 
"CONTENT_LENGTH"
"SET_COOKIE"
LOCATION = 
"LOCATION"
HOST = 
"HOST"
HTTP_REQUEST_HEADER = 
"%s %s HTTP/1.1\r\n"
REQ_CONTENT_LENGTH = 
"Content-Length"
REQ_HOST = 
"Host"

Constants included from HttpEncoding

RFuzz::HttpEncoding::COOKIE

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from HttpEncoding

#encode_cookies, #encode_field, #encode_headers, #encode_host, #encode_param, #encode_query, #escape, #query_parse, #unescape

Constructor Details

#initialize(host, port, options = {}) ⇒ HttpClient

Doesn’t make the connect until you actually call a .put,.get, etc.



214
215
216
217
218
219
220
221
222
 
# File 'lib/rfuzz/client.rb', line 214

def initialize(host, port, options = {})
  @options = options
  @host = host
  @port = port
  @cookies = options[:cookies] || {}
  @allowed_methods = options[:allowed_methods] || [:put, :get, :post, :delete, :head]
  @notifier = options[:notifier]
  @redirect = options[:redirect] || false
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(symbol, *args) ⇒ Object

Translates unknown function calls into PUT, GET, POST, DELETE, HEAD methods. The allowed HTTP methods allowed are restricted by the during construction with :allowed_methods => [:put, :get, …]



401
402
403
404
405
406
407
408
409
410
411
 
# File 'lib/rfuzz/client.rb', line 401

def method_missing(symbol, *args)
  if @allowed_methods.include? symbol
    method = symbol.to_s.upcase
    resp = send_request(method, args[0], args[1] || {})
    resp = redirect(symbol, resp) if @redirect

    return resp
  else
    raise "Invalid method: #{symbol}"
  end
end

Instance Attribute Details

#allowed_methods ⇒ Object

Access to the host, port, default options, and cookies currently in play



211
212
213
 
# File 'lib/rfuzz/client.rb', line 211

def allowed_methods
  @allowed_methods
end

#cookies ⇒ Object

Access to the host, port, default options, and cookies currently in play



211
212
213
 
# File 'lib/rfuzz/client.rb', line 211

def cookies
  @cookies
end

#host ⇒ Object

Access to the host, port, default options, and cookies currently in play



211
212
213
 
# File 'lib/rfuzz/client.rb', line 211

def host
  @host
end

#notifier ⇒ Object

Access to the host, port, default options, and cookies currently in play



211
212
213
 
# File 'lib/rfuzz/client.rb', line 211

def notifier
  @notifier
end

#options ⇒ Object

Access to the host, port, default options, and cookies currently in play



211
212
213
 
# File 'lib/rfuzz/client.rb', line 211

def options
  @options
end

#port ⇒ Object

Access to the host, port, default options, and cookies currently in play



211
212
213
 
# File 'lib/rfuzz/client.rb', line 211

def port
  @port
end

Instance Method Details

#build_request(out, method, uri, req) ⇒ Object

Builds a full request from the method, uri, req, and @cookies using the default @options and writes it to out (should be an IO).

It returns the body that the caller should use (based on defaults resolution).



230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
 
# File 'lib/rfuzz/client.rb', line 230

def build_request(out, method, uri, req)
  ops = @options.merge(req)
  query = ops[:query]

  # merge head differently since that's typically what they mean
  head = req[:head] || {}
  head = ops[:head].merge(head) if ops[:head]

  # setup basic headers we always need
  head[REQ_HOST] = encode_host(@host,@port)
  head[REQ_CONTENT_LENGTH] = ops[:body] ? ops[:body].length : 0

  # blast it out
  out.write(HTTP_REQUEST_HEADER % [method, encode_query(uri,query)])
  out.write(encode_headers(head))
  out.write(encode_cookies(@cookies.merge(req[:cookies] || {})))
  out.write("\r\n")
  ops[:body] || ""
end

#notify(event) ⇒ Object

Sends the notifications to the registered notifier, taking a block that it runs doing the :begins, :ends states around it.

It also catches errors transparently in order to call the notifier when an attempt fails.



447
448
449
450
451
452
453
454
455
456
457
 
# File 'lib/rfuzz/client.rb', line 447

def notify(event)
  @notifier.send(event, :begins) if @notifier

  begin
    yield
    @notifier.send(event, :ends) if @notifier
  rescue Object
    @notifier.send(event, :error) if @notifier
    raise $!
  end
end

#read_chunked_encoding(resp, sock, parser) ⇒ Object 



286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
 
# File 'lib/rfuzz/client.rb', line 286

def read_chunked_encoding(resp, sock, parser)
  out = StringIO.new
  input = StringIO.new(resp.http_body)

  # read from the http body first, then continue at the socket
  status, result = read_chunks(input, out, parser)

  case status
  when :incomplete_trailer
    if result.nil?
      sock.read(2)
    else
      sock.read((result.length - 2).abs)
    end
  when :incomplete_body
    out.write(sock.read(result))  # read the remaining
    sock.read(2)
  when :incomplete_header
    # push what we read back onto the socket, but backwards
    result.reverse!
    result.each_byte {|b| sock.ungetc(b) }
  when :finished
    # all done, get out
    out.rewind; return out.read
  when :eof_error
    # read everything we could, ignore
  end

  # then continue reading them from the socket
  status, result = read_chunks(sock, out, parser)

  # and now the http_body is the chunk
  out.rewind; return out.read
end

#read_chunks(input, out, parser) ⇒ Object 



250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
 
# File 'lib/rfuzz/client.rb', line 250

def read_chunks(input, out, parser)
  begin
    until input.closed?
      parser.reset
      chunk = HttpResponse.new
      line = input.readline("\r\n")
      nread = parser.execute(chunk, line, 0)

      if !parser.finished?
        # tried to read this header but couldn't
        return :incomplete_header, line
      end

      size = chunk.http_chunk_size ? chunk.http_chunk_size.to_i(base=16) : 0

      if size == 0
        return :finished, nil
      end

      remain =  size - out.write(input.read(size))
      return :incomplete_body, remain if remain > 0

      line = input.read(2)
      if line.nil? or line.length < 2
        return :incomplete_trailer, line
      elsif line != "\r\n"
        raise HttpClientParserError.new("invalid chunked encoding trailer")
      end
    end
  rescue EOFError
    # this is thrown when the header read is attempted and 
    # there's nothing in the buffer
    return :eof_error, nil
  end
end

#read_response(sock) ⇒ Object

Reads an HTTP response from the given socket. It uses readpartial which only appeared in Ruby 1.8.4. The result is a fully formed HttpResponse object for you to play with.

As with other methods in this class it doesn’t stop any exceptions from reaching your code. It’s for experts who want these exceptions so either write a wrapper, use net/http, or deal with it on your end.



328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
 
# File 'lib/rfuzz/client.rb', line 328

def read_response(sock)
  data, resp = nil, nil
  parser = HttpClientParser.new
  resp = HttpResponse.new

  notify :read_header do
    data = sock.readpartial(1024)
    nread = parser.execute(resp, data, 0)

    while not parser.finished?
      data += sock.readpartial(1024)
      nread += parser.execute(resp, data, nread)
    end
  end

  notify :read_body do
    if resp[TRANSFER_ENCODING] and resp[TRANSFER_ENCODING].index("chunked")
      resp.http_body = read_chunked_encoding(resp, sock, parser)
    elsif resp[CONTENT_LENGTH]
      cl = resp[CONTENT_LENGTH].to_i
      if cl - resp.http_body.length > 0
        resp.http_body += sock.read(cl - resp.http_body.length)
      elsif cl < resp.http_body.length
        STDERR.puts "Web site sucks, they said Content-Length: #{cl}, but sent a longer body length: #{resp.http_body.length}"
      end
    else
      resp.http_body += sock.read
    end
  end

  if resp[SET_COOKIE]
    cookies = query_parse(resp[SET_COOKIE], ';,')
    @cookies.merge! cookies
    @cookies.delete "path"
  end

  notify :close do
    sock.close
  end

  resp
end

#redirect(method, resp, *args) ⇒ Object

Keeps doing requests until it doesn’t receive a 3XX request.



414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
 
# File 'lib/rfuzz/client.rb', line 414

def redirect(method, resp, *args)
  @redirect.times do
    break if resp.http_status.index("3") != 0

    host = encode_host(@host,@port)
    location = resp[LOCATION]

    if location.index(host) == 0
      # begins with the host so strip that off
      location = location[host.length .. -1]
    end

    @notifier.redirect(:begins) if @notifier
    resp = self.send(method, location, *args)
    @notifier.redirect(:ends) if @notifier
  end

  return resp
end

#reset ⇒ Object

Clears out the cookies in use so far in order to get a clean slate.



436
437
438
 
# File 'lib/rfuzz/client.rb', line 436

def reset
  @cookies.clear
end

#send_request(method, uri, req) ⇒ Object

Does the socket connect and then build_request, read_response calls finally returning the result.



373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
 
# File 'lib/rfuzz/client.rb', line 373

def send_request(method, uri, req)
  begin
    sock = nil
    notify :connect do
      sock = TCPSocket.new(@host, @port)
    end

    out = StringIO.new
    body = build_request(out, method, uri, req)

    notify :send_request do
      sock.write(out.string + body)
      sock.flush
    end

    return read_response(sock)
  rescue Object
    raise $!
  ensure
    sock.close unless (!sock or sock.closed?)
  end
end