Class: FastHttp::HttpClient

Inherits:
Object
  • Object
show all
Includes:
HttpEncoding
Defined in:
lib/fast_http/client.rb

Overview

The actual HttpClient that does the work with the thinnest layer between you and the protocol. All exceptions and leaks are allowed to pass through since those are important when testing. It doesn’t pretend to be a full client, but instead is just enough client to track cookies, form proper HTTP requests, and return HttpResponse hashes with the results.

It’s designed so that you create one client, and then you work it with a minimum of parameters as you need. The initialize method lets you pass in defaults for most of the parameters you’ll need, and you can simple call the method you want and it’ll be translated to an HTTP method (client.get => GET, client.foobar = FOOBAR).

Here’s a few examples:

client = HttpClient.new(:head => {"X-DefaultHeader" => "ONE"})
resp = client.post("/test")
resp = client.post("/test", :head => {"X-TestSend" => "Status"}, :body => "TEST BODY")
resp = client.put("/testput", :query => {"q" => "test"}, :body => "SOME JUNK")
client.reset

The HttpClient.reset call clears cookies that are maintained.

It uses method_missing to do the translation of .put to “PUT /testput HTTP/1.1” so you can get into trouble if you’re calling unknown methods on it. By default the methods are PUT, GET, POST, DELETE, HEAD. You can change the allowed methods by passing :allowed_methods => [:put, :get, ..] to the initialize for the object.

Notifications

You can register a “notifier” with the client that will get called when different events happen. Right now the Notifier class just has a few functions for the common parts of an HTTP request that each take a symbol and some extra parameters. See FastHttp::Notifier for more information.

Parameters

:head => {K => V}  or {K => [V1,V2]}
:query => {K => V} or {K => [V1,V2]}
:body => "some body" (you must encode for now)
:cookies => {K => V} or {K => [V1, V2]}
:allowed_methods => [:put, :get, :post, :delete, :head]
:notifier => Notifier.new
:redirect => false (give it a number and it'll follow redirects for that count)

Constant Summary collapse

TRANSFER_ENCODING =
"TRANSFER_ENCODING"
CONTENT_LENGTH =
"CONTENT_LENGTH"
"SET_COOKIE"
LOCATION =
"LOCATION"
HOST =
"HOST"
HTTP_REQUEST_HEADER =
"%s %s HTTP/1.1\r\n"
REQ_CONTENT_LENGTH =
"Content-Length"
REQ_HOST =
"Host"
CHUNK_SIZE =
1024 * 16
CRLF =
"\r\n"

Constants included from HttpEncoding

FastHttp::HttpEncoding::COOKIE, FastHttp::HttpEncoding::FIELD_ENCODING

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from HttpEncoding

#encode_cookies, #encode_field, #encode_headers, #encode_host, #encode_param, #encode_query, #escape, #query_parse, #unescape

Constructor Details

#initialize(host, port, options = {}) ⇒ HttpClient

Doesn’t make the connect until you actually call a .put,.get, etc.



229
230
231
232
233
234
235
236
237
238
# File 'lib/fast_http/client.rb', line 229

def initialize(host, port, options = {})
  @options = options
  @host = host
  @port = port
  @cookies = options[:cookies]
  @allowed_methods = options[:allowed_methods] || [:put, :get, :post, :delete, :head]
  @redirect = options[:redirect] || false
  @parser = HttpClientParser.new
  @ignore_data = options[:ignore_data]
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(symbol, *args) ⇒ Object

Translates unknown function calls into PUT, GET, POST, DELETE, HEAD methods. The allowed HTTP methods allowed are restricted by the during construction with :allowed_methods => [:put, :get, …]



402
403
404
405
406
407
408
409
410
411
412
# File 'lib/fast_http/client.rb', line 402

def method_missing(symbol, *args)
  if @allowed_methods.include? symbol
    method = symbol.to_s.upcase
    resp = send_request(method, args[0], args[1] || {})
    resp = redirect(symbol, resp) if @redirect

    return resp
  else
    raise HttpClientError.new("Invalid method: #{symbol}")
  end
end

Instance Attribute Details

#allowed_methodsObject

Access to the host, port, default options, and cookies currently in play



226
227
228
# File 'lib/fast_http/client.rb', line 226

def allowed_methods
  @allowed_methods
end

#cookiesObject

Access to the host, port, default options, and cookies currently in play



226
227
228
# File 'lib/fast_http/client.rb', line 226

def cookies
  @cookies
end

#hostObject

Access to the host, port, default options, and cookies currently in play



226
227
228
# File 'lib/fast_http/client.rb', line 226

def host
  @host
end

#optionsObject

Access to the host, port, default options, and cookies currently in play



226
227
228
# File 'lib/fast_http/client.rb', line 226

def options
  @options
end

#portObject

Access to the host, port, default options, and cookies currently in play



226
227
228
# File 'lib/fast_http/client.rb', line 226

def port
  @port
end

#sockObject

Access to the host, port, default options, and cookies currently in play



226
227
228
# File 'lib/fast_http/client.rb', line 226

def sock
  @sock
end

Instance Method Details

#build_request(out, method, uri, req) ⇒ Object

Builds a full request from the method, uri, req, and @cookies using the default @options and writes it to out (should be an IO).



243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
# File 'lib/fast_http/client.rb', line 243

def build_request(out, method, uri, req)
  ops = @options.merge(req)
  query = ops[:query]

  # merge head differently since that's typically what they mean
  head = req[:head] || {}
  head = ops[:head].merge(head) if ops[:head]

  # setup basic headers we always need
  head[REQ_HOST] = encode_host(@host,@port)
  head[REQ_CONTENT_LENGTH] = ops[:body] ? ops[:body].length : 0

  # blast it out
  out.write(HTTP_REQUEST_HEADER % [method, encode_query(uri,query)])
  out.write(encode_headers(head))
  if @cookies
    out.write(encode_cookies(@cookies.merge(req[:cookies] || {})))
  elsif req[:cookies]
    out.write(encode_cookies(req[:cookies]))
  end
  out.write(CRLF)
end

#read_chunked_body(header) ⇒ Object

Collects up a chunked body both collecting the body together and collecting the chunks into HttpResponse.raw_chunks[] for alternative analysis.



305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
# File 'lib/fast_http/client.rb', line 305

def read_chunked_body(header)
  @sock.push(header.http_body)
  header.http_body = ""
  header.raw_chunks = []

  while true
    chunk = read_chunked_header
    header.raw_chunks << chunk unless @ignore_data
    if !chunk.last_chunk?
      header.http_body << chunk.http_body unless @ignore_data
    else
      break # last chunk, done
    end
  end

  header
end

#read_chunked_headerObject

Used to process chunked headers and then read up their bodies.



285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
# File 'lib/fast_http/client.rb', line 285

def read_chunked_header
  resp = read_parsed_header
  @sock.push(resp.http_body)

  if !resp.last_chunk?
    resp.http_body = @sock.read(resp.chunk_size)

    trail = @sock.read(2)
    if trail != CRLF
      raise HttpClientParserError.new("Chunk ended in #{trail.inspect} not #{CRLF.inspect}")
    end
  end

  return resp
end

#read_parsed_headerObject

Does the read operations needed to parse a header with the @parser. A “header” in this case is either an HTTP header or a Chunked encoding header (since the @parser handles both).



269
270
271
272
273
274
275
276
277
278
279
280
281
# File 'lib/fast_http/client.rb', line 269

def read_parsed_header
  @parser.reset
  resp = HttpResponse.new
  data = @sock.read(CHUNK_SIZE, partial=true)
  nread = @parser.execute(resp, data, 0)

  while !@parser.finished?
    data << @sock.read(CHUNK_SIZE, partial=true)
    nread = @parser.execute(resp, data, nread)
  end

  return resp
end

#read_responseObject

Reads an HTTP response from the given socket. It uses readpartial which only appeared in Ruby 1.8.4. The result is a fully formed HttpResponse object for you to play with.

As with other methods in this class it doesn’t stop any exceptions from reaching your code. It’s for experts who want these exceptions so either write a wrapper, use net/http, or deal with it on your end.



340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
# File 'lib/fast_http/client.rb', line 340

def read_response
  resp = HttpResponse.new

  resp = read_parsed_header

  if resp.chunked_encoding?
    read_chunked_body(resp)
  elsif resp[CONTENT_LENGTH]
    needs = resp[CONTENT_LENGTH].to_i - resp.http_body.length
    # Some requests can actually give a content length, and then not have content
    # so we ignore HttpClientError exceptions and pray that's good enough
    if @ignore_data
      @sock.read(needs) if needs > 0 rescue HttpClientError
    else
      resp.http_body += @sock.read(needs) if needs > 0 rescue HttpClientError
    end
  else
    while true
      begin
        if @ignore_data
          @sock.read(CHUNK_SIZE, partial=true)
        else
          resp.http_body += @sock.read(CHUNK_SIZE, partial=true)
        end
      rescue HttpClientError
        break # this is fine, they closed the socket then
      end
    end
  end

  store_cookies(resp) if @cookies
  return resp
end

#redirect(method, resp, *args) ⇒ Object

Keeps doing requests until it doesn’t receive a 3XX request.



415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
# File 'lib/fast_http/client.rb', line 415

def redirect(method, resp, *args)
  @redirect.times do
    break if resp.http_status.index("3") != 0

    host = encode_host(@host,@port)
    location = resp[LOCATION]

    if location.index(host) == 0
      # begins with the host so strip that off
      location = location[host.length .. -1]
    end

    resp = self.send(method, location, *args)
  end

  return resp
end

#resetObject

Clears out the cookies in use so far in order to get a clean slate.



435
436
437
# File 'lib/fast_http/client.rb', line 435

def reset
  @cookies.clear
end

#send_request(method, uri, req) ⇒ Object

Does the socket connect and then build_request, read_response calls finally returning the result.



376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
# File 'lib/fast_http/client.rb', line 376

def send_request(method, uri, req)
  begin
    @sock = PushBackIO.new(TCPSocket.new(@host, @port))

    out = StringIO.new
    build_request(out, method, uri, req)
    body = req[:body] || "" unless method == :head or method == :get

    @sock.write(out.string + body)
    @sock.flush

    return read_response
  rescue Object
    raise $!
  ensure
    if @sock
      @sock.close
    end
  end
end

#store_cookies(resp) ⇒ Object

Reads the SET_COOKIE string out of resp and translates it into the @cookies store for this HttpClient.



325
326
327
328
329
330
331
# File 'lib/fast_http/client.rb', line 325

def store_cookies(resp)
  if @cookies and resp[SET_COOKIE]
    cookies = query_parse(resp[SET_COOKIE], ';')
    @cookies.merge! cookies
    @cookies.delete "path"
  end
end