Class: FTW::Agent

Inherits:
Object
  • Object
show all
Includes:
Configuration, Protocol
Defined in:
lib/ftw/agent.rb

Overview

This should act as a proper web agent.

  • Reuse connections.

  • SSL/TLS.

  • HTTP Upgrade support.

  • HTTP 1.1 (RFC2616).

  • WebSockets (RFC6455).

  • Support Cookies.

All standard HTTP methods defined by RFC2616 are available as methods on this agent: get, head, put, etc.

Example:

agent = FTW::Agent.new
request = agent.get("http://www.google.com/")
response = agent.execute(request)
puts response.body.read

For any standard http method (like ‘get’) you can invoke it with ‘!’ on the end and it will execute and return a FTW::Response object:

agent = FTW::Agent.new
response = agent.get!("http://www.google.com/")
puts response.body.head

TODO(sissel): TBD: implement cookies… delicious chocolate chip cookies.

Defined Under Namespace

Modules: Configuration Classes: TooManyRedirects

Constant Summary collapse

STANDARD_METHODS =

List of standard HTTP methods described in RFC2616

%w(options get head post put delete trace connect)

Constants included from Configuration

Configuration::REDIRECTION_LIMIT

Constants included from CRLF

CRLF::CRLF

Instance Method Summary collapse

Methods included from Configuration

#configuration

Methods included from Protocol

#encode_chunked, #read_http_body, #read_http_body_chunked, #read_http_body_length, #read_http_message, #write_http_body, #write_http_body_chunked, #write_http_body_normal

Constructor Details

#initialize ⇒ Agent

Returns a new instance of Agent.



60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/ftw/agent.rb', line 60

def initialize
  @pool = FTW::Pool.new
  @logger = Cabin::Channel.get

  configuration[REDIRECTION_LIMIT] = 20

  @certificate_store = OpenSSL::X509::Store.new
  @certificate_store.add_file("/etc/ssl/certs/ca-bundle.trust.crt")
  @certificate_store.verify_callback = proc do |*args|
    p :verify_callback => args
    true
  end

end

Instance Method Details

#execute(request) ⇒ FTW::Response

Execute a FTW::Request in this Agent.

If an existing, idle connection is already open to the target server of this Request, it will be reused. Otherwise, a new connection is opened.

Redirects are always followed.

Parameters:

Returns:



180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
 
# File 'lib/ftw/agent.rb', line 180

def execute(request)
  # TODO(sissel): Make redirection-following optional, but default.

  connection, error = connect(request.headers["Host"], request.port)
  if !error.nil?
    p :error => error
    raise error
  end

  if request.protocol == "https"
    connection.secure(:certificate_store => @certificate_store)
  end
  response = request.execute(connection)

  redirects = 0
  # Follow redirects
  while response.redirect? and response.headers.include?("Location")
    # RFC2616 section 10.3.3 indicates HEAD redirects must not include a
    # body. Otherwise, the redirect response can have a body, so let's
    # throw it away.
    if request.method == "HEAD" 
      # Head requests have no body
      connection.release
    elsif response.content?
      # Throw away the body
      response.body = connection
      # read_body will consume the body and release this connection
      response.read_http_body { |chunk| }
    end

    # TODO(sissel): If this response has any cookies, store them in the
    # agent's cookie store

    redirects += 1
    if redirects > configuration[REDIRECTION_LIMIT]
      # TODO(sissel): include original a useful debugging information like
      # the trace of redirections, etc.
      raise TooManyRedirects.new("Redirect more than " \
          "#{configuration[REDIRECTION_LIMIT]} times, aborting.", response)
      # I don't like this api from FTW::Agent. I think 'get' and other methods
      # should return (object, error), and if there's an error 
    end

    @logger.debug("Redirecting", :location => response.headers["Location"])
    request.use_uri(response.headers["Location"])
    connection, error = connect(request.headers["Host"], request.port)
    # TODO(sissel): Do better error handling than raising.
    if !error.nil?
      p :error => error
      raise error
    end
    if request.protocol == "https"
      connection.secure(:certificate_store => @certificate_store)
    end
    response = request.execute(connection)
  end # while being redirected

  # RFC 2616 section 9.4, HEAD requests MUST NOT have a message body.
  if request.method != "HEAD"
    response.body = connection
  else
    connection.release
  end
 
  # TODO(sissel): If this response has any cookies, store them in the
  # agent's cookie store
  return response
end

#shutdown ⇒ Object

shutdown this agent.

This will shutdown all active connections.



252
253
254
255
256
257
258
 
# File 'lib/ftw/agent.rb', line 252

def shutdown
  @pool.each do |identifier, list|
    list.each do |connection|
      connection.disconnect("stopping agent")
    end
  end
end

#upgrade!(uri, protocol, options = {}) ⇒ Object

Send the request as an HTTP upgrade.

Returns the response and the FTW::Connection for this connection. If the upgrade was denied, the connection returned will be nil.



103
104
105
106
107
108
109
110
111
112
113
 
# File 'lib/ftw/agent.rb', line 103

def upgrade!(uri, protocol, options={})
  req = request("GET", uri, options)
  req.headers["Connection"] = "Upgrade"
  req.headers["Upgrade"] = protocol
  response = execute(req)
  if response.status == 101
    return response, response.body
  else
    return response, nil
  end
end

#websocket!(uri, options = {}) ⇒ Object

Make a new websocket connection.

This will send the http request. If the websocket handshake is successful, a FTW::WebSocket instance will be returned. Otherwise, a FTW::Response will be returned.

See #request for what the ‘uri’ and ‘options’ parameters should be.



122
123
124
125
126
127
128
129
130
131
132
133
134
 
# File 'lib/ftw/agent.rb', line 122

def websocket!(uri, options={})
  # TODO(sissel): Use FTW::Agent#upgrade! ?
  req = request("GET", uri, options)
  ws = FTW::WebSocket.new(req)
  response = execute(req)
  if ws.handshake_ok?(response)
    # response.body is a FTW::Connection
    ws.connection = response.body
    return ws
  else
    return response
  end
end