Class: Manticore::Client

Inherits:
Object
  • Object
show all
Includes:
ProxiesInterface
Defined in:
lib/manticore/client.rb,
lib/manticore/client/proxies.rb

Overview

Core Manticore client, with a backing PoolingHttpClientConnectionManager

Defined Under Namespace

Modules: ProxiesInterface Classes: AsyncProxy, BackgroundProxy, BaseProxy, StubProxy

Constant Summary collapse

DEFAULT_MAX_POOL_SIZE =

The default maximum pool size for requests

50
DEFAULT_REQUEST_TIMEOUT = 
60
DEFAULT_SOCKET_TIMEOUT = 
10
DEFAULT_CONNECT_TIMEOUT = 
10
DEFAULT_MAX_REDIRECTS = 
5
DEFAULT_EXPECT_CONTINUE = 
false
DEFAULT_STALE_CHECK = 
false

Instance Method Summary collapse

Methods included from ProxiesInterface

#async, #background, #respond_with

Constructor Details

#initialize(options = {}) {|builder, request_config| ... } ⇒ Client

Create a new HTTP client with a backing request pool. if you pass a block to the initializer, the underlying HttpClientBuilder and RequestConfig.Builder will be yielded so that you can operate on them directly.

Examples:

Simple instantiation and usage

client = Manticore::Client.new
client.get("http://www.google.com")

Instantiation with a block

client = Manticore::Client.new(socket_timeout: 5) do |http_client_builder, request_builder|
  http_client_builder.disable_redirect_handling
end

Parameters:

  • options (Hash) (defaults to: {}) —

    Client pool options

Options Hash (options):

  • user_agent (String) —

    The user agent used in requests.

  • pool_max (Integer) — default: 50 —

    The maximum number of active connections in the pool

  • pool_max_per_route (integer) — default: pool_max —

    Sets the maximum number of active connections for a given target endpoint

  • cookies (boolean) — default: true —

    enable or disable automatic cookie management between requests

  • compression (boolean) — default: true —

    enable or disable transparent gzip/deflate support

  • request_timeout (integer) — default: 60 —

    Sets the timeout for requests. Raises Timeout on failure.

  • connect_timeout (integer) — default: 10 —

    Sets the timeout for connections. Raises Manticore::Timeout on failure.

  • socket_timeout (integer) — default: 10 —

    Sets SO_TIMEOUT for open connections. A value of 0 is an infinite timeout. Raises Manticore::Timeout on failure.

  • request_timeout (integer) — default: 60 —

    Sets the timeout for a given request. Raises Manticore::Timeout on failure.

  • max_redirects (integer) — default: 5 —

    Sets the maximum number of redirects to follow.

  • expect_continue (boolean) — default: false —

    Enable support for HTTP 100

  • stale_check (boolean) — default: false —

    Enable support for stale connection checking. Adds overhead.

  • proxy (String) —

    Proxy host in form: proxy.org:1234

  • proxy (Hash) —

    Proxy host in form: ‘proxy.org’[, port: 80[, scheme: ‘http’]]

  • proxy (URI) —

    Proxy host as a URI object

  • keepalive (Boolean/Integer) — default: true —

    Whether to allow connections to be reused. Defaults to true. If an integer, then connections will be kept alive for this long when Connection: keep-alive is sent, but no Keep-Alive header is sent.

Yields:

  • (builder, request_config)

See Also:



121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/manticore/client.rb', line 121

def initialize(options = {})
  builder  = client_builder
  builder.set_user_agent options.fetch(:user_agent, "Manticore #{VERSION}")
  @use_cookies = options.fetch(:cookies, false)
  builder.disable_cookie_management unless @use_cookies
  builder.disable_content_compression if options.fetch(:compression, true) == false
  builder.set_proxy get_proxy_host(options[:proxy]) if options.key?(:proxy)

  # This should make it easier to reuse connections
  # TODO: Determine what this actually does!
  # builder.disable_connection_state

  keepalive = options.fetch(:keepalive, true)
  if keepalive == false
    builder.set_connection_reuse_strategy {|response, context| false }
  else
    builder.set_connection_reuse_strategy DefaultConnectionReuseStrategy.new
  end

  socket_config_builder = SocketConfig.custom
  socket_config_builder.setSoTimeout( options.fetch(:socket_timeout, DEFAULT_SOCKET_TIMEOUT) * 1000 )
  builder.set_default_socket_config socket_config_builder.build

  builder.set_connection_manager pool(options)

  request_config = RequestConfig.custom
  request_config.set_connection_request_timeout     options.fetch(:request_timeout, DEFAULT_REQUEST_TIMEOUT) * 1000
  request_config.set_connect_timeout                options.fetch(:connect_timeout, DEFAULT_CONNECT_TIMEOUT) * 1000
  request_config.set_socket_timeout                 options.fetch(:socket_timeout, DEFAULT_SOCKET_TIMEOUT) * 1000
  request_config.set_max_redirects                  options.fetch(:max_redirects, DEFAULT_MAX_REDIRECTS)
  request_config.set_expect_continue_enabled        options.fetch(:expect_continue, DEFAULT_EXPECT_CONTINUE)
  request_config.set_stale_connection_check_enabled options.fetch(:stale_check, DEFAULT_STALE_CHECK)
  request_config.set_circular_redirects_allowed false

  yield builder, request_config if block_given?

  builder.set_default_request_config request_config.build
  @client = builder.build
  @options = options
  @async_requests = []
  @stubs = {}
end

Instance Method Details

#clear_pending ⇒ Object

Remove all pending asynchronous requests.

Returns:

  • nil



245
246
247
248
 
# File 'lib/manticore/client.rb', line 245

def clear_pending
  @async_requests.clear
  nil
end

#clear_stubs! ⇒ Object

Wipe all currently-set stubs.



238
239
240
 
# File 'lib/manticore/client.rb', line 238

def clear_stubs!
  @stubs.clear
end

#delete(url, options = {}, &block) ⇒ Response

Perform a HTTP DELETE request

Examples:

Simple usage

body = client.delete("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).body

Passing a block as the success handler:

body = client.delete("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}) {|response| response.body }

Explicit success handler:

body = client.delete("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).
  on_success {|response| response.body }

Parameters:

  • url (String) —

    URL to request

  • options (Hash) (defaults to: {})

Options Hash (options):

  • params (Hash) —

    Hash of options to pass as request parameters

  • headers (Hash) —

    Hash of options to pass as additional request headers

  • proxy (String) —

    Proxy host in form: proxy.org:1234

  • proxy (Hash) —

    Proxy host in form: ‘proxy.org’[, port: 80[, scheme: ‘http’]]

  • proxy (URI) —

    Proxy host as a URI object

  • connect_timeout (Integer) —

    Request-specific connect timeout

  • socket_timeout (Integer) —

    Request-specific socket timeout

  • request_timeout (Integer) —

    Request-specific request timeout

  • max_redirects (Integer) —

    Request-specific maximum redirect limit

  • follow_redirects (Boolean) —

    Specify whether this request should follow redirects

Returns:

Raises:



203
204
205
 
# File 'lib/manticore/client.rb', line 203

def delete(url, options = {}, &block)
  request HttpDelete, url, options, &block
end

#execute! ⇒ Array

Execute all queued async requests

Returns:

  • (Array) —

    An array of the responses from the requests executed.



253
254
255
256
257
258
 
# File 'lib/manticore/client.rb', line 253

def execute!
  method = executor.java_method(:submit, [java.util.concurrent.Callable.java_class])
  result = @async_requests.map {|r| method.call r }
  @async_requests.clear
  result.map(&:get)
end

#executor ⇒ Object

Get at the underlying ExecutorService used to invoke asynchronous calls.



261
262
263
264
 
# File 'lib/manticore/client.rb', line 261

def executor
  create_executor_if_needed
  @executor
end

#get(url, options = {}, &block) ⇒ Response

Perform a HTTP GET request

Examples:

Simple usage

body = client.get("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).body

Passing a block as the success handler:

body = client.get("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}) {|response| response.body }

Explicit success handler:

body = client.get("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).
  on_success {|response| response.body }

Parameters:

  • url (String) —

    URL to request

  • options (Hash) (defaults to: {})

Options Hash (options):

  • params (Hash) —

    Hash of options to pass as request parameters

  • headers (Hash) —

    Hash of options to pass as additional request headers

  • proxy (String) —

    Proxy host in form: proxy.org:1234

  • proxy (Hash) —

    Proxy host in form: ‘proxy.org’[, port: 80[, scheme: ‘http’]]

  • proxy (URI) —

    Proxy host as a URI object

  • connect_timeout (Integer) —

    Request-specific connect timeout

  • socket_timeout (Integer) —

    Request-specific socket timeout

  • request_timeout (Integer) —

    Request-specific request timeout

  • max_redirects (Integer) —

    Request-specific maximum redirect limit

  • follow_redirects (Boolean) —

    Specify whether this request should follow redirects

Returns:

Raises:



179
180
181
 
# File 'lib/manticore/client.rb', line 179

def get(url, options = {}, &block)
  request HttpGet, url, options, &block
end

#head(url, options = {}, &block) ⇒ Response

Perform a HTTP HEAD request

Examples:

Simple usage

body = client.head("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).body

Passing a block as the success handler:

body = client.head("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}) {|response| response.body }

Explicit success handler:

body = client.head("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).
  on_success {|response| response.body }

Parameters:

  • url (String) —

    URL to request

  • options (Hash) (defaults to: {})

Options Hash (options):

  • params (Hash) —

    Hash of options to pass as request parameters

  • headers (Hash) —

    Hash of options to pass as additional request headers

  • proxy (String) —

    Proxy host in form: proxy.org:1234

  • proxy (Hash) —

    Proxy host in form: ‘proxy.org’[, port: 80[, scheme: ‘http’]]

  • proxy (URI) —

    Proxy host as a URI object

  • connect_timeout (Integer) —

    Request-specific connect timeout

  • socket_timeout (Integer) —

    Request-specific socket timeout

  • request_timeout (Integer) —

    Request-specific request timeout

  • max_redirects (Integer) —

    Request-specific maximum redirect limit

  • follow_redirects (Boolean) —

    Specify whether this request should follow redirects

Returns:

Raises:



191
192
193
 
# File 'lib/manticore/client.rb', line 191

def head(url, options = {}, &block)
  request HttpHead, url, options, &block
end

#options(url, options = {}, &block) ⇒ Response

Perform a HTTP OPTIONS request

Examples:

Simple usage

body = client.options("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).body

Passing a block as the success handler:

body = client.options("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}) {|response| response.body }

Explicit success handler:

body = client.options("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).
  on_success {|response| response.body }

Parameters:

  • url (String) —

    URL to request

  • options (Hash) (defaults to: {})

Options Hash (options):

  • params (Hash) —

    Hash of options to pass as request parameters

  • headers (Hash) —

    Hash of options to pass as additional request headers

  • proxy (String) —

    Proxy host in form: proxy.org:1234

  • proxy (Hash) —

    Proxy host in form: ‘proxy.org’[, port: 80[, scheme: ‘http’]]

  • proxy (URI) —

    Proxy host as a URI object

  • connect_timeout (Integer) —

    Request-specific connect timeout

  • socket_timeout (Integer) —

    Request-specific socket timeout

  • request_timeout (Integer) —

    Request-specific request timeout

  • max_redirects (Integer) —

    Request-specific maximum redirect limit

  • follow_redirects (Boolean) —

    Specify whether this request should follow redirects

Returns:

Raises:



209
210
211
 
# File 'lib/manticore/client.rb', line 209

def options(url, options = {}, &block)
  request HttpOptions, url, options, &block
end

#patch(url, options = {}, &block) ⇒ Response

Perform a HTTP PATCH request

Examples:

Simple usage

body = client.patch("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).body

Passing a block as the success handler:

body = client.patch("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}) {|response| response.body }

Explicit success handler:

body = client.patch("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).
  on_success {|response| response.body }

Parameters:

  • url (String) —

    URL to request

  • options (Hash) (defaults to: {})

Options Hash (options):

  • params (Hash) —

    Hash of options to pass as request parameters

  • headers (Hash) —

    Hash of options to pass as additional request headers

  • proxy (String) —

    Proxy host in form: proxy.org:1234

  • proxy (Hash) —

    Proxy host in form: ‘proxy.org’[, port: 80[, scheme: ‘http’]]

  • proxy (URI) —

    Proxy host as a URI object

  • connect_timeout (Integer) —

    Request-specific connect timeout

  • socket_timeout (Integer) —

    Request-specific socket timeout

  • request_timeout (Integer) —

    Request-specific request timeout

  • max_redirects (Integer) —

    Request-specific maximum redirect limit

  • follow_redirects (Boolean) —

    Specify whether this request should follow redirects

  • body (Hash) —

    Hash of options to pass as request body

Returns:

Raises:



215
216
217
 
# File 'lib/manticore/client.rb', line 215

def patch(url, options = {}, &block)
  request HttpPatch, url, options, &block
end

#pool_stats ⇒ Object

Return a hash of statistics about this client’s HTTP connection pool



165
166
167
168
169
170
171
172
173
 
# File 'lib/manticore/client.rb', line 165

def pool_stats
  stats = @pool.get_total_stats
  {
    max: stats.get_max,
    leased: stats.get_leased,
    pending: stats.get_pending,
    available: stats.get_available
  }
end

#post(url, options = {}, &block) ⇒ Response

Perform a HTTP POST request

Examples:

Simple usage

body = client.post("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).body

Passing a block as the success handler:

body = client.post("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}) {|response| response.body }

Explicit success handler:

body = client.post("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).
  on_success {|response| response.body }

Parameters:

  • url (String) —

    URL to request

  • options (Hash) (defaults to: {})

Options Hash (options):

  • params (Hash) —

    Hash of options to pass as request parameters

  • headers (Hash) —

    Hash of options to pass as additional request headers

  • proxy (String) —

    Proxy host in form: proxy.org:1234

  • proxy (Hash) —

    Proxy host in form: ‘proxy.org’[, port: 80[, scheme: ‘http’]]

  • proxy (URI) —

    Proxy host as a URI object

  • connect_timeout (Integer) —

    Request-specific connect timeout

  • socket_timeout (Integer) —

    Request-specific socket timeout

  • request_timeout (Integer) —

    Request-specific request timeout

  • max_redirects (Integer) —

    Request-specific maximum redirect limit

  • follow_redirects (Boolean) —

    Specify whether this request should follow redirects

  • body (Hash) —

    Hash of options to pass as request body

Returns:

Raises:



197
198
199
 
# File 'lib/manticore/client.rb', line 197

def post(url, options = {}, &block)
  request HttpPost, url, options, &block
end

#put(url, options = {}, &block) ⇒ Response

Perform a HTTP PUT request

Examples:

Simple usage

body = client.put("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).body

Passing a block as the success handler:

body = client.put("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}) {|response| response.body }

Explicit success handler:

body = client.put("http://example.com/some/resource", params: {foo: "bar"}, headers: {"X-Custom-Header" => "whee"}).
  on_success {|response| response.body }

Parameters:

  • url (String) —

    URL to request

  • options (Hash) (defaults to: {})

Options Hash (options):

  • params (Hash) —

    Hash of options to pass as request parameters

  • headers (Hash) —

    Hash of options to pass as additional request headers

  • proxy (String) —

    Proxy host in form: proxy.org:1234

  • proxy (Hash) —

    Proxy host in form: ‘proxy.org’[, port: 80[, scheme: ‘http’]]

  • proxy (URI) —

    Proxy host as a URI object

  • connect_timeout (Integer) —

    Request-specific connect timeout

  • socket_timeout (Integer) —

    Request-specific socket timeout

  • request_timeout (Integer) —

    Request-specific request timeout

  • max_redirects (Integer) —

    Request-specific maximum redirect limit

  • follow_redirects (Boolean) —

    Specify whether this request should follow redirects

  • body (Hash) —

    Hash of options to pass as request body

Returns:

Raises:



185
186
187
 
# File 'lib/manticore/client.rb', line 185

def put(url, options = {}, &block)
  request HttpPut, url, options, &block
end

#stub(url, stubs) ⇒ Object

Cause this client to return a stubbed response for this URL

Parameters:

  • url (String) —

    URL to stub for

  • stubs (Hash) —

    Hash of options to return for the stubbed response



228
229
230
 
# File 'lib/manticore/client.rb', line 228

def stub(url, stubs)
  @stubs[url] = stubs
end

#unstub(url) ⇒ Object

Cause this client to unstubbed previously-stubbed URL



233
234
235
 
# File 'lib/manticore/client.rb', line 233

def unstub(url)
  @stubs.delete(url)
end