Module: ProxyFetcher

Defined in:
lib/proxy_fetcher.rb,
lib/proxy_fetcher/proxy.rb,
lib/proxy_fetcher/manager.rb,
lib/proxy_fetcher/version.rb,
lib/proxy_fetcher/document.rb,
lib/proxy_fetcher/exceptions.rb,
lib/proxy_fetcher/null_logger.rb,
lib/proxy_fetcher/client/client.rb,
lib/proxy_fetcher/configuration.rb,
lib/proxy_fetcher/document/node.rb,
lib/proxy_fetcher/client/request.rb,
lib/proxy_fetcher/providers/base.rb,
lib/proxy_fetcher/providers/xroxy.rb,
lib/proxy_fetcher/document/adapters.rb,
lib/proxy_fetcher/utils/http_client.rb,
lib/proxy_fetcher/providers/proxy_list.rb,
lib/proxy_fetcher/providers/http_tunnel.rb,
lib/proxy_fetcher/utils/proxy_validator.rb,
lib/proxy_fetcher/providers/gather_proxy.rb,
lib/proxy_fetcher/client/proxies_registry.rb,
lib/proxy_fetcher/providers/free_proxy_list.rb,
lib/proxy_fetcher/utils/proxy_list_validator.rb,
lib/proxy_fetcher/document/adapters/oga_adapter.rb,
lib/proxy_fetcher/providers/free_proxy_list_ssl.rb,
lib/proxy_fetcher/configuration/providers_registry.rb,
lib/proxy_fetcher/document/adapters/abstract_adapter.rb,
lib/proxy_fetcher/document/adapters/nokogiri_adapter.rb

Overview

Ruby / JRuby lib for managing proxies

Defined Under Namespace

Modules: Client, Exceptions, Providers, VERSION Classes: Configuration, Document, HTTPClient, Manager, NullLogger, ProvidersRegistry, Proxy, ProxyListValidator, ProxyValidator

Constant Summary collapse

Error =

Base exception class for all the ProxyFetcher exceptions.

Class.new(StandardError)

Class Method Summary collapse

Class Method Details

.config ⇒ ProxyFetcher::Configuration

Returns ProxyFetcher configuration.

Examples:

ProxyFetcher.config

#=> #<ProxyFetcher::Configuration:0x0000000241eec8 @user_agent="Mozilla/5.0, ...", @pool_size=10,
        @client_timeout=3, @proxy_validation_timeout=3, @provider_proxies_load_timeout=30,
        @http_client=ProxyFetcher::HTTPClient, @proxy_validator=ProxyFetcher::ProxyValidator,
        @providers=[:free_proxy_list, ...], @adapter=ProxyFetcher::Document::NokogiriAdapter>

Returns:



60
61
62
 
# File 'lib/proxy_fetcher.rb', line 60

def config
  @config ||= ProxyFetcher::Configuration.new
end

.configure {|config| ... } ⇒ ProxyFetcher::Configuration

Returns Configuration object.

Yields:

Yield Returns:

  • (optional, types, ...) —

    description

Returns:



72
73
74
 
# File 'lib/proxy_fetcher.rb', line 72

def configure
  yield config
end

.gem_version ⇒ Object

ProxyFetcher gem version.



6
7
8
 
# File 'lib/proxy_fetcher/version.rb', line 6

def self.gem_version
  Gem::Version.new VERSION::STRING
end

.logger ⇒ Logger, ProxyFetcher::NullLogger

Returns ProxyFetcher logger instance.

Returns:



80
81
82
83
84
 
# File 'lib/proxy_fetcher.rb', line 80

def logger
  return @logger if defined?(@logger)

  @logger = config.logger || NullLogger.new
end