Module: ProxyFetcher

Defined in:
lib/proxy_fetcher.rb,
lib/proxy_fetcher/proxy.rb,
lib/proxy_fetcher/manager.rb,
lib/proxy_fetcher/version.rb,
lib/proxy_fetcher/document.rb,
lib/proxy_fetcher/exceptions.rb,
lib/proxy_fetcher/null_logger.rb,
lib/proxy_fetcher/client/client.rb,
lib/proxy_fetcher/configuration.rb,
lib/proxy_fetcher/document/node.rb,
lib/proxy_fetcher/client/request.rb,
lib/proxy_fetcher/providers/base.rb,
lib/proxy_fetcher/providers/mtpro.rb,
lib/proxy_fetcher/providers/xroxy.rb,
lib/proxy_fetcher/document/adapters.rb,
lib/proxy_fetcher/utils/http_client.rb,
lib/proxy_fetcher/providers/proxy_list.rb,
lib/proxy_fetcher/providers/proxypedia.rb,
lib/proxy_fetcher/providers/http_tunnel.rb,
lib/proxy_fetcher/utils/proxy_validator.rb,
lib/proxy_fetcher/client/proxies_registry.rb,
lib/proxy_fetcher/providers/free_proxy_list.rb,
lib/proxy_fetcher/providers/proxyscrape_http.rb,
lib/proxy_fetcher/utils/proxy_list_validator.rb,
lib/proxy_fetcher/providers/free_proxy_list_us.rb,
lib/proxy_fetcher/providers/proxyscrape_socks4.rb,
lib/proxy_fetcher/providers/proxyscrape_socks5.rb,
lib/proxy_fetcher/document/adapters/oga_adapter.rb,
lib/proxy_fetcher/providers/free_proxy_list_ssl.rb,
lib/proxy_fetcher/providers/free_proxy_list_socks.rb,
lib/proxy_fetcher/configuration/providers_registry.rb,
lib/proxy_fetcher/document/adapters/abstract_adapter.rb,
lib/proxy_fetcher/document/adapters/nokogiri_adapter.rb

Overview

Ruby / JRuby lib for managing proxies

Defined Under Namespace

Modules: Client, Exceptions, Providers, VERSION Classes: Configuration, Document, HTTPClient, Manager, NullLogger, ProvidersRegistry, Proxy, ProxyListValidator, ProxyValidator

Constant Summary collapse

Error =

Base exception class for all the ProxyFetcher exceptions.

Class.new(StandardError)

Class Method Summary collapse

Class Method Details

.config ⇒ ProxyFetcher::Configuration

Returns ProxyFetcher configuration.

Examples:

ProxyFetcher.config

#=> #<ProxyFetcher::Configuration:0x0000000241eec8 @user_agent="Mozilla/5.0, ...", @pool_size=10,
       @client_timeout=3, @proxy_validation_timeout=3, @provider_proxies_load_timeout=30,
       @http_client=ProxyFetcher::HTTPClient, @proxy_validator=ProxyFetcher::ProxyValidator,
       @providers=[:free_proxy_list, ...], @adapter=ProxyFetcher::Document::NokogiriAdapter>

Returns:



69
70
71
72
73
 
# File 'lib/proxy_fetcher.rb', line 69

def config
  @__config_definition_lock__.synchronize do
    @config ||= ProxyFetcher::Configuration.new
  end
end

.configure ⇒ ProxyFetcher::Configuration

Returns Configuration object.

Yield Returns:

  • (optional, types, ...) —

    description

Returns:



83
84
85
 
# File 'lib/proxy_fetcher.rb', line 83

def configure
  @__config_access_lock__.synchronize { yield config }
end

.gem_version ⇒ Object

ProxyFetcher gem version.



6
7
8
 
# File 'lib/proxy_fetcher/version.rb', line 6

def self.gem_version
  Gem::Version.new VERSION::STRING
end

.logger ⇒ Logger, ProxyFetcher::NullLogger

Returns ProxyFetcher logger instance.

Returns:



91
92
93
94
95
 
# File 'lib/proxy_fetcher.rb', line 91

def logger
  return @logger if defined?(@logger)

  @logger = config.logger || NullLogger.new
end