Class: ProxyFetcher::Providers::Base
- Inherits:
-
Object
- Object
- ProxyFetcher::Providers::Base
- Defined in:
- lib/proxy_fetcher/providers/base.rb
Overview
Base class for all the ProxyFetcher providers.
Direct Known Subclasses
FreeProxyList, FreeProxyListSSL, FreeProxyListSocks, FreeProxyListUS, HTTPTunnel, MTPro, ProxyList, Proxypedia, XRoxy
Class Method Summary collapse
-
.fetch_proxies!(*args) ⇒ Object
Just synthetic sugar to make it easier to call #fetch_proxies! method.
Instance Method Summary collapse
-
#fetch_proxies(filters = {}) ⇒ Object
(also: #fetch_proxies!)
Loads proxy provider page content, extract proxy list from it and convert every entry to proxy object.
-
#provider_headers ⇒ Hash
Provider headers required to fetch the proxy list.
- #provider_method ⇒ Object
- #provider_params ⇒ Object
- #provider_url ⇒ Object
- #xpath ⇒ Object
Class Method Details
.fetch_proxies!(*args) ⇒ Object
Just synthetic sugar to make it easier to call #fetch_proxies! method.
42 43 44 |
# File 'lib/proxy_fetcher/providers/base.rb', line 42 def self.fetch_proxies!(*args) new.fetch_proxies!(*args) end |
Instance Method Details
#fetch_proxies(filters = {}) ⇒ Object Also known as: fetch_proxies!
Loads proxy provider page content, extract proxy list from it and convert every entry to proxy object.
9 10 11 12 13 |
# File 'lib/proxy_fetcher/providers/base.rb', line 9 def fetch_proxies(filters = {}) raw_proxies = load_proxy_list(filters) proxies = raw_proxies.map { |html_node| build_proxy(html_node) }.compact proxies.reject { |proxy| proxy.addr.nil? } end |
#provider_headers ⇒ Hash
Returns Provider headers required to fetch the proxy list.
33 34 35 |
# File 'lib/proxy_fetcher/providers/base.rb', line 33 def provider_headers {} end |
#provider_method ⇒ Object
22 23 24 |
# File 'lib/proxy_fetcher/providers/base.rb', line 22 def provider_method :get end |
#provider_params ⇒ Object
26 27 28 |
# File 'lib/proxy_fetcher/providers/base.rb', line 26 def provider_params {} end |
#provider_url ⇒ Object
18 19 20 |
# File 'lib/proxy_fetcher/providers/base.rb', line 18 def provider_url raise NotImplementedError, "#{__method__} must be implemented in a descendant class!" end |
#xpath ⇒ Object
37 38 39 |
# File 'lib/proxy_fetcher/providers/base.rb', line 37 def xpath raise NotImplementedError, "#{__method__} must be implemented in a descendant class!" end |