Class: PrettyProxy

Inherits:
Rack::Proxy
  • Object
show all
Defined in:
lib/pretty_proxy.rb

Overview

The PrettyProxy class aggregate and validate the configuration of a proxy based in simple pretty url oriented rewriting rules. It’s too a rack app, and offers a abstract method for rewrite the responses returned by the proxy. The (X)HTML responses are rewritten to make the hyperlinks point to the proxy version of the page if it exist.

If you want to make a Rack app who use the proxy to point to another path of the same app you have to use a server in multithread mode, otherwise requests to the proxy will end in a deadlock. The proxy request the original page but the server don’t respond because is waiting the proxy request to be resolved. The proxy request don’t end because need the original page. A timeout error occur.

What this class can’t do but maybe will do in the future: smart handling of 3xx status response and chunked encoding (the chunks are concatened in the proxy and the transfer-encoding header removed); support more than deflate and gzip; exception classes with more than a message;

The exception classes (except Error) inherit Error, and Error inherit ArgumentError. They are empty yet, only have a message.

Glossary:

‘a valid proxy url/path’: The path (or the path of the url) start with the proxy_path and is followed by a original_path.

‘in(side)/out(side) the proxy control’: The url have (or not) the path starting with a original_path, and the scheme, port and host are the same of the original_domain.

CHANGELOG:

4.0.0
  * proxify_hyperlink don't take relative paths or urls anymore, only
    absolute urls. This is because the proxy url was used for a double
    purpose (know the proxy scheme+host+port and resolve relative
    hyperlinks). This can lead to the mistake of believing that the
    base url to resolve relative links in the page is the page url
    (what's false if the page has a base tag). See more in:
    http://www.w3.org/TR/html5/infrastructure.html#base-urls
  * proxify_html (and other methods who use it, as #call) use the base
    tag from the page to determine the base url, and add the the base
    tag (if the page don't have one) to simplify the assets proxification.
    All a[href] are changed to absolute urls.
  * rspec-html-matchers added as development dependency
3.0.0
  * return a String for unproxify_url (and not more a URI)
     because this is a change in the API (and can break code) the major
     version is now 3, if you don't use this method you can safely upgrade
  * depends in addressable gem
  * handles correctly the URIs without scheme (but with host)
    like '//duckduckgo.com/' (spec added for that)

Examples:

A terrible example

# You can run this example with 'rake heresy_example' in the gem folder
# and see the result in localhost:9292/proxy/
require 'pretty_proxy'

class Heresy < PrettyProxy
  def sugared_rewrite_response(triplet, requested_to_proxy_env, rewritten_env)
    status, headers, page = triplet
    page = page.gsub(/(MTG )?Magic(: The Gathering)?/, 'Yu-Gi-Oh')
    [status, headers, page]
  end
end

run Heresy.new('/proxy/', 'http://magiccards.info', '/')

Author:

  • Henrique Becker

Defined Under Namespace

Classes: ConfigError, Error, ProxyError

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(proxy_path, original_domain, original_paths, ignore_html_errors = false) ⇒ PrettyProxy

Note:

See the specs pretty_proxy_spec for examples and complete definition of invalid args.

Create a new PrettyProxy instance or raise a ConfigError. Clone the arguments.

Parameters:

  • proxy_path (String) —

    Start and end with slashes, represent the path in the proxy site who map to the proxy app (and, in consequence, to another path in the same or another site).

  • original_domain (String, URI) —

    A URL without path (no trailing slash), query or fragment (can have scheme (http), domain and port), the site to where the proxy map.

  • original_paths (String, #each) —

    The path (or the paths) to be mapped right inside the proxy_path (has to begin with slash).

  • ignore_html_errors (TrueClass, FalseClass) (defaults to: false) —

    If the argument of this parameter is true the #proxify_html try to ignore some exceptions that can be caused by an malformed (X)HTML and continue. Don’t silence #sugared_,rewrite_response. Experimental.

Raises:

  • PrettyProxy::ConfigError



108
109
110
111
112
113
114
115
116
117
118
119
120
121
 
# File 'lib/pretty_proxy.rb', line 108

def initialize(proxy_path, original_domain, original_paths, ignore_html_errors = false)
  Utils.validate_proxy_path(proxy_path)
  Utils.validate_original_domain_and_paths(original_domain, original_paths)

  @ignore_html_errors = ignore_html_errors
  @proxy_path = proxy_path.clone
  @original_domain = Addressable::URI.parse(original_domain.clone)
  @original_paths = Set.new 
  if original_paths.respond_to? :each
    original_paths.each { | value | @original_paths << value.clone }
  else
    @original_paths << original_paths.clone
  end
end

Instance Attribute Details

#ignore_html_errors ⇒ Object

Returns the value of attribute ignore_html_errors.



89
90
91
 
# File 'lib/pretty_proxy.rb', line 89

def ignore_html_errors
  @ignore_html_errors
end

#original_domain ⇒ Object

return the clone of the internal value



130
131
132
 
# File 'lib/pretty_proxy.rb', line 130

[:proxy_path, :original_domain, :original_paths].each do | reader |
  define_method(reader) { instance_variable_get("@#{reader.to_s}").clone }
end

#original_paths ⇒ Object

return the clone of the internal value (always a Set, no matter what is passed to initialize).



130
131
132
 
# File 'lib/pretty_proxy.rb', line 130

[:proxy_path, :original_domain, :original_paths].each do | reader |
  define_method(reader) { instance_variable_get("@#{reader.to_s}").clone }
end

#proxy_path ⇒ Object

return the clone of the internal value



130
131
132
 
# File 'lib/pretty_proxy.rb', line 130

[:proxy_path, :original_domain, :original_paths].each do | reader |
  define_method(reader) { instance_variable_get("@#{reader.to_s}").clone }
end

Instance Method Details

#call(env) ⇒ Object

Make this class a Rack app. It’s overriden to repass to the rewrite_response the original Rack environment (request to the proxy) and the rewritten env (modified to point the original page request). If you don’t know the parameters and return of this method, please read http://rack.rubyforge.org/doc/SPEC.html.



392
393
394
395
396
397
398
399
 
# File 'lib/pretty_proxy.rb', line 392

def call(env)
  # in theory we only need to repass the rewritten_env, any original env info
  #  needed can be passed as a environment application variable
  #  example: (env['app_name.original_path'] = env['PATH_INFO'])
  #  but to avoid this to be a common idiom we repass the original env too
  rewritten_env = rewrite_env(env)
  rewrite_response(perform_request(rewritten_env), env, rewritten_env)
end

#inside_proxy_control?(uri) ⇒ Boolean

Check if the URI::HTTP(S) is a page who can be accessed through the proxy.

Returns:

  • (Boolean) 


408
409
410
411
 
# File 'lib/pretty_proxy.rb', line 408

def inside_proxy_control?(uri)
  same_domain_as_original?(uri) &&
    valid_path_for_proxy?(@proxy_path + uri.path[1..-1])
end

#point_to_a_proxy_page?(hyperlink, proxy_domain) ⇒ Boolean

Take a url and the proxy domain (scheme, host and port) and return if the url point to a valid proxy page.

Returns:

  • (Boolean) 


433
434
435
436
 
# File 'lib/pretty_proxy.rb', line 433

def point_to_a_proxy_page?(hyperlink, proxy_domain)
  Utils.same_domain?(hyperlink, proxy_domain) &&
    valid_path_for_proxy?(hyperlink.path)
end

#proxify_html(html, proxy_url, mime_type) ⇒ String

Take a (X)HTML Document add a base tag (if none) and apply proxify_hyperlink to the ‘href’ attribute of each ‘a’ element. If the page has a base tag leave it unchanged. If a valid mime_type is passed as argument, but the html argument can’t be parsed by this mime-type it simple returns the first argument unchanged.

Parameters:

  • html (String) —

    A (X)HTML document.

  • proxy_url (String, URI::HTTP, URI::HTTPS) —

    The url where the the proxified version of the page will be displayed.

  • mime_type (String) —

    A string containing ‘text/html’ or ‘application/xhtml+xml’ (insensitive to case and any characters before or after the type). Define if the content will be parsed as xml or html. See this link for more info: www.w3.org/TR/xhtml-media-types/. Raise an exception if an invalid value is provided.

Returns:

  • (String) —

    A copy of the document with the changes applied, or the original string, if the document can’t be parsed.

Raises:

  • PrettyProxy::ProxyError



211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
 
# File 'lib/pretty_proxy.rb', line 211

def proxify_html(html, proxy_url, mime_type)
  parsed_html = Utils.parse_html_or_xhtml(html, mime_type)

  if parsed_html.nil?
    return html
  end

  # This isn't in conformance with the following document
  # http://www.w3.org/TR/html5/infrastructure.html#base-urls
  # but support to frames is not a priority
  document_original_url = unproxify_url(proxy_url)
  # in theory base must have a href... but to avoid an exception by bad html
  base_tag = parsed_html.at_css('base[href]')
  base_url = nil
  if base_tag
    base_url = Addressable::URI.parse(document_original_url)
                               .join(base_tag['href']).to_s
  else
    base_url = document_original_url
  end

  # the href isn't a obrigatory attribute of an anchor element
  parsed_html.css('a[href]').each do | hyperlink |
    begin
      absolute_hyperlink = Addressable::URI.parse(base_url)
                                           .join(hyperlink['href']).to_s
      hyperlink['href'] = proxify_hyperlink(absolute_hyperlink, proxy_url)
    rescue => e
      # Here we catch any exception derived from StandardError and do nothing
      # with it. This is a little risky, but the link in the href can be
      # wrong in many ways and yet be accepted by nokogiri. So to not
      # complexify the code we simply ignore when we can't proxify a link.
      raise e unless @ignore_html_errors
    end
  end

  unless base_tag
    is_XML = %r{application/xhtml\+xml}.match(mime_type)
    base_tag = "<base href='#{document_original_url}' #{is_XML ? '/' : ''}>"
    parsed_html.at_css('head').first_element_child
               .add_previous_sibling(base_tag)
  end

  parsed_html.to_s
end

#proxify_hyperlink(hyperlink, proxy_site) ⇒ String

Take a absolute URL and the scheme://host of the proxy page (can have path/?query#fragment, but they are ignored) and return the rewritten hyperlink. The url only is rewritten to point the proxyfied version if it’s under proxy control. If the url is under proxy control, but it’s also a url to the proxy, the url isn’t changed (to not double proxyfy, /proxy/ ~> /proxy/proxy/).

Parameters:

  • hyperlink (String, URI::HTTP, URI::HTTPS) —

    A string or URI object with a absolute url.

  • proxy_site (String, URI::HTTP, URI::HTTPS) —

    A URL with scheme://host to use in the hyperlink proxification.

Returns:

  • (String) —

    A absolute URL.

Raises:

  • PrettyProxy::ProxyError



182
183
184
185
186
187
188
189
190
191
192
 
# File 'lib/pretty_proxy.rb', line 182

def proxify_hyperlink(hyperlink, proxy_site)
  hyperlink = Addressable::URI.parse(hyperlink)
  proxy_site = Addressable::URI.parse(proxy_site)

  if inside_proxy_control?(hyperlink) &&
        ! point_to_a_proxy_page?(hyperlink, proxy_site)
    proxify_uri(hyperlink, proxy_site).to_s
  else
    hyperlink.to_s
  end
end

#rewrite_env(env) ⇒ Hash{String => String}

Modify a Rack environment hash of a request to the proxy version of a page to a request to the original page. As in Rack::proxy is used by #call for require the original page before call rewrite_response in the response. If you want to use your own rewrite rules maybe is more wise to subclass Rack::Proxy instead subclass this class. The purpose of this class is mainly implement and enforce these rules for you.

Parameters:

Returns:

  • (Hash{String => String}) —

    A unproxified copy of the argument.

Raises:

  • PrettyProxy::ProxyError



267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
 
# File 'lib/pretty_proxy.rb', line 267

def rewrite_env(env)
  env = env.clone
  url_requested_to_proxy = Rack::Request.new(env).url
  # Using URI, and not Addressable::URI because the port value is incorrect in the last
  unproxified_url = Addressable::URI.parse(unproxify_url(url_requested_to_proxy))

  if env['HTTP_HOST']
    env['HTTP_HOST'] = unproxified_url.host
  end
  env['SERVER_NAME'] = unproxified_url.host
  env['SERVER_PORT'] = unproxified_url.inferred_port.to_s

  if env['SCRIPT_NAME'].empty? && !env['PATH_INFO'].empty?
    env['PATH_INFO'] = unproxified_url.path
  end
  if !env['SCRIPT_NAME'].empty? && env['PATH_INFO'].empty?
    env['SCRIPT_NAME'] = unproxified_url.path
  end
  # Seriously, i don't know how to split again the unproxified url, so PATH_INFO gonna have the full path
  if (!env['SCRIPT_NAME'].empty? && !env['PATH_INFO'].empty?) ||
      (env['SCRIPT_NAME'].empty? && env['PATH_INFO'].empty?)
    env['PATH_INFO'] = unproxified_url.path
    env['SCRIPT_NAME'] = ''
  end

  env['REQUEST_PATH'] = unproxified_url.path
  env['REQUEST_URI'] = unproxified_url.path

  env
end

#rewrite_response(triplet, requested_to_proxy_env, rewritten_env) ⇒ Array<(Integer, Hash{String => String}, #each)>

Mainly apply the proxify_html to the body of the response if it is a html. Raise an error if the ‘content-encoding’ is other than deflate, gzip or identity. Change the ‘content-length’ header for the new body bytesize. Remove the ‘transfer-encoding’ if it is chunked, and act as not chunked. This method is inherited of Rack::Proxy, but in the original it have only the first parameter (the triplet). This version have the Rack env requested to the proxy and the rewritten Rack env as second and third parameters, respectively.

Parameters:

  • triplet (Array<(Integer, Hash{String => String}, #each)>) —

    A Rack response (see http://rack.rubyforge.org/doc/SPEC.html) for the request to the original site.

  • requested_to_proxy_env (Hash{String => String}) —

    A Rack environment hash. The requested to the proxy version.

  • rewritten_env (Hash{String => String}) —

    A Rack environment hash. The rewritten by the proxy to point to the original version.

Returns:

  • (Array<(Integer, Hash{String => String}, #each)>) —

    A unproxified copy of the first argument.

Raises:

  • PrettyProxy::ProxyError



316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
 
# File 'lib/pretty_proxy.rb', line 316

def rewrite_response(triplet, requested_to_proxy_env, rewritten_env)
  status, headers, body = triplet
  content_type = headers['content-type']
  return triplet unless 200 == status && (%r{text/html} =~ content_type ||
                        %r{application/xhtml\+xml} =~ content_type)


  # the #each method of body can't be called twice, but we need to call it here and it is called
  # after this method return, so we fake the body with a array of one string
  # we can't return a string (even it responds to #each) see: http://rack.rubyforge.org/doc/SPEC.html (section 'The Body')
  page = ''
  body.each do | chunk |
    page << chunk
  end

  case headers['content-encoding']
  when 'gzip' then page = Zlib::GzipReader.new(StringIO.new(page)).read
  when 'deflate' then page = Zlib::Inflate.inflate(page)
  when 'identity' then page = page
  when nil then page = page
  else
    fail ProxyError, 'unknown content-encoding, only encodings known are gzip, deflate and identity'
  end

  request_to_proxy = Rack::Request.new(requested_to_proxy_env)
  page = proxify_html(page, request_to_proxy.url, content_type)
  status, headers, page = sugared_rewrite_response([status, headers, page],
                                                    requested_to_proxy_env,
                                                    rewritten_env)

  case headers['content-encoding']
  when 'gzip'
    page_ = ''
    gzip_stream = Zlib::GzipWriter.new(StringIO.new(page_))
    gzip_stream.write page
    gzip_stream.close
    page = page_
  when 'deflate' then page = Zlib::Deflate.deflate(page)
  end

  headers['content-length'] = page.bytesize.to_s if headers['content-length']

  # TODO: find a way to make the code work with chunked encoding
  if 'chunked' == headers['transfer-encoding']
    headers.delete('transfer-encoding') 
    headers['content-length'] = page.bytesize.to_s
  end

  [status, headers, [page]]
end

#same_domain_as_original?(uri) ⇒ Boolean

Check if the #scheme, #host, and #port of the argument are equal to the original_domain ones.

Returns:

  • (Boolean) 


403
404
405
 
# File 'lib/pretty_proxy.rb', line 403

def same_domain_as_original?(uri)
  Utils.same_domain?(@original_domain, uri)
end

#sugared_rewrite_response(triplet, requested_to_proxy_env, rewritten_env) ⇒ Array<(Integer, Hash{String => String}, String)>

This method is abstract.

This method is called only over (X)HTML responses, after they are decompressed and the hyperlinks proxified, before they are compressed again and the new content-length calculated.

Note:

The body of the triplet is a String and not a object who respond to #each, the same has to be true in the return. Return a modified clone of the response, don’t change the argument.

The simplest way to make use of this class is subclass this class and redefine this method.

Parameters:

  • triplet (Array<(Integer, Hash{String => String}, String)>) —

    Not a valid Rack response, the third element is a string with the response body.

  • requested_to_proxy_env (Hash{String => String}) —

    A Rack environment hash. The requested to the proxy version.

  • rewritten_env (Hash{String => String}) —

    A Rack environment hash. The rewritten by the proxy to point to the original version.

Returns:

  • (Array<(Integer, Hash{String => String}, String)>) —

    A unproxified copy of the first argument.



383
384
385
 
# File 'lib/pretty_proxy.rb', line 383

def sugared_rewrite_response(triplet, requested_to_proxy_env, rewritten_env)
  triplet
end

#unproxify_url(url) ⇒ String

Take a proxy url and return the original URL behind the proxy. Preserve the query and fragment, if any. For the rewrite of a request @see rewrite_env.

Parameters:

  • url (String, URI::HTTP, URI::HTTPS) —

    A URL.

Returns:

  • (String) —

    The unproxified URI in a string.

Raises:

  • PrettyProxy::ProxyError



154
155
156
157
158
159
160
161
162
163
164
165
166
167
 
# File 'lib/pretty_proxy.rb', line 154

def unproxify_url(url)
  url = Addressable::URI.parse(url.clone)
  
  unless valid_path_for_proxy? url.path
    fail ProxyError, "'#{url.to_s}' isn't inside the proxy control, it can't be unproxified"
  end

  url.site = @original_domain.site
  url.path = url.path.slice((@proxy_path.size-1)..-1)

  url.to_s
rescue Addressable::URI::InvalidURIError
  raise ArgumentError, "the url argument isn't a valid uri"
end

#valid_path_for_proxy?(absolute_path) ⇒ Boolean

Check if the absolute path begin with a proxy_path and is followed by a original_paths element.

Returns:

  • (Boolean) 


415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
 
# File 'lib/pretty_proxy.rb', line 415

def valid_path_for_proxy?(absolute_path)
  return false unless absolute_path.start_with?(@proxy_path)

  path_without_proxy_prefix = absolute_path[(@proxy_path.size-1)..-1]

  @original_paths.any? do | original_path |
    # if we don't test this '/about' and '/about_us' will match
    if original_path.end_with? '/'
      path_without_proxy_prefix.start_with? original_path
    else
      path_without_proxy_prefix == original_path ||
        path_without_proxy_prefix.start_with?("#{original_path}/")
    end
  end
end