NormalizeUrl
This gem can normalize HTTP(S) URLs by applying a certain set of transformations. After normalization, two different URLs that point to the same resource should look exactly the same.
For example:
http://example.com/products?product_id=123
HTTP://EXAMPLE.COM/products/?product_id=123
http://example.com/products/?product_id=123
http://example.com/foo/../products?product_id=123
http://example.com/products?product_id=123#comments-section
http://example.com//products/?product_id=123
http://example.com//products/?product_id=123&
http://example.com/products?utm_source=whatever&product_id=123&utm_medium=twitter&utm_campaign=blah
will all become http://example.com/products?product_id=123
after normalization.
Some of the transformations are potentially dangerous, since not all webservers comform to standards and some of them are just plain weird. So there is no gurantee that the URL will still work.
Installation
Add this line to your application's Gemfile:
gem "normalize_url"
And then execute:
$ bundle
Or install it yourself as:
$ gem install normalize_url
Usage
NormalizeUrl.process("http://example.com/products/?foo=bar&baz") # => "http://example.com/products?baz&foo=bar"
Each tranformation could be skipped by demand. All you need is to pass it as a
optional value to normalize
method:
NormalizeUrl.process("http://example.com/foo/", remove_trailing_slash: false) # => "http://example.com/foo/"
Transformations
Remove trailing slash. Option
:remove_trailing_slash
.Example:
http://example.com/products/
->http://example.com/products
Remove repeating slashes. Option
:remove_repeating_slashes
.Example:
http://example.com/foo//bar
->http://example.com/foo/bar
Remove hash fragment. Option
:remove_hash
.Example:
http://example.com/foo#bar
->http://example.com/foo
Reparse query params. Option
:reparse_query
.Example:
http://example.com/?foo=BAR+BAZ
->http://example.com/?foo=BAR%20BAZ
Remove known commonly used tracking query parameters. Option
:remove_tracking
.Example:
http://example.com/?utm_source=whatever
->http://example.com/
Sort query string. Option
:sort_query
.Example:
http://example.com/products/?foo=bar&baz
->http://example.com/products?baz&foo=bar
Contributing
- Fork it (https://github.com/rwz/normalize_url/fork)
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create a new Pull Request