Mongoid Traffic

Aggregated traffic logs stored in MongoDB. Fast and efficient logging via atomic updates of nested hashes in small number of MongoDB documents, semi-fast retrieveal and aggregation.

Installation

Add this line to your application's Gemfile:

gem 'mongoid_traffic'

And then execute:

$ bundle

Or install it yourself as:

$ gem install mongoid_traffic

Usage

Setup your class for storing the log:

class MyLog
    include Mongoid::Document
    include MongoidTraffic::Log
end

Log your traffic like this:

MongoidTraffic::Logger.log(MyLog, *args)

Or, if you prefer, directly on the MyLog class:

MyLog.log(*args)

This will (by default) create two MyLog documents: first with :df(date_from) and :dt(date_to) fields specifying monthly log, second with dates specifying daily log. Each log has an :access_count attribute that is incremented with subsequent .log calls.

Optional arguments

Time scope:

By default, the .log method creates/updates a document with aggregations for month and a document with aggregations for a day. You can however customize this behaviour like this:

MyLog.log(time_scope: %i(month week day))

The available options are: %(year month week day)

Scope:

MyLog.log(scope: '/pages/123')

Allows to create several logs for different scopes of your application (typically URLs).

User-Agent:

MyLog.log(user_agent: user_agent_string)

Logs platform-browser-version access count:

{ "Macintosh" => { "Safari" => { "8%2E0" => 1, "7%2E1" => 2 } } }

Please note the keys are escaped. You might want to unescape them using for example CGI::unescape.

Referer:

MyLog.log(referer: http_referer_string)

Logs referer access count:

{ "http%3A%2F%2Fwww%2Egoogle%2Ecom" => 1 }

Please note the keys are escaped. You might want to unescape them using for example CGI::unescape.

If the referer is included in the bot list the log will not be created.

Country (via IP address):

MyLog.log(ip_address: '123.123.123.123')

Logs access count by country code 2:

{ "CZ" => 100, "DE" => 1 }

Uses the GeoIP library to infer country_code from IP address.

You can use the countries gem to convert the country code to country name etc.

Unique id:

MyLog.log(unique_id: unique_id_string)

Logs access count by id:

{ "0123456789" => 100, "ABCDEFGHIJ" => 1 }

Typically you would pass it something like session_id to track unique visitors.

Rails

In case of Rails, you can use the .after_action macro with the #log_traffic helper method in your controllers:

class MyController < ApplicationController
    after_action -> { log_traffic(MyLog) }, only: [:show]
end

The method automatically infers most of the options from the controller request method (User-Agent, Referer, IP address) and unique id from the Rails session.

Additionally the :log_scoped_traffic method adds a scope by the current request path (/pages/123):

class MyController < ApplicationController
    after_action -> { log_scoped_traffic(MyLog) }, only: [:show]
end

You can override this behavior with custom scope like this:

class MyController < ApplicationController
    after_action -> { log_scoped_traffic(MyLog, scope: 'my-scope-comes-here') }, only: [:show]
end

It might be good idea to use both methods in order to log access to the whole site as well as access to individual pages:

class MyController < ApplicationController
    after_action -> { log_traffic(MyLog) }, only: [:show]
    after_action -> { log_scoped_traffic(MyLog) }, only: [:show]
end

Accessing the log

The log is accessed with a combination of Mongoid Criteria and aggregation methods.

Criteria

The following time based criteria are predefined as Mongoid scopes:

.yearly(year)
.monthly(month, year)
.weekly(week, year)
.daily(date)

To narrow down by scope:

.scoped_to(scope)

Aggregation method

.aggregate_on(:access_count)
.aggregate_on(:browsers)
.aggregate_on(:referers)
.aggregate_on(:countries)
.aggregate_on(:unique_ids)

Behind the scenes, this method will take all documents returned by your criteria and combines the values of the specified field (in case of :access_count it is simple sum of the values, in other cases it is sum of nested hashes).

Unique visits

Lastly, to retrieve the number of unique visits:

.sum(:unique_ids)

Examples

Typically you first query by time:

MyLog.daily(Date.today)

And eventually by scope:

MyLog.daily(Date.today).scoped_to('/pages/123')

Followed by an aggregation. For example on access count:

MyLog.daily(Date.today).scoped_to('/pages/123').aggregate_on(:access_count)

The scope query accepts regular expressions, which allows for aggregations on specific parts of your site. For example should you want to query for all pages that have path beginning with '/blog':

MyLog.monthly(8, 2014).scoped_to(/\A\/blog/).aggregate_on(:countries)

TODO

JavaScript + Rails helper to track client-side properties (screen/browser size)

Contributing

Fork it ( https://github.com/tomasc/mongoid_traffic/fork )
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create a new Pull Request