About ClickSpotter

ClickSpotter is a real-time web server logfile monitor with IP geolocation support. It has three main features that differentiate it from the rest of the log file analyzers.

It processes your Apache access log file in real-time and produces consolidated live reports about your visitors and current page accesses.
It can pinpoint your visitors and the path they used through the Internet on a worldmap using IP geolocation.
It is a real desktop application, not just a bunch of generated web pages with a clumsy browser-based interface.

Installation

ClickSpotter is written in Ruby. It uses the standard Ruby mechanism for distribution. The format is called Ruby Gems. Alternatively, you can install from a tarball using setup.rb.

This is the first release. Consider it being beta quality right now!

Requirements

Ruby applications are platform independent. There is no need to compile anything. But ClickSpotter has a very small set of dependencies that you have to take care of first. Most Linux version already have them included. Just have sure you have the minimum required version.

Ruby: ClickSpotter is written in Ruby. You need a Ruby runtime environment to run it. This can be downloaded from RubyForge. Most Linux distributions usually have Ruby already included. ClickSpotter currently needs at least version 1.8.4. Older version have been reported to show frequent crashes due to what seems to be bugs in the networking libraries.
Qt: ClickSpotter uses the Qt Toolkit from Trolltech. This toolkit was choosen because it matches the elegance and beauty of Ruby. Almost every Linux distribution has a recent version of Qt included.
qtruby: To use Qt with Ruby, you also need the qtruby bindings. This is the magic glue that let’s Ruby interact with Qt. See the page on RubyForge for details. You need at least version 1.0.11.
traceroute: This is the commandline tool that comes with many Linux

distributions. If you don’t have it and can’t find a distribution package, you can get it at this site.

ClickSpotter: Get it from the Download Page

Installation Process

Just run as root

gem install ClickSpotter-0.1.1.gem

to install the gem file. It will install all components of the Gem in the appropriate place.

See here for instructions on how to install the tarball. If you use a non-standard prefix or datadir you need to add the path to AppConfig.rb. Otherwise ClickSpotter will not be able to find the worldmap images.

Update from previous versions

Updates work just like the installation.

gem update ClickSpotter-0.1.1.gem

Using ClickSpotter

ClickSpotter’s features were wrapped in an easy to use graphical user interface. It does not create an entry in your desktop menu or put an icon on the desktop. You need to start it from the commandline as clickspotter. Most other functions should be obvious.

Configuring your server log files

The first thing you need to do is define your log file(s). You need to provide a name, the location of the file and the base URL(s) of your server. The file must be accessible through the local filesystem of the machine that you are running on. If you don’t want to run ClickSpotter on your web server or log server, you can use

ssh your.web.server tail -f /var/log/apache/my.server-access_log > local_copy.log

to get a live copy of your server log on your local machine.

The log file must be in Apache2 combined log file format. The log entries should look like this:

212.114.251.44 - - [18/Jan/2006:16:11:20 +0100] "GET / HTTP/1.1" 200 9406 "-" "Mozilla/5.0 (compatible; Konqueror/3.4) KHTML/3.4.2 (like Gecko)"

ClickSpotter can only monitor one log at a time. But you can run multiple instances of ClickSpotter. You can pass the name of a log that you have defined when you start ClickSpotter from the command line. Here we assume that you have defined a log file with the name mylog.

clickspotter mylog

Other settings

You should also check out the second tab of the configuration dialog. It can be found from the menu under ‘Settings’ => ‘Configure ClickSpotter’.

The most important setting is the browser launch command. Set this to your favorit browser. The URL of the page is appended to this command to open certain web pages from within ClickSpotter.

The cache settings are also worth a view. See the peformance section later on for more details on this topic.

Geolocation Support

One of the highlights of ClickSpotter is the Geolocation support. It can not only map your visitors on a world map, but also show the path they took through the Internet to reach your server.

The Geolocation information is obtained during run-time from hostip.info. This is a public Geolocation service. The project that runs this site is still fairly young, so the records are not always correct. You can directly access their database by clicking the ‘Correct Data’ button or by clicking on the router list view. Please help to improve their database!

Performance considerations

In contrast to most other log analyzer tools, ClickSpotter is amazingly fast. This is certainly a plus for a real-time tool. But it has it’s limits. Both the memory consuption as well as the performance of the resolvers can limit the amount of traffic you can monitor. The DNS, Geolocation and Traceroute resolver work asynchronosly in the background. Information is displayed when available, but it does not prevent ClickSpotter from showing other details it already knowns. All screen are updated ever 5s with new information when available.

Bugs reports and other feedback

ClickSpotter was my first Ruby program. I mostly did it to practice my Ruby skills and evaluate the state of Ruby. Since I enjoyed programming Ruby so much I decided to turn it into a releaseable piece of software. But I’m not planning to spend lots of time on turning ClickSpotter into the Swiss Army Knife of log file monitors. I’m open to constructive feedback and patches, but bare with me if I don’t find the time to reply.

Copyright and License

This program is free software; you can redistribute it and/or modify it under the terms of version 2 of the GNU General Public License as published by the Free Software Foundation.