AWS Tracker

Models the state of one or more Amazon Web Services accounts in an ActiveRecord object graph.

ALPHA VERSION - not yet fully functional

What is it?

The AWS Tracker monitors one or more AWS accounts and the current state of their associated "resources" -- Instances, EBS volumes, RDS servers, and so on. The Tracker periodically polls the Amazon Web Services API for the latest information on each account's resources, and then maintains the corresponding rows in an ActiveRecord-compatible database. Users of the library can thereafter run multiple complex queries on the state of their AWS assets, with no network overhead.

Why is it?

The AWS Tracker is intended to be a foundation library, on top of which more complex AWS applications can be built. It decouples the process of downloading AWS state information from the process of displaying and/or acting on that information. Although an executable 'tracker' command-line program is included, the library is primarily intended for use from within Rails, or some other ActiveRecord context. An ActiveRecord database connection must be initialized before the aws_tracker library is loaded from within Ruby.

Installation

1) Install the AWS Tracker gem (use sudo as necessary).

gem install aws_tracker

2) Install your database adaptor of choice. Sqlite3 is installed with AWS Tracker, so this step may be optional.

Usage [from the command line]

1) First, generate an ActiveRecord-style configuration file.

Here are the sample contents of tracker_db.yml:

adapter: sqlite3
database: aws_tracker.sqlite3
pool: 5
timeout: 5000

2) Create the database to contain the data. This is not necessary if using sqlite.

3) To store your AWS account information in the database, export the credentials as environment variables. This is only needed the first time you run.

export AWS_ACCOUNT_NUMBER=[your AWS account number]
export AWS_ACCESS_KEY_ID=[your EC2 access key ID]
export AWS_ACCESS_KEY_SECRET=[your EC2 access key secret]

4) Run the tracker, and point it at the database config file

tracker tracker_db.yml --migrate --create-account
  • The --migrate argument updates the database to the latest version of the schema, and is only necessary for new databases, or when upgrading to a new version of the Tracker gem.
  • The --create-account argument stores the above environment values as an AWSAccount object in the tracker database.

Usage [from within Ruby]

1) Insert the necessary tables into your database. First, require 'AWSTracker/tasks' from your Rakefile, then run

rake db:migrate:tracker

2) In your Ruby app, first set up an ActiveRecord connection. In Rails, this is done for you automatically, but here's an example for a non-Rails app:

require 'active_record'
ActiveRecord::Base.establish_connection({
  :adapter => 'sqlite3', :database => 'aws_tracker.sqlite3'
})

3) Now require 'aws_tracker'. This will load the models and allow you to perform queries on the following AWS resources and their relationships:

  • AWSTracker::AWSAccount
  • AWSTracker::SecurityGroup
  • AWSTracker::Instance
  • AWSTracker::EBSVolume

But first, you need to create an AWSAccount, and use a Tracker object to fetch its status from AWS:

include AWSTracker          # Load model class names into namespace
a = AWSAccount.from_memory  # Create an account (see command-line usage)
a.save!                     # Save it in the database
t = Tracker.new             # A Tracker updates all accounts in the DB
t.delay = 15                # Update every 15 seconds
t.start                     # Runs in the background. Call 'stop' later

Development

This project is still in its early stages, but most of the framework is in place. More AWS resources need to be modeled, but the patterns for the code to do so are now laid out. Helping hands are appreciated!

1) Fetch the project code and bundle up...

git clone https://github.com/benton/aws_tracker.git
cd aws_tracker
bundle install

2) Create a database config and the necessary tables

cp config/database.yml.example config/database.yml
rake db:migrate:tracker

3) To run the tests, first export the credentials as environment variables, then rake spec.

export AWS_ACCOUNT_NUMBER=[your AWS account number]
export AWS_ACCESS_KEY_ID=[your EC2 access key ID]
export AWS_ACCESS_KEY_SECRET=[your EC2 access key secret]
rake spec