cloud-manager

This is a cloud services manager tool.
It was created to make the process of the deployment, the configuration and the testing of the managed services easier.
After installing the ruby gem, you only have to create your config file for the cloud-manager, run the manager, than in a few minutes, your desired config will be up and running.

Currently, it can handle the following services:

  • Hosting
    • AWS
      • EC2
  • Configuration management
    • Ansible
  • Testing
    • HTTP

I assume, you already familiar with the services you want to configure, so there won't be detailed instructions regarding those service parts.

Installation

Install the gem, than it's ready to use:

$ gem install cloud_manager

The gem provides two commands:

  • cloud_manager
    This is the main program. It has multiple subcommads for the services it can handle, i.e. list AWS EC2 instances.
    See the Usage section for more information.
  • cloud_manager_ansible_inventory
    This command is used internally when calling the ansible-playbook, to feed the generated inventory file into ansible.

Usage

Common usage

For a correct setup, the usage of the cloud-manager is as simple as running the following command:

$ cloud_manager manage my-cloud.yml

It will automatically do whatever you configured.

The configuration management setup (i.e. an ansible playbook) is assumed to be in the same directory where the cloud-manager is started from.
For a good example, clone this ready-to-use drupal project with cloud-manager support. You just have to run the following command from its directory:

$ cloud_manager manage cloud-manager/test1.yml

Depending on your setup, environment variables should be set, to make the cloud-manager run correctly, and to provide nice output.
I.e. to provide colored output in the ansible running stage, and to prevent host key checking, one should run something like this:

$ ANSIBLE_FORCE_COLOR=true ANSIBLE_HOST_KEY_CHECKING=false AWS_REGION=eu-central-1 cloud_manager manage my-cloud.yml

Other commands

The meaning of the subcommands can be read in the help:

$ cloud_manager help
Commands:
  cloud_manager aws COMMANDS          # Amazon AWS Commands
  cloud_manager help [COMMAND]        # Describe available commands or one specific command
  cloud_manager manage CONFIGURATION  # manage the given CONFIGURATION

You can see the help of AWS subcommands as well:

$ cloud_manager help aws
Commands:
  cloud_manager aws ec2-list-instances                                        # list EC2 instances
  cloud_manager aws ec2-list-vpcs                                             # list EC2 VPCs
  cloud_manager aws ec2-start-instances -i, --instance-ids=one two three      # start EC2 instances
  cloud_manager aws ec2-stop-instances -i, --instance-ids=one two three       # stop EC2 instances
  cloud_manager aws ec2-terminate-instances -i, --instance-ids=one two three  # terminate EC2 instances
  cloud_manager aws help [COMMAND]         

Configuration

A cloud-manager configuration file has to be in YAML format.

An example config looks like this:

services:
  ec2:
    machine1: # alias, the instance will have this name tag
      image_id: ami-accff2b1 # Amazon Machine Image (AMI) id
      key_name: 'smatyas' # AWS IAM key name
      instance_type: 't2.micro'

  ansible:
    machine1:
      playbooks: ['site.yml'] # these playbooks will be run against the machine
      hostgroups: ['drupal_hosts'] # these are the hostgroups the machine is part of
      hostvars: # these variables will be set for the host
        ansible_ssh_user: ubuntu
        drupal_site_name: "Drupal @ machine1 cloud server"
        drupal_db_name: drupal_db
        drupal_db_user: drupal_db_user
        drupal_db_password: uu6u6u5e546gwtsruj
        drupal_account_name: admin
        drupal_account_password: adminpass

  test:
    machine1:
      - type: http # the type parameter is mandatory
        body_regex: '/.*Welcome to Drupal @ machine1 cloud server.*/'
      - type: http
        url: /nonexistent-url # if the url starts with "/", than it will be appended to the base url
        code: 404

The cloud-manager does not handle (for now) dependencies among the configured services. They will be handled in the order they are given in the config file.

In the above example, a machine will be fired up in the Amazon EC2 cloud.
After the machine is reported to run, the cloud-manager tries to apply the configuration management service. As the machine may be not yet reachable, there are some fault-tolerance included: after an unsuccessful run, the cloud-manager waits 10 seconds, then tries to run the configuration management again. After the 5-th unsuccessful run, it'll give up.
When the configuration management finishes, the tests will run. For the tests, only the type is mandatory. For the http test, the public DNS name is get from the AWS EC2 service (but it could be overridden, see other examples). The test compares the HTTP response code and matches the regex to the response body.

More configuration file examples with comments, can be found in the example directory.

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake false to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/smatyas/cloud-manager.