ey_cloud_awareness

This gem makes it a little easier to live on the EngineYard cloud:

allow your instances to get information (metadata) about the app_master, db_master, utility workers, etc. in its EngineYard environment
automatically map cap tasks to all your instances
automatically update your ssh aliases

We use it over at brighterplanet.com.

DEPRECATED

Use rubygems.org/gems/engineyard-metadata instead.

Purpose

To define an unchanging interface to certain metadata that is buried deep inside JSON config files and EC2 API calls.

/etc/chef/dna.json must be readable

This gem depends on

/etc/chef/dna.json

being READABLE and containing certain attributes as named by EngineYard. You might have to add:

sudo 'chmod a+r /etc/chef/dna.json'

to your before_restart.rb hook.

Using the commandline tool directly

Running this command only works from EngineYard AppCloud Amazon EC2 instances.

seamus-absheres-macbook:~ $ ssh my_app_staging-app_master
Last login: Wed Sep 22 10:33:56 2010 from 9.9.9.9
deploy@ip-10-202-197-254 ~ $ sudo gem install ey_cloud_awareness --no-rdoc --no-ri
Successfully installed string_replacer-0.0.1
Successfully installed ey_cloud_awareness-0.2.0
2 gems installed
deploy@ip-10-202-197-254 ~ $ sudo chmod a+r /etc/chef/dna.json
deploy@ip-10-202-197-254 ~ $ ey_cloud_awareness
{"apps":[{"name":"my_app","ebuilds":[{"name":"dev-libs/oniguruma"}],"repository_name":"[email protected]:seamusabshere/my_app.git","database_name":"my_app", [...]

SSH into one of your EngineYard AppCloud Amazon EC2 instances.
Running sudo gem install ey_cloud_awareness --no-rdoc --no-ri makes sure the gem binary is installed somewhere in your path.
Running sudo chmod a+r /etc/chef/dna.json makes sure everybody can read the EngineYard DNA information.
Running ey_cloud_awareness, you should get back a JSON-encoded hash of instance, environment, app, and other metadata.

It returns a cleaned-up combination of Amazon EC2 and EngineYard metadata.

Capistrano

Put this in your config/deploy.rb (or your Capfile):

# don't miss this line just because it's at the top
load "#{Gem.searcher.find('ey_cloud_awareness').full_gem_path}/lib/ey_cloud_awareness/capistrano_tasks.rb"

task :my_production do
  role :app_master, 'my_app.com'
  find_and_execute_task 'eyc_setup'       # gets a fresh list of your environment's instances and sets roles
end

task :my_staging do
  role :app_master, 'staging.my_app.com'
  find_and_execute_task 'eyc_setup'       # gets a fresh list of your environment's instances and sets roles
end

# add more tasks if you have more cloud environments

If you want to use EngineYard tasks like

cap my_production monit:status
cap my_production deploy:web:disable
cap my_production deploy:web:enable
cap my_production passenger:restart
cap my_production nginx:restart

then you should make sure the tasks look like

[...]
require 'eycap/recipes'
ssh_options[:compression] = false
set :user, 'deploy'
set :runner, 'deploy'
[...]

task :my_production do
  role :app_master, 'my_app.com'
  find_and_execute_task 'eyc_setup'
  set :rails_env, 'production'          # required for eycap
  set :deploy_to, '/data/my_app'        # required for eycap too
end

task :my_staging do
  role :app_master, 'staging.my_app.com'
  find_and_execute_task 'eyc_setup'
  set :rails_env, 'production'          # required for eycap
  set :deploy_to, '/data/my_app'        # required for eycap too
end

SSH into your instances

You can easily create SSH aliases like…

ssh my_production-app_master

using the following task:

cap my_production eyc:ssh
cap my_staging eyc:ssh

That will magically update ~/.ssh/config with aliases like

ssh my_production-db_master
ssh my_production-app_master
ssh my_production-app0
ssh my_production-app1
ssh my_production-app2
ssh my_production-util0
ssh my_production-util1

ssh my_staging-app_master
ssh my_staging-db_master

Unfortunately, the counters are arbitrary and so the assignment of util0 versus util1, etc. may change every time you update your .ssh/config file with this task.

Once you’ve done the quickstart, try:

cap my_production eyc:app_servers # gets a list of your app instances, including app_master
cap my_production eyc:utilities   # ditto for utility instances
cap my_production eyc:db_servers  # ditto for db instances
cap my_production eyc:all         # gets a list of all your instances

Making your instances aware of each other

You can use EngineYardCloudInstance to pull up information on other instances in an environment:

>> db_master = EngineYardCloudInstance.db_master
=> #<EngineYardCloudInstance:0xb5e12f70 @instance_id="i-50cf5838">
>> db_master.dns_name
=> "ec2-67-201-47-30.compute-1.amazonaws.com"

You can also inspect the whole set of metadata itself. This may change over time:

>> pp db_master.data
=> {:block_device_mapping=>
      [{:ebs=>
         {:status=>"attached",
          :volumeId=>"vol-26172ee8",
          :deleteOnTermination=>"false",
          :attachTime=>"2010-04-07T21:09:37.000Z"},
        :deviceName=>"/dev/sdz2"},
       {:ebs=>
         {:status=>"attached",
          :volumeId=>"vol-26172ee8",
          :deleteOnTermination=>"false",
          :attachTime=>"2010-04-07T21:09:37.000Z"},
        :deviceName=>"/dev/sdz1"}],
      :launch_time=>"2010-04-07T21:08:10.000Z",
      :instance_type=>"c1.medium",
      :private_dns_name=>"domU-44-44-44-44-A4-02.compute-1.internal",
      :instance_state=>{:code=>"16", :name=>"running"},
      :ami_launch_index=>"0",
      :users=>
      [{:password=>"hoppAugEv",
        :username=>"deploy",
        :uid=>"1000",
        :comment=>"",
        :gid=>"1000"}],
      :environment=>
      {:framework_env=>"production",
       :name=>"my_staging",
       :stack=>"nginx_passenger"},
      :instance_id=>"i-50cf5838",
      :group_id=>"ey-my_staging-XXXXXXXXXXXXXXXXXXXX",
      :root_device_type=>"instance-store",
      :private_ip_address=>"10.201.102.201",
      :kernel_id=>"aki-9b00e5f2",
      :placement=>{:availabilityZone=>"us-east-1a"},
      :product_codes=>nil,
      :image_id=>"ami-7044a419",
      :reason=>nil,
      :dns_name=>"ec2-67-201-47-30.compute-1.amazonaws.com",
      :ip_address=>"199.99.99.99",
      :architecture=>"i386",
      :role=>"db_master",
      :monitoring=>{:state=>"disabled"}}

A note on caching and network needs

I tried to be smart about caching the results of network calls.

Stuff like the present instance id, which is pulled from an EC2 metadata server, is stored in

~/.ey_cloud_awareness/engine_yard_cloud_instance_id

Please let me know if this causes problems.