seesaw
by Matt Allen and Max Muermann
(http://rubyforge.org/projects/rails-oceania)
== DESCRIPTION:
seesaw: [verb] to change rapidly from one state or condition to another and back again.
Seesaw is a replacement/addition to the mongrel_cluster gem and allows you to perform a safe rippling restart of your mongrel cluster without dropping any requests.
Let's say you have a mongrel cluster setup with 7 individual mongrels on your server. Let's also say that you have to deploy a code update and restart all of your mongrels but your site is really busy and you cannot afford any downtime whatsoever.
When you execute:
mongrel_rails seesaw::bounce
This will happen:
1. your webserver configuration is switched to only use the front half of your mongrel pack (mongrels 1-4)
2. the webserver (apache or nginx) is gracefully restarted
3. the back half of your mongrel pack (mongrels 5-7) is restarted
4. your webserver configuration is switched to only use the back half of the pack
5. the webserver is gracefully restarted
6. front half mongrels are restared
7. webserver configuration switched back to full cluster configuration
8. webserver restarted one last time
Halfway through the process, your site is only running on half of the mongrels, but that will only result in reduced performance for a small amount of time, rather than downtime.
== SYNOPSIS:
* mongrel_rails::bounce
perform a rippling restart of the mongrel cluster
* mongrel_rails::switch --cluster CLUSTER
transfers the cluster into a guaranteed running configuration. CLUSTER can be one of [1|2|all], starting the front half, back half or full cluster respectively. If only a half cluster is started, the mongrels for the other half will be shut down.
* mongrel_rails::configure --server TYPE
generate a starting configuration for the chosen webserver [nginx|apache]. Default is apache.
== REQUIREMENTS:
* mongrel installation
* mongrel_cluster installed and configured for the application
== INSTALL:
sudo gem install seesaw
== CONFIGURATION
Generate an initial configuration with
mongrel_rails seesaw::configure
This will generate the webserver includes in config/http_cluster. You should move them out of the Rails project and into a well-known location.
*IMPORTANT*: the configuration files and the symlink are owned by the user excecuting the seesaw command. You can change the ownership, but you may need to change the symlink_cmd ocnfiguration parameter to use sudo.
Now change your webserver configuration as outlined below, and you're ready to go.
=== seesaw.yml
The seesaw configuration looks like this (example for nginx):
---
restart_cmd: sudo kill -HUP `cat /opt/local/nginx/logs/nginx.pid`
config_symlink: cluster.conf
mongrel_config_file: config/mongrel_cluster.yml
config_path: /opt/local/nginx/conf/cluster
config_files:
all: cluster_all.conf
1: cluster_1.conf
2: cluster_2.conf
symlink_cmd: ln -sf
In order to switch between cluster halves seamlessly, seesaw symlinks one of three different configuration files. The symlink must be references by the webserver configuration, see webserver-specific documentation below.
* config_path
The full path to the directory where the webserver configuration files live. More on that follows.
* config_symlink
The name of the symlink that points to one of the three cluster configurations. Defaults to cluster.conf. This needs to be included by the webserver configuration.
* config_files
* all
the name of the configuration fragment containing the full cluster setup
* 1
the name of the configuration fragment containing the front half of the cluster
* 2
the name of the configuration fragment containing the back half of the cluster
Other options:
* mongrel_config_file
the location of the mongrel_cluster configuration file (config/mongrel_cluster.yml by default)
* restart_cmd
the command to execute to grecefully restart the webserver. You will need to adjust this according to your filesystem layout. The defaults are:
* nginx: sudo kill -HUP `cat /opt/local/nginx/logs/nginx.pid`
* apache: sudo apachectl graceful
For nginx, adjust the path to the nginx.pid file to match your setup. You can adjust the location of the pid file in you nginx configuration.
=== Nginx
To include the seesaw-managed symlink pointing to the current cluster configuration, use something like this (mongrel_pack is the default name of the mongrel cluster generated by seesaw. Change in the three cluster configurations and the webserver config):
http {
...
include /full/path/to/generated/files/http_cluster/cluster.conf;
server {
...
location / {
#root share/nginx/html;
proxy_pass http://mongrel_pack;
}
...
}
}
=== Apache
Simply add
Include "/full/path/to/generated/files/http_cluster/cluster.conf"
where you would normally have your <Proxy balancer> section.
If you have any rewrite rules referencing the cluster name, change it accordingly:
RewriteRule ^/(.*)$ balancer://mongrel_pack%REQUEST_URI [P,QSA,L]
== LICENSE:
(The MIT License)
Copyright (c) 2007 FIX
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
by Matt Allen and Max Muermann
(http://rubyforge.org/projects/rails-oceania)
== DESCRIPTION:
seesaw: [verb] to change rapidly from one state or condition to another and back again.
Seesaw is a replacement/addition to the mongrel_cluster gem and allows you to perform a safe rippling restart of your mongrel cluster without dropping any requests.
Let's say you have a mongrel cluster setup with 7 individual mongrels on your server. Let's also say that you have to deploy a code update and restart all of your mongrels but your site is really busy and you cannot afford any downtime whatsoever.
When you execute:
mongrel_rails seesaw::bounce
This will happen:
1. your webserver configuration is switched to only use the front half of your mongrel pack (mongrels 1-4)
2. the webserver (apache or nginx) is gracefully restarted
3. the back half of your mongrel pack (mongrels 5-7) is restarted
4. your webserver configuration is switched to only use the back half of the pack
5. the webserver is gracefully restarted
6. front half mongrels are restared
7. webserver configuration switched back to full cluster configuration
8. webserver restarted one last time
Halfway through the process, your site is only running on half of the mongrels, but that will only result in reduced performance for a small amount of time, rather than downtime.
== SYNOPSIS:
* mongrel_rails::bounce
perform a rippling restart of the mongrel cluster
* mongrel_rails::switch --cluster CLUSTER
transfers the cluster into a guaranteed running configuration. CLUSTER can be one of [1|2|all], starting the front half, back half or full cluster respectively. If only a half cluster is started, the mongrels for the other half will be shut down.
* mongrel_rails::configure --server TYPE
generate a starting configuration for the chosen webserver [nginx|apache]. Default is apache.
== REQUIREMENTS:
* mongrel installation
* mongrel_cluster installed and configured for the application
== INSTALL:
sudo gem install seesaw
== CONFIGURATION
Generate an initial configuration with
mongrel_rails seesaw::configure
This will generate the webserver includes in config/http_cluster. You should move them out of the Rails project and into a well-known location.
*IMPORTANT*: the configuration files and the symlink are owned by the user excecuting the seesaw command. You can change the ownership, but you may need to change the symlink_cmd ocnfiguration parameter to use sudo.
Now change your webserver configuration as outlined below, and you're ready to go.
=== seesaw.yml
The seesaw configuration looks like this (example for nginx):
---
restart_cmd: sudo kill -HUP `cat /opt/local/nginx/logs/nginx.pid`
config_symlink: cluster.conf
mongrel_config_file: config/mongrel_cluster.yml
config_path: /opt/local/nginx/conf/cluster
config_files:
all: cluster_all.conf
1: cluster_1.conf
2: cluster_2.conf
symlink_cmd: ln -sf
In order to switch between cluster halves seamlessly, seesaw symlinks one of three different configuration files. The symlink must be references by the webserver configuration, see webserver-specific documentation below.
* config_path
The full path to the directory where the webserver configuration files live. More on that follows.
* config_symlink
The name of the symlink that points to one of the three cluster configurations. Defaults to cluster.conf. This needs to be included by the webserver configuration.
* config_files
* all
the name of the configuration fragment containing the full cluster setup
* 1
the name of the configuration fragment containing the front half of the cluster
* 2
the name of the configuration fragment containing the back half of the cluster
Other options:
* mongrel_config_file
the location of the mongrel_cluster configuration file (config/mongrel_cluster.yml by default)
* restart_cmd
the command to execute to grecefully restart the webserver. You will need to adjust this according to your filesystem layout. The defaults are:
* nginx: sudo kill -HUP `cat /opt/local/nginx/logs/nginx.pid`
* apache: sudo apachectl graceful
For nginx, adjust the path to the nginx.pid file to match your setup. You can adjust the location of the pid file in you nginx configuration.
=== Nginx
To include the seesaw-managed symlink pointing to the current cluster configuration, use something like this (mongrel_pack is the default name of the mongrel cluster generated by seesaw. Change in the three cluster configurations and the webserver config):
http {
...
include /full/path/to/generated/files/http_cluster/cluster.conf;
server {
...
location / {
#root share/nginx/html;
proxy_pass http://mongrel_pack;
}
...
}
}
=== Apache
Simply add
Include "/full/path/to/generated/files/http_cluster/cluster.conf"
where you would normally have your <Proxy balancer> section.
If you have any rewrite rules referencing the cluster name, change it accordingly:
RewriteRule ^/(.*)$ balancer://mongrel_pack%REQUEST_URI [P,QSA,L]
== LICENSE:
(The MIT License)
Copyright (c) 2007 FIX
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.