Promotion makes it possible to repeatedly deploy an application in a fast and reliable way.
A key concept is the staging area. It is a folder (
/var/staging by default) where the files for each application are loaded into a named sub-folder (eg.
/var/staging/myapp). For example, a staging area may look like this:
/var |__ staging |__ webapp1 -- a web application |__ monitor -- sysadmin tool to monitor availability |__ analytics -- analytics app for admins |__ snort -- a custom config for snort
In this example, note that we also have an area for an installed application like snort. Although the binaries are installed through normal package installation, you may have a lot of time and work invested in the configuration. These files can of course be backed up and restored manually, but the purpose of Promotion is to make deployment fast and reliable. One way to achieve that is to store your valuable configuration files in source control, so you can reliably checkout the latest version for deployment each and every time. So the
snort staging area need only hold a few configuration files.
- Aside: why not just use version control?
The important files are spread all over the file system, so the only way to restore them all in a single command is to make the root directory a subversion working folder, add selected files to version control and then have
.svnfolders spread all over the place. Moreover, you'd need to run svn as root.
Each application has an XML deployment descriptor (eg.
/var/staging/myapp/deploy.xml) that describes the conditions necessary for the successful operation of the application.
The deployment descriptor describes how to move the files from staging to their correct locations in the file system. This simple requirement unrolls into a chain of others:
we need the folder to be there to put the file in
the permissions need to be correct on the file and folder
the ownerships must be set
that means we need the necessary user accounts created
which means we first need the right groups set up.
In addition to installing the application's files, there are system-wide configuration files that need to be adjusted to enable an application to startup or run properly:
Deployment descriptors can include recommended settings for these sensitive system files, but it does not change them. That would be too intrusive. Instead, Promotion reads the files to check if the recommended entries are present, and if they are not it displays a message for the admin explaining what needs to be done.
An important aspect is that a deployment is idempotent - you can do it as many times as you like and the result will be the same. This means its safe to make a small change to an application and re-deploy it in a few seconds, ready for testing.
Promotion is useful in a testing environment when changes and redeployments are frequent, but it shines in a production environment by reducing maintenance downtime and risk.
Promotion was originally designed for use on an OpenBSD system, but is configurable enough to work on any *nix system. Just modify the
promotion/config.rb configuration file to suit your system's paths.
Promotion will makes changes to your operating system.
DO NOT USE ON A PRODUCTION SYSTEM without rigorous testing in a virtual machine first.
Install the promotion gem, which displays a post-installation message:
$ sudo gem install promotion
To install the executables issue the following command:
$ sudo ruby -rubygems -e "require 'promotion/install'"
This will install the executables (
mkdeploy) and create the staging area
Now you are ready to use Promotion to deploy other applications.
For each application you want to deploy, create its staging folder:
$ cd /var/staging $ sudo svn checkout https://hosted/path/to/project/dist myapp
The project (or perhaps a
dist folder designed for deployment) is checked out into a folder with the given name.
Now you can promote the application:
$ sudo promote myapp Promoting myapp...OK.
If a database is involved, you can also migrate the database schema:
$ sudo evolve myapp Evolving the database to version 1023
which will execute any new schema migration scripts. If you execute it again, you will see:
$ sudo evolve myapp Already at version 1023.
If you want to revert to an earlier version of the database schema:
$ sudo devolve myapp 1017 Devolving the database to version 1017
which will execute the schema migration scripts in the
devolve folder, in reverse order from 1023 down to 1017.
To make it easier to create a deployment descriptor, use the
$ cd /var/staging $ mkdir myapp $ cd myapp $ mkdeploy Deployment descriptor template written to deploy.xml.
If you already have a deploy.xml file,
mkdeploy will not overwrite it:
$ mkdeploy deploy.xml already exists in the current directory. Exiting.
Deployment descriptor syntax
The deployment descriptor is typically named
deploy.xml and placed at the top of the application's staging folder (eg.
It's structure is described by the XML schema which can be found at at
/var/staging/promotion/promotion.xsd or at finalstep.com.au/schema/promotion.v100.xsd
The top level element is a Specification
<?xml version="1.0" encoding="UTF-8" ?> <Specification Name="myapp" Title="My Test App" xmlns="http://finalstep.com.au/promotion/v100" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://finalstep.com.au/schema/promotion.v100.xsd ../promotion/promotion.xsd">
which has a
Name matching the staging folder it resides in (
myapp), and a fuller
A description element is the first child:
<Description> The promotion manager ensures that the environment for an application is set up correctly, including folder and file permissions and ownership, startup scripts, sudoers privileges and environment variables. </Description>
The sequence is then driven by dependencies: you cannot set ownerships until you have users, and you cannot create a user without a group, so Groups come first. Each of these elements is optional however.
<Groups> <Group Gid="502" Name="_mysql" /> </Groups>
This creates a group with group ID
502 and the name
We can also create an admin and a user to run the MySQL daemon:
<Users> <User Name="richard" Class="staff" Gecos="Richard Kernahan" Uid="1002" Gid="1002" Home="/home/richard" Shell="/bin/ksh" Groups="wheel webadmin"/> <User Name="_mysql" Class="daemon" Gecos="MySQL Account" Uid="502"Gid="502" Home="/nonexistent" Shell="/sbin/nologin" /> </Users>
Gecosdefaults to the Name
Groupsis an optional space-separated list of groups the user should be added to
Before we can move files into place, we need the Folders set up:
<Folders Mode="0750" Owner="root" Group="wheel"> <Folder Owner="_mysql" Group="_mysql">/var/mysql</Folder> <Folder>/home/myapp</Folder> <Folder Clear="true">/var/myapp</Folder> </Folders>
You can have several
Folders elements, if desired. The defaults for all
Folder child elements may be set as attributes of the parent
Folders element. In this example, all the folders will have permissions
rwxr-x---, and ownership of
root:wheel. Note that the folder
/var/mysql has a different owner and group. Any Folder can override the default Mode, Owner, and Group.
Clear attribute will clear the contents of the folder (actually it completely removes the folder and recreates it). This happens before files are copied to it of course.
Finally we can specify how to move the files into place:
<Files Owner="root" Group="wheel" Mode="0644" Source="conf"> <File Backup="true">/etc/my.cnf</File> <File>/etc/myapp.conf</File> </Files> <Files Owner="root" Group="wheel" Mode="0750" Source="sbin"> <File>/usr/local/sbin/up!</File> <File>/usr/local/sbin/down!</File> <File>/usr/local/sbin/push!</File> </Files>
In this example, we have two sets of files. The first set is expected to be found under the
conf folder. Looking at the first file, we see the destination path
/etc/my.cnf. The source file should have the same filename
my.cnf and be located in the
conf folder. This results in Promotion executing a command such as:
cp -f /var/staging/myapp/conf/my.cnf /etc/my.cnf
This allows a flatter, more convenient project folder structure, since the deployment descriptor maps the files into their proper operating system locations.
Note that the file
my.cnf has an extra Backup attribute. This makes a backup of the original file to
my.cnf-original once only. This is a good idea for important original configuration files such as snort.conf or httpd.conf. They often have hundreds of comments that are useful for reference.
As for Folders, the Files element can define the default Owner, Group and Mode for all File children.
A convenient shorthand for copying all the files in a folder to another folder is the
Allfiles element. In this example, we will copy everything from the conf folder to the destination folder specified.
<Allfiles Group="bin" Mode="0644" Owner="root" Source="conf">/var/axonsec/conf</Allfiles>
To enable an application to startup automatically, a
Daemon element is needed. The name of the startup script is expected to be
/etc/rc.d/myapp, but may be overridden by the optional
<Daemon Flags="" Priority="10" User="" /> <Daemon Flags="-d $MYAPP_HOME" Priority="20" User="_myapp" Name="otherapp"/>
Flags attribute contains the command line options provided to the executable by
The lower the
Priority value, the earlier that script is run (high values start later).
User is the user running the process, typically an unprivileged user named
_myapp. Leave blank to run the startup script as root (eg. as when dropping privileges).
Cron jobs are often needed for an application to run smoothly.
<Crontab> <Schedule User="root" Hour="2" Minute="7" Comment="Backup the entire database at 2:07am each morning"> <Command><![CDATA[/usr/local/sbin/myappbak]]></Command> </Schedule> </Crontab>
In this example, we specify a job to be added to root's crontab, to backup the database at 2:07am each morning. The time specification is as defined in
Command is best wrapped safely in a CDATA section in case of XML-unsafe characters like $.
Comment will be inserted into the final crontab file just before the job specification.
If the application has a database, we need to specify the path to the DBMS client command line tool.
If using SQLite3, a
Database attribute is also needed to specify the file to operate on:
Log files can be automatically rotated with
newsyslog. This element specifies how to perform the log rotation:
<Newsyslog Owner="root" Group="wheel" Mode="0640" Count="5" When="M1D1" Zip="false" Binary="true" Restart="myappd">/var/log/myapp.log</Newsyslog>
In this example, the log file at
/var/log/myapp.log will be rotated at 1am on the first of each month (newsyslog.conf schedule format $M1D1). 5 backup copies will be kept in addition to the original. It will not be compressed with bzip and no log rotation message will be inserted (Binary=“false”). The
Restart attribute causes a command to be executed after the log file has been rotated:
so the application can reopen its log files.
Database schema migration
Four components are required to automate database schema migration:
Database migration scripts, stored in the
devolvesub-folders of the application's staging folder. These are normal migration scripts you might apply manually.
<Database>element in the deployment descriptor, containing the path to the database client command line tool. In the case of SQLite3, it also needs a
databaseattribute containing the path of the database file to operate on.
Privileges to allow the user to apply the migration scripts to the database, or else it will of course fail.
Credentials for the user stored privately in
~/.pgpass(unless using SQLite3). This allows the scripts to be executed in batch mode, without prompting for a password.