Concourse::Deployer

Provides easy installation and maintenance of an opinionated Concourse deployment.

  • external Postgres database
  • Github auth integration
  • LetsEncrypt integration for SSL cert management
  • Windows™ workers

Today this only supports deployment to GCP.

TL;DR

These five commands will give you a full Concourse deployment, with user-friendly prompting for configuration to external resources like a Postgres database and Github auth.

rake bbl:gcp:init[GCP_PROJECT_ID]
rake bbl:gcp:up
rake bosh:init
rake bosh:update
rake bosh:deploy

You can create and deploy a LetsEncrypt SSL cert:

rake letsencrypt:create letsencrypt:backup letsencrypt:import
rake bosh:deploy

Requirements

This gem requires:

Installation

Add this line to your application's Gemfile:

gem 'concourse-deployer'

And then run bundle or else install it directly with gem install concourse-deployer.

Usage

In your Rakefile:

require "concourse/deployer"

Concourse::Deployer.new.create_tasks!

Available tasks:

rake bbl:gcp:init[gcp_project_id]  # initialize bosh-bootloader for GCP
rake bbl:gcp:up                    # terraform your environment and deploy the bosh director
rake bosh:deploy                   # deploy concourse
rake bosh:init                     # prepare the concourse bosh deployment
rake bosh:update                   # upload stemcells and releases to the director
rake bosh:update:ubuntu_stemcell   # upload ubuntu stemcell to the director
rake letsencrypt:backup            # backup web:/etc/letsencrypt to local disk
rake letsencrypt:create            # create a cert
rake letsencrypt:import            # import letsencrypt keys into `secrets.yml` from backup
rake letsencrypt:renew             # renew the certificate
rake letsencrypt:restore           # restore web:/etc/letsencrypt from backup

See full instructions below.

A Note on Security

It's incredibly important that you don't risk leaking your credentials by committing them in the clear to a public git repository. This gem will use git-crypt to ensure files are encrypted which contain sensitive credentials.

Files which contain sensitive data:

  • service-account.key.json
  • bbl-state.json
  • secrets.yml
  • cluster-creds.yml
  • the vars subdirectory
  • letsencrypt.tar.gz (if you're using the letsencrypt SSL cert functionality)

You will see these files listed in .gitattributes invoking git-crypt for them.

Deploying to GCP

Step 0: create a GCP project, and create and config a Postgres database

Spin up a postgres database. Note the following information as you do so:

  • password
  • IP address

To set up connectivity to it, we'll first create client SSL certs, then only allow access via SSL, and finally allow inbound connections from any source IP (so long as it's via SSL).

  1. Under "SSL", create a client SSL cert, and download client-key.pem, client-cert.pem, and server-ca.pem for later use.
  2. Click "Allow only secured connections"
  3. Under "Authorization", add "0.0.0.0/0" as an allowed network.
  4. Under "Databases", create a database named atc.

Using an external db is a little annoying to do, but in the opinion of the author, it's worth it to have state persisted outside of the bosh-administered cluster, so that it can be torn down and rebuilt easily when necessary.

Step 1: Initialize bosh-bootloader and the project directory

$ rake bbl:gcp:init[gcp_project_id]

This will:

  • check that required dependencies are installed,
  • create an .envrc file with environment variables for bbl to work with GCP,
  • create .gitattributes entries to prevent sensitive files from being committed in the clear,
  • create a GCP service account, associate it with your project, and give it the necessary permissions,
  • and save that GCP service account information in service-account.key.json

NOTE: At this point, if you want to use a region/zone besides us-central1, you can edit your .envrc.

NOTE: service-account.key.json contains sensitive data.

Step 2: bbl up

$ rake bbl:gcp:up

Go get a coffee. In about 5 minutes, this will:

  • terraform a GCP environment,
  • spin up VMs running a bosh director and a jumpbox (a.k.a. "bastion host"),
  • create a load balancer with an external IP,
  • and save your state and credentials into bbl-state.json and the vars subdirectory.

NOTE: This task is idempotent. If you want to upgrade your bosh director (or stemcell) using a future version of bbl, you can re-run this (but read the bbl upgrade notes first).

NOTE: bbl-state.json and vars contain sensitive data.

Step 3: Prepare the Bosh deployment for Concourse

$ rake bosh:init

This will:

  • create a git submodule with a clone of concourse-bosh-deployment
  • create a secrets.yml file with credentials and external configuration you'll use to access concourse

You may be prompted for several things at this step, including:

  • database IP and password (noted in Step 0 above)
  • database server CA, client cert, and client key filenames (downloaded in Step 0 above)
  • Github OAuth2 credentials

NOTE: secrets.yml contains sensitive data.

NOTE: This task is idempotent! You can re-run this whenever you like.

NOTE: If you'd like a github user, team, or org to be members of Concourse's admin team, edit secrets.yml and add them to the /main_team section.

Step 4: Upload stemcell to the director

$ rake bosh:update

This will:

  • upload to the director the latest GCP stemcell
  • upload all necessary Bosh releases

NOTE: This task is idempotent! If you want to upgrade your releases or stemcell in the future, you should re-run this.

Step 5: deploy!

$ rake bosh:deploy

This will:

  • create or update a cluster-creds.yml file with automatically-generated cluster credentials,
  • bosh-deploy Concourse

NOTE: cluster-creds.yml and secrets.yml contain sensitive data.

NOTE: This task is idempotent! Yay Bosh.

Other Fun Things This Gem Does

Scale your Concourse deployment

Your first deployment will spin up one (1) web VM, and two (2) Linux worker VMs. But you can scale these numbers up as needed by editing the file scale-vars.yml, whose default contents looks like:

---
web_instances: 1
worker_instances: 2

Edit this file as appropriate for your needs, and re-run rake bosh:deploy.

Manage your letsencrypt SSL cert

$ rake letsencrypt:backup
$ rake letsencrypt:create
$ rake letsencrypt:restore
$ rake letsencrypt:import
$ rake letsencrypt:renew

NOTE: These tasks will create and use letsencrypt.tar.gz which contains sensitive data.

Custom bosh ops files

If you want to perform any custom operations on the manifest, put them in a file named operations.yml and they'll be pulled in as the final ops file during deployment.

Upgrading bbl

When a new version of bosh-bootloader comes out, just download it and make sure it's in your path as bbl (check by running bbl -v) and then:

$ rake bbl:gcp:up

... which will generate a new plan and then update the jumpbox, director, and cloud config. (See https://github.com/cloudfoundry/bosh-bootloader/blob/master/docs/upgrade.md for details.)

Make sure to commit into source control all the changes in your project directory (bbl-state.json, vars/, bosh-deployment/, etc.).

Upgrading concourse-bosh-deployment

If a new version of concourse comes out, and you'd like to upgrade, first read the release notes for Concourse to check for any relevant breaking changes.

Then:

$ rake bosh:update:concourse_deployment
$ rake bosh:deploy

Make sure you commit to source control the updated git submodule.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/flavorjones/concourse-deployer. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

TODO

  • [ ] update windows stemcell
  • [ ] include windows worker in manifest
  • [ ] deploy windows ruby tools release to the windows vms
  • [x] + x_frame_options: "SAMEORIGIN"
  • [x] + container_placement_strategy: random
  • [ ] enable encryption https://concourse.ci/encryption.html
  • [ ] allow scaling up/down by locally setting number of VMs (currently hardcoded in gem)
  • [ ] start using https://github.com/dpb587/caddy-bosh-release instead of the letsencrypt rake tasks

Things to follow up on:

Things I'm not immediately planning to do but that might be nice:

  • [ ] ops file to make the cloud-config come in under default GCP quota
  • [ ] ops files for a few variations on size/cost tradeoffs