CLC-Promote Gem

Drives our CI/CD pipeline and provides functionality for:

  • Versioning cookbooks, data bags, roles, and environments
  • Manages Cookbook version constraints within environments
  • Manages the uploading of chef artifacts from CI to the QA chef Server and eventually to the production chef server
  • Promotes one environment's cookbooks to another

This functionality is exposed via a collection of rake tasks used on the CI server and a knife plugin used to perform deployments.

Configuration

CLC-Promote uses configuration settings to determine the values of:

  • Chef Server to contact
  • User and .pem file to use for authentication
  • Location of the chef repo

Knife Configuration

When using the knife promote commands, these are pulled from your knife.rb file as long as you are inside of the chef-repo.You can also set these using the usual knife parameters.

Additional knife config settings are used when interacting with a production chef server. The typical chef server config settings in a knife.rb are used for talking to a QA server. Production server settings are stored in the following settings:

knife[:promote_prod_url] = "https://chef.t3n.dom/organizations/clc_prod"
knife[:promote_prod_user] = user
knife[:promote_prod_client_key] = "#{current_dir}/#{node_name}_VA1.pem"

Rake Configuration

The Rake Tasks receive these values by passing a config object to the RakeTasks class.

config = Promote::Config.new({
  :repo_root => REPO_TOPDIR,
  :node_name => 'provisioner',
  :client_key => ENV['client_key'] || File.join(REPO_TOPDIR, 'cookbooks/provisioner/files/provisioner.pem'),
  :chef_server_url => ENV['chef_server_url'] || "https://172.22.10.121/organizations/clc_qa"
  })
Promote::RakeTasks.new(config)

The Config class can receive the following settings: Setting | :repo_root | mandatory :cookbook_directory | Defaults to #{root}/cookbooks :environment_directory | Defaults to #{root}/environments :data_bag_directory | Defaults to #{root}/data_bags :role_directory | Defaults to #{root}/roles :temp_directory | Defaults to /tmp/promote :node_name | mandatory :client_key | mandatory :chef_server_url | mandatody

CLC-Promote Rake Tasks

Note: See the below section on versioning for details on how version numbers are generated.

Promote:version_cookbook

Bumps the version of an individual cookbook.

Promote:version_cookbooks

Same as Promote:version_cookbook but iterates all cookbooks.

Promote:version_environment

Bumps the version of an individual environment file.

Promote:version_environments

Same as Promote:version_environment but iterates all environment files.

Promote:version_role

Bumps the version of an individual role file.

Promote:version_roles

Same as Promote:version_role but iterates all role files.

Promote:version_data_bag

Bumps the version of an individual databag entry. DEPRECATED (breaks encrypted data bags and vaults).

Promote:version_data_bags

Bumps the versions of all databag entries. DEPRECATED (breaks encrypted data bags and vaults).

Promote:sync_berksfiles

Performs a berks install on all cookbooks in the chef repo.

Promote:upload_cookbooks

Uploads all cookbook versions of an environment to the chef server. Note: only cookbooks that have been bumped will be uploaded. Uploaded cookbooks are frozen.

Promote:upload_environment

Uploads an environment file to the chef server

Promote:upload_roles

Uploads all role files to the chef server

Promote:upload_data_bags

Uploads all data bags to the chef server

Promote: constrain_environment

Given an environment and its environment cookbook, this task edits the environment file and creates cookbook constraints based on the Berksfile.lock of the environment cookbook.

Promote:promote_environment

Promotes one environment from another environment by copying the cookbook constraints of the source environment to the target (promoted) environment.

How are version numbers generated?

Version numbers are based on the last version tag which forms the major and minor version numbers and the number of commits in that tag which forms the build number. The number of commits are calculated from the root of the artifact being versioned. For instance, a cookbook's version would be based on the number of commits performed on all files within the top level directory of the cookbook being versioned.

Knife Promote

The knife promote command deploys an environment to production.

knife promote environment SOURCE_ENVIRONMENT TARGET_ENVIRONMENT [ --data-bags LIST ]

This command performs the following:

  1. Copies the cookbook version constraints from source to target
  2. Commits the constraints to version control
  3. Uploads the target environment to the QA chef server
  4. Downloads all secrets_* data bags and all roles from QA. Additional data bags can be specified on the command line. Note that data bags containing chef-vault keys ( *_keys.json ) are always skipped.
  5. Performs a version diff on all cookbooks between the target environment and production chef server
  6. Downloads all cookbooks from QA that are not on the production server
  7. Uploads all new cookbooks, the target environment, databags and roles to the production chef server.