Heroku to Control Plane `cpl` CLI

A playbook for migrating from Heroku to Control Plane

This playbook shows how to move "Heroku apps" to "Control Plane workloads" via an open-source cpl CLI on top of Control Plane's cpln CLI.

Heroku provides a UX and CLI that enables easy publishing of Ruby on Rails and other apps. This ease of use comes via many "Heroku" abstractions and naming conventions.

Control Plane, on the other hand, gives you access to raw cloud computing power. However, you need to know precisely how to use it.

To simplify migration to and usage of Control Plane for Heroku users, this repository provides a concept mapping and a helper CLI based on templates to save lots of day-to-day typing (and human errors).

Key Features
Concept Mapping
Installation
Steps to Migrate
Config
Environment
Database
In-memory Databases
Scheduled Jobs
CLI Commands Reference
Mapping of Heroku Commands to cpl and cpln
Examples
Migrating Postgres Database from Heroku Infrastructure
Migrating Redis Database from Heroku Infrastructure
Tips

Key Features

A cpl command to complement the default Control Plane cpln command with "Heroku style scripting." The Ruby source can serve as inspiration for your own scripts.
Easy to understand Heroku to Control Plane conventions in setup and naming.
Safe, production-ready equivalents of heroku run and heroku run:detached for Control Plane.
Automatic sequential release tagging for Docker images.
A project-aware CLI that enables working on multiple projects.

Concept Mapping

On Heroku, everything runs as an app, which means an entity that:

runs code from a Git repository.
runs several process types, as defined in the Procfile.
has dynos, which are Linux containers that run these process types.
has add-ons, including the database and other services.
has common environment variables.

On Control Plane, we can map a Heroku app to a GVC (Global Virtual Cloud). Such a cloud consists of workloads, which can be anything that can run as a container.

Heroku	Control Plane
app	GVC (Global Virtual Cloud)
dyno	workload
add-on	either a workload or an external resource
review app	GVC (app) in staging organization
staging env	GVC (app) in staging organization
production env	GVC (app) in production organization

On Heroku, dyno types are specified in the Procfile and configured via the CLI/UI; add-ons are configured only via the CLI/UI.

On Control Plane, workloads are created either by templates (preferred way) or via the CLI/UI.

For the typical Rails app, this means:

Function	Examples	On Heroku	On Control Plane
web traffic	`rails`, `sinatra`	`web` dyno	workload with app image
background jobs	`sidekiq`, `resque`	`worker` dyno	workload with app image
db	`postgres`, `mysql`	add-on	external provider or can be set up for development/testing with Docker image (lacks persistence between restarts)
in-memory db	`redis`, `memcached`	add-on	external provider or can be set up for development/testing with Docker image (lacks persistence between restarts)
others	`mailtrap`	add-on	external provider or can be set up for development/testing with Docker image (lacks persistence between restarts)

Installation

Install Node.js (required for Control Plane CLI).
Install Ruby (required for these helpers).
Install Control Plane CLI (adds cpln command) and configure credentials.

npm install -g @controlplane/cli
cpln login

Install Heroku to Control Plane cpl CLI, either as a Ruby gem or a local clone. For information on the latter, see CONTRIBUTING.md.

gem install cpl

Note: Do not confuse the cpl CLI with the cpln CLI. The cpl CLI is the Heroku to Control Plane playbook CLI. The cpln CLI is the Control Plane CLI.

Steps to Migrate

Click here to see the steps to migrate.

Config

Here's a complete example of all supported config keys explained:

# Keys beginning with "cpln_" correspond to your settings in Control Plane.

aliases:
  common: &common
    # Organization name for staging (customize to your needs).
    # Production apps will use a different organization, specified below, for security.
    cpln_org: my-org-staging

    # Example apps use only one location. Control Plane offers the ability to use multiple locations.
    default_location: aws-us-east-2

    # Allows running the command `cpl setup-app`
    # instead of `cpl apply-template gvc redis postgres memcached rails sidekiq`.
    setup:
      - gvc
      - redis
      - postgres
      - memcached
      - rails
      - sidekiq

    # Configure the workload name used as a template for one-off scripts, like a Heroku one-off dyno.
    one_off_workload: rails

    # Workloads that are for the application itself and are using application Docker images.
    app_workloads:
      - rails
      - sidekiq

    # Additional "service type" workloads, using non-application Docker images.
    additional_workloads:
      - redis
      - postgres
      - memcached

    # Configure the workload name used when maintenance mode is on (defaults to "maintenance").
    maintenance_workload: maintenance

    # Fixes the remote terminal size to match the local terminal size
    # when running the commands `cpl run` or `cpl run:detached`.
    fix_terminal_size: true

    # Apps with a deployed image created before this amount of days will be listed for deletion
    # when running the command `cpl cleanup-stale-apps`.
    stale_app_image_deployed_days: 5

    # Images that exceed this quantity will be listed for deletion
    # when running the command `cpl cleanup-images`.
    image_retention_max_qty: 20

    # Images created before this amount of days will be listed for deletion
    # when running the command `cpl cleanup-images` (`image_retention_max_qty` takes precedence).
    image_retention_days: 5

    # Run workloads created before this amount of days will be listed for deletion
    # when running the command `cpl run:cleanup`.
    stale_run_workload_created_days: 2

apps:
  my-app-staging:
    # Use the values from the common section above.
    <<: *common

  my-app-review:
    <<: *common

    # If `match_if_app_name_starts_with` is `true`, then use this config for app names starting with this name,
    # e.g., "my-app-review-pr123", "my-app-review-anything-goes", etc.
    match_if_app_name_starts_with: true

  my-app-production:
    <<: *common

    # Use a different organization for production.
    cpln_org: my-org-production

    # Allows running the command `cpl promote-app-from-upstream -a my-app-production`
    # to promote the staging app to production.
    upstream: my-app-staging

    # Used by the command `cpl promote-app-from-upstream` to run a release script before deploying.
    # This is relative to the `.controlplane/` directory.
    release_script: release_script

  my-app-other:
    <<: *common

    # You can specify a different `Dockerfile` relative to the `.controlplane/` directory (defaults to "Dockerfile").
    dockerfile: ../some_other/Dockerfile

Environment

There are two main places where we can set up environment variables in Control Plane:

In workload/container/env - those are container-specific and must be set up individually for each container.
In gvc/env - this is a "common" place to keep env vars which we can share among different workloads. Those common variables are not visible by default, and we should explicitly enable them via the inheritEnv property.

Generally, gvc/env vars are useful for "app" types of workloads, e.g., rails, sidekiq, as they can easily share common configs (the same way as on a Heroku app). They are not needed for non-app workloads, e.g., redis, memcached.

It is ok to keep most of the environment variables for non-production environments in the app templates as, in general, they are not secret and can be committed to the repository.

It is also possible to set up a Secret store (of type Dictionary), which we can reference as, e.g., cpln://secret/MY_SECRET_STORE_NAME/MY_SECRET_VAR_NAME. In such a case, we must set up an app Identity and proper Policy to access the secret.

# In `templates/gvc.yml`:
spec:
  env:
    - name: MY_GLOBAL_VAR
      value: 'value'
    - name: MY_SECRET_GLOBAL_VAR
      value: 'cpln://secret/MY_SECRET_STORE_NAME/MY_SECRET_GLOBAL_VAR'

# In `templates/rails.yml`:
spec:
  containers:
    - name: rails
      env:
        - name: MY_LOCAL_VAR
          value: 'value'
        - name: MY_SECRET_LOCAL_VAR
          value: 'cpln://secret/MY_SECRET_STORE_NAME/MY_SECRET_LOCAL_VAR'
      inheritEnv: true # To enable global env inheritance.

Database

There are several options for a database setup on Control Plane:

Heroku Postgres. It is the least recommended but simplest. We only need to provision the Postgres add-on on Heroku and copy its XXXXXX_URL connection string. This is good for quick testing but unsuitable for the long term.
Control Plane container. We can set it up as a workload using one of the default Docker Hub images. However, such a setup lacks persistence between container restarts. We can use this only for an example or test app where the database doesn't keep any serious data and where such data is restorable.
Any other cloud provider for Postgres, e.g., Amazon's RDS can be a quick go-to. Here are instructions for setting up a free tier of RDS.

Tip: If you are using RDS for development/testing purposes, you might consider running such a database publicly accessible (Heroku actually does that for all of its Postgres databases unless they are within private spaces). Then we can connect to such a database from everywhere with only the correct username/password.

By default, we have structured our templates to accomplish this with only a single free tier or low tier AWS RDS instance that can serve all your development/testing needs for small/medium applications, e.g., as follows:

aws-rds-single-pg-instance
  mydb-staging
  mydb-review-111
  mydb-review-222
  mydb-review-333

Additionally, we provide a default postgres template in this repository optimized for Control Plane and suitable for development purposes.

In-memory Databases

E.g., Redis, Memcached.

For development purposes, it's useful to set those up as Control Plane workloads, as in most cases, they don't keep any valuable data and can be safely restarted, which doesn't affect application performance.

For production purposes or where restarts are not an option, you should use external cloud services.

We provide default redis and memcached templates in this repository optimized for Control Plane and suitable for development purposes.

Scheduled Jobs

Control Plane supports scheduled jobs via cron workloads.

Here's a partial example of a template for a cron workload, using the app image:

kind: workload
name: daily-task
spec:
  type: cron
  job:
    # Run daily job at 2am.
    schedule: 0  2  *  *  *
    # "Never" or "OnFailure"
    restartPolicy: Never
  containers:
    - name: daily-task
      args:
        - bundle
        - exec
        - rails
        - db:prepare
      image: "/org/APP_ORG/image/APP_IMAGE"

A complete example can be found at templates/daily-task.yml, optimized for Control Plane and suitable for development purposes.

You can create the cron workload by adding the template for it to the .controlplane/templates/ directory and running cpl apply-template my-template -a my-app, where my-template is the name of the template file (e.g., my-template.yml).

Then to view the logs of the cron workload, you can run cpl logs -a my-app -w my-template.

CLI Commands Reference

Click here to see the commands.

You can also run the following command:

cpl --help

Mapping of Heroku Commands to `cpl` and `cpln`

Heroku Command	`cpl` or `cpln`
heroku ps	`cpl ps`
heroku config	?
heroku maintenance	`cpl maintenance`
heroku logs	`cpl logs`
heroku pg	?
heroku pipelines:promote	`cpl promote-app-from-upstream`
heroku psql	?
heroku redis	?
heroku releases	?

Examples

See the examples/ and templates/ directories of this repository.
See the .controlplane/ directory of this live example: react-webpack-rails-tutorial

Heroku to Control Plane cpl CLI