Gem Version Code Climate Pull Requests Commits since latest

Purpose

Cimas handles synchronizing CI configuration across multiple repositories.

When there are many repositories that need to share identical or similar CI (continuous integration) configuration, it is not only cumbersome but error prone to keep these configuration files in sync.

For example, adding new value to the CI test matrix for multiple repositories should not itself be a chore.

Note
 This problem can technically be mitigated if we can externalize and consolidate CI configuration outside of data repositories. However, most CI solutions don’t support external configuration (i.e. Travis, GitHub Actions, Appveyor). For GitHub, see: GHA external workflow, GHA calling another action

History

Cimas was built to synchronize CI configuration across the the Metanorma and Relaton repositories.

We had a hard time managing them individually!

Installation

Cimas (and its CLI command, cimas) relies on Git. Please ensure git is installed on your system.

Installing is as easy as:

gem install cimas

Or, by specifying it in your Gemfile if you’re using Bundler.

Concepts

CI configuration file

file to configure CI behavior of a repository

CI configuration master file

CI configuration file that applies to one or more target repositories

CI configuration master directory

directory (or a repo) where master CI configuration files reside

Cimas working area

local directory where Cimas works within

Cimas managed repository

repository with CI configuration managed by Cimas

Demonstration

The metanorma-build-scripts repository contains Cimas configuration under cimas-config/ for the Metanorma repositories.

  • cimas-config/: the CI master configuration directory

  • cimas-config/cimas.yml: the Cimas configuration file

Have a look to see how it’s being used!

Prerequisites

Setting up the CI configuration master directory

You need to first create a “CI configuration master directory” to contain master files for CI configuration. It is advisable to store the directory in Git for better version management.

For example, with Metanorma repositories, the CI master configuration directory is placed in the metanorma-build-scripts repository under cimas-config/.

Usage

Command line

Cimas works through the cimas executable. cimas provides the following sub-commands.

  • setup

  • sync

  • diff

  • push

  • pull

  • open-prs

  • lint (disabled right now)

Sub-command options

These options work across all sub-commands:

  • -d or --dry-run: Does not perform any destructive behavior.

  • -v (disabled for now)

cimas setup

The setup sub-command sets up the Cimas working area/directory with Git repos cloned as described in your Cimas configuration file (default: cimas.yml).

This command must be run before any other sub-commands are run, and must be run when the repository addresses change within the Cimas configuration file (it will prompt you).

cimas setup -f {cimas-config-file} -r {cimas-working-area}

# e.g.
# cimas setup -f cimas.yml -r ~/src/cimas-wd
Note
 This sub-command is considered “safe” as it does not alter the state of existing Git repositories within the Cimas working area.

cimas pull

The pull sub-command resets all the Git repositories to the original branch key set per-repo.

This command should be run before the cimas sync command to clean up state.

cimas pull -f {cimas-config-file} -r {cimas-working-area}

# e.g.
# cimas pull -f cimas.yml -r ~/src/cimas-wd
Note
 This sub-command is considered “destructive” as it resets states of all Git repositories in the Cimas working area.

cimas sync

The sync sub-command places the necessary CI configuration files, as described in the Cimas configuration file, into the desired repositories.

After copying the files, Cimas will also stage the changed files to Git.

This command must be run before the cimas push command.

cimas sync -f {cimas-config-file} -r {cimas-working-area} \
  -d {cimas-master-config-dir}

# e.g.
# cimas sync -f cimas.yml -r ~/src/cimas-wd \
#   -d ~/src/cimas-config
Note
 This sub-command is considered “destructive” as it resets states of all Git repositories in the Cimas working area before adding CI configuration files.

cimas diff

The diff sub-command provides a consolidated diff output between the repositories in the Cimas working area and remote. This is useful prior to running the cimas push command.

After copying the files, Cimas will also stage the changed files to Git.

This command must be run before the cimas push command.

cimas diff -f {cimas-config-file} -r {cimas-working-area} \
  -d {cimas-master-config-dir}

# e.g.
# cimas diff -f cimas.yml -r ~/src/cimas-wd \
#   -d ~/src/cimas-config
Note
 This sub-command is considered “safe” as it does not alter state of Git repositories.

cimas push

The push sub-command:

  • commits the changes made by the sync sub-command in a new branch;

  • pushes the new branch to the first Git remote.

This command must be run before the cimas open-prs command as the branches need to be pushed before pull-requests can be opened against them.

cimas push -f {cimas-config-file} -r {cimas-working-area} \
  -b {new-branch-for-commit} \
  -m {commit-message} \

# e.g.
# cimas push -f cimas.yml -r ~/src/cimas-wd \
#  -b my-new-ci-branch \
#  -m 'My commit message' \
#  [-g {group1,group2,...}]
Note
 This sub-command is considered “destructive” as it alters the state of all Git repositories in the Cimas working area by adding commits and branches.

cimas open-prs

The open-prs sub-command:

  • opens Pull Requests for all the specified repositories of the given branch;

  • creates Pull Request Review Requests for the created Pull Requests (if reviewers are set in cimas.yaml:settings or via the -w option);

  • assigns the created Pull Requests to assignees (if assignees are set in cimas.yaml:settings or via the -a option).

Since this command depends on GitHub privileged functionality, you must supply your GitHub Personal Access Token (PAT) via the GITHUB_TOKEN environment variable.

GITHUB_TOKEN=deadbeefdeadbeef; \
cimas open-prs -f {cimas-config-file} -r {cimas-working-area} \
  -b {new-branch-to-pr} \
  -m {pr-message} \

# e.g.
# cimas open-prs -f cimas.yml -r ~/src/cimas-wd \
#   -b my-new-ci-branch \
#   -m 'My pull-request message' \
#   [-w {reviewer1,reviewer2...}] \
#   [-a {assignee1,assignee2...}] \
#   [-g {group1,group2,...}]
Note
 This sub-command is considered “destructive” as it alters the state of GitHub repositories by creating pull requests.

Configuration

General

Cimas relies on reading a Cimas configuration file (default: cimas.yml) that specifies:

  • repository settings;

  • group settings; and

  • Cimas behavior

This YAML file needs to be in the following structure:

---
settings:
  {option-key}: {option-value}
  ...

repositories:
  {repo-name}:
    remote: {remote-name}
    branch: {branch-name}
    files:
      {CI-file-target-location}: {CI-configuration-master-file-location}
    ...
  ...

groups:
  {group-name}:
    - {repo-name}
    - ...

EXAMPLE: See metanorma/metanorma-build-scripts/cimas-config/cimas.yml for a working configuration.

settings

The settings object specifies run-time configuration. These options are merged with the command-line options, which have higher priority.

Syntax:

settings:
  {option-key}: {option-value}
  ...

The following options are available:

  • reviewers takes an array of GitHub user names as PR reviewers. This is only relevant to the cimas open-prs sub-command.

  • assignees takes an array of GitHub user names as assignees to PRs. This is only relevant to the cimas open-prs sub-command.

EXAMPLE: This example comes from metanorma/metanorma-build-scripts/cimas-config/cimas.yml.

settings:
  reviewers:
    - opoudjis
    - ronaldtse

repositories

The repositories object specifies all Git repositories that are managed by Cimas under this configuration file.

For example, when the cimas setup command is run, all of these repositories will be cloned under the Cimas working area.

Each repository is represented by a key under the repositories object.

  {repo-name-1}:
    remote: {remote-name}
    branch: {branch-name}
    files:
      {CI-file-target-location-1}: {CI-configuration-master-file-location-1}
      {CI-file-target-location-2}: {CI-configuration-master-file-location-2}

These attributes are mandatory for each repository:

  • remote: the remote Git location of this repository (i.e. where git clone can find this repository). SSH and HTTPS paths are supported. Single valued.

  • branch: the source branch and eventual branch to commit to (where a PR should be created against). Single valued.

  • files: composed of key value pairs of the “target file location within the repository” to the “master file location within the configuration master directory”. Multiple files are supported.

Syntax:

repositories:
  {repo-name-1}:
    remote: {remote-name}
    branch: {branch-name}
    files:
      {CI-file-target-location-1}: {CI-configuration-master-file-location-1}
      {CI-file-target-location-2}: {CI-configuration-master-file-location-2}
  {repo-name-2}:
    remote: {remote-name}
    branch: {branch-name}
    files:
      {CI-file-target-location-3}: {CI-configuration-master-file-location-3}
    ...
  ...

EXAMPLE: This example comes from metanorma/metanorma-build-scripts/cimas-config/cimas.yml.

repositories:
  metanorma-model-gb:
    remote: ssh://[email protected]/metanorma/metanorma-model-gb
    branch: master
    files:
      .github/workflows/macos.yml: gh-actions/model/macos.yml
      .github/workflows/ubuntu.yml: gh-actions/model/ubuntu.yml
      .github/workflows/windows.yml: gh-actions/model/windows.yml

groups

Cimas offers “grouping” functionality to allow you to work with groups of repositories. This is useful if your repositories fall into different categories, e.g. repositories for Ruby code vs C code that have different build routines.

There is a default group of all which applies if no group is specified.

Groups under the groups key are collections of repository names. Each group is represented by a key of the group’s name, with names of its repositories as array content.

One repository may belong to multiple groups. Groups have no bearing on what files to synchronize; the files must be specified per repository in the configuration file under the repositories section.

The -g switch in the various commands directly refer to the {group-name} specified in the configuration file.

Syntax:

groups:
  {group-name-1}:
    - {repo-name-1}
    - {repo-name-2}
    ...
  ...

EXAMPLE: This example comes from metanorma/metanorma-build-scripts/cimas-config/cimas.yml.

groups:
  model:
  - metanorma-model-iso
  - metanorma-model-gb
  - metanorma-model-standoc

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/metanorma/cimas. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

Code of Conduct

Everyone interacting in the Cimas project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.