PowerStencil

Gem Version Pipeline status

Power Stencil is the Swiss-army knife templating workflow for developers and ops.

See official website.

Overview

PowerStencil proposes a radical approach on how to manage your shared configuration.

Configuration is one of the most complex things to maintain, and anyone who participated in a large scale piece of software, be it for code, tools configuration or even documentation, knows how complex it is to maintain, keep consistent and avoid duplication on the long run, across the various projects that may share this configuration.

From a very high level point of view, the workfow PowerStencil proposes looks like:

simple-flow-image

PowerStencil provides development and operations friendly workflows to fully manage the maintenance of a complex shared config in order to generate anything you may want like documentation, static sites, code, configuration for multiple tools..., while avoiding duplication, and manage it like code !

It is a common pattern to introduce a database to manage shared configuration for CI/CD processes, and it is a very bad idea (see F.A.Q.) because it implies extra tooling, and specific developments that de-facto become single point of failure at the heart of your development process. Why would you do that when something as versatile and ubiquitous as Git exists that will integrate smoothly within your existing development and operations workflows, builds, CI/CD processes, while benefiting from all the goodness provided by Git, like branches, decentralization, versioning, tags, code reviews, traceability, simplified fallbacks etc... ?

A PowerStencil project is composed of a data repository and a pretty standard templating flow to generate whatever is needed by your project or organization and with PowerStencil everything is maintained in a git repository: data, templates, specific code without sacrificing on data integrity and relational features. No more config loss, no more change applied without a full lineage.

:information_source: With PowerStencil data can either be managed as documents to edit (Yaml) or fully as code within a powerful shell !!. Choose the method that fits the most to what you are doing.

The PowerStencil gem provides the power_stencil executable which is a pluggable CLI to:

  • Manage your project relational-integrity-checked data in a versioned, git diff friendly yet flexible format.
  • Visualize your data relations with graphviz. Customize generated graphs easily.
  • Easily manage templates to generate whatever your project/organization requires.
  • Define builds that you can easily embed in your CI/CD process or use them standalone.
  • Propose mechanism to temporarily override some data to test changes impact without polluting the official repository data.
  • Ensure re-usability with a versatile extension mechanism, including plugins.

Pfff, yet another templating engine...?

Actually PowerStencil is not a templating engine. It uses already existing and proven templating engines.

PowerStenil currently embeds two templating engines:

  • ERB is the default. This is a pretty standard and very well known engine in the Ruby world and especially for people who already used the Rails framework. This is a generic templating engine which can handle any use-case. Each template will be processed by default using the ERB engine.
  • When it comes to generate mark-up documents like XML or HTML, many people prefer to use Haml, as it designed towards generating these kinds of documents and tends to end-up with much clearer templates. Any template whose names ends by .haml will be processed by the Haml engine by default instead of ERB.

So PowerStencil is definitely not a templating engine, it is actually much more than that. It is a framework to manage your templates, design the relational data you can use in your templates, manage the versioning, define the way you build things etc... Read this documentation (and especially the getting started section which is a tutorial in itself).

Installation

You need a working install of the Ruby language (>= 2.3). You should have bundler installed (gem install bundler). This is not added as an automatic dependency as updating bundler may not be what you want. Yet any working install of Ruby should consider having bundler installed.

:hand: It is strongly advised to have git installed on your system, as PowerStencil is integrated with git to provide a better user experience.

Then you can install the PowerStencil gem by issuing the usual:

$ gem install power_stencil

If you want to create Graphviz graphs, you probably may want to install it for your system. If you are using an apt-based system like Ubuntu or Debian it may probably be as simple as:

$ sudo apt install graphviz

If you use Windows, install Graphviz using the stable version for Windows and add the bin directory to the %PATH%.

Usage

Help

The power_stencil CLI provides a contextual help. You can begin with:

$ power_stencil --help

That will display a basic help:

PowerStencil is the Swiss-army knife templating workflow for developers and ops.
-- Options ---------------------------------------------------------------------
    -v, --verbose  Displays extra runtime information.
    -h, --help  Displays this help.
    --program-version, -V, --version  Displays program version.
    --simulate  Will not perform actual actions
    --debug  Debug mode
    --debug-on-err, --debug-on-stderr  Sends debugging to SDTERR
    --log-level  Defines the level of logging (0 to 5)
    --log-file  Specifies a file to log into
    --truncate-log-file  Truncates the log file (appends by default)
    --project-path  Specifies a startup path to use instead of '.'
    --auto  Bypasses command-line confirmations to the user

--------------------------------------------------------------------------------
 Following subcommands exist too:
 For more information you can always issue sub_command_name --help...
--------------------------------------------------------------------------------
 * init: Initializes a PowerStencil repository ...
 * info: Generic information about the repository ...
 * plugin: Manipulates plugins ...
 * get: Query entities from repository ...
 * shell: Opens a shell to interact with entities ...
 * adm: Administrative commands ...
 * check: Check repository entities consistency ...
 * create: Creates entities in the repository ...
 * edit: Edit entities from repository ...
 * delete: Delete entities from repository ...
 * describe: Detailed information about entity types ...
 * build: Builds entities ...

The program uses the standard paradigm of sub-commands (à-la-git), and each can have its own options that you can check using:

$ power_stencil <subcommand> --help

Plugins may bring extra subcommands and/or options, so depending on the project you are working in, the output of --help may differ...

Getting started

See the following topics as a kind of tutorial on PowerStencil. You should read them in order, as each of them assumes you already read the previous one.

Creating a PowerStencil project

To create a new project, use the init sub-command. It works as you would expect. If you run it in an existing directory it will create the project here but you can specify the directory where you want the project to be created (the path will be created provided you have enough rights on the filesystem):

$ power_stencil init /where/you/want/your/project

Once the project created, if you are anywhere within the project tree, you don't need to specify the project path anymore (if needed, any power_stencil sub-command supports a --project-path option). If you're used to the Git command line, it works the same way, and you should not feel in uncharted territory...

:information_source: The rest of this documentation will assume you are at the root of this created project.

:information_source: If you have git installed on your system, **the repository of the newly created project has been automatically turned into a git repository. And any action done through the power_stencil command-line will be automatically tracked by git. Only things you will do outside of the power_stencil command-line (adding or modifying templates, creating or modifying entity types... any manual action) will require a user action to make git take it in account. You can completely de-activate this behaviour if you want to fully manage things by yourself by adding :no-git: true in the .ps_project/versioned-config.yaml, yet there is no good reason for that... Unless you know what you are doing, you should keep the default settings. See Git integration for further information.

PowerStencil project structure overview

The structure of a brand new PowerStencil project is the following:

├── .gitignore
└── .ps_project
    ├── entities
    │   └── README.md
    ├── entity_definitions
    │   └── README.md
    ├── personal-config.yaml
    ├── plugins
    ├── templates-templates
    │   └── README.md
    ├── user_entities
    │   └── README.md
    └── versioned-config.yaml

Other directories will appear once you start working with the project, perform some builds or when you will define templates, but this is the initial state.

Not all the files in this directory should be versioned, and this the reason why there is a preconfigured .gitignore at the root of the project. It is prefilled so that unless you want to specifically avoid a file to be versioned, you don't need to worry regarding PowerStencil project files. By default everything that must be versioned actually is and everything that shouldn't isn't.

The project has two distinct config files.

  • Keep the official project config in the .ps_project/versioned-config.yaml which, as its name suggest, will be git versioned.
  • But any developer who wants to temporarily override some configuration should do it in the .ps_project/personal-config.yaml, which is not to be versioned. The content defined in the latter will override the content of the versioned one.

This is a clean way for anyone who wants to test something to safely do it locally without the risk to pollute the central repo.

There is the same mechanism between .ps_project/entities (where the project data is stored and which is versioned) and .ps_project/user_entities (where local data is stored and not versioned), but as opposed to the two aforementioned config files, you should not edit anything directly there, but instead use the CLI or the shell. More information on how to manage entities here.

The plugins directory can optionally contain project specific plugins. Plugins are a great way to extend PowerStencil for the needs of a project or organization.

Command line auto-completion

PowerStencil comes with a powerful command-line auto-completion feature for zsh. (It is not yet available for bash).

You can install it by running:

$ power_stencil adm --zsh-completion

It provides a very useful auto-completion for sub-commands, options as well as options parameters...

PowerStencil uses a two-stages auto-completion mechanism:

  • Project independent completion is generated in ~/.zsh/completion/_power_stencil, whereas completion related to new commands or options, coming from plugins used in a project, is generated (and versioned) within the project itself, in .ps_project/.zsh_project_completion.

  • Meaning that if you want to benefit from the "full" completion from within a project (ie including extra command line options brought by the project itself), you may have to re-run power_stencil add --zsh-completion from within any project where you use extra plugins, in order to (re)generate the .ps_project/.zsh_project_completion file. If you don't, it is not a big deal, but then you will only have access to the "generic" completion, ie completion not taking in account any of the extra commands or options you are using in this project brought by plugins you may use...

Entities

The core of the system is the entity, so you should start by having a look at what entities are, how they are easily managed using the PowerStencil CLI and shell.

Templates

Then it is important to understand how to use entities within templates. Templates are the way to generate whatever you need (doc, code, descriptors, sites, whatever...) from the data that the entities represent.

Builds

The mechanism that combines entities and templates is called a build. A build is always attached to an entity (you build something). The result of a build is a set of files. Optionally an action can be triggered after the files are generated (could be as simple as calling a script that has been generated).

Plugins

PowerStencil could stop there, and you would be able to do whatever you want, but there is a whole world beyond. Plugins provide a way to completely extend PowerStencil and ease cross-projects share and re-usability, further control relations between entities, implement complex post-build actions, add CLI sub-commands and options. Plugins can be local to the project or coming in the form of standard Ruby Gems ! The icing on the cake...

Git integration

Git is closely integrated at the heart of PowerStencil. See how works the Git integration.


And of course, you may want to read the F.A.Q.

Project status

Check PowerStencil status in the F.A.Q..

Contributing

Bug reports and pull requests are welcome on Gitlab at https://gitlab.com/tools4devops/power_stencil. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the PowerStencil project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.