PowerStencil
Power Stencil
is the Swiss-army knife templating workflow for developers and ops.
See official website.
Overview
PowerStencil
proposes a radical approach on how to manage your shared configuration.
Configuration is one of the most complex things to maintain, and anyone who participated in a large scale piece of software, be it for code, tools configuration or even documentation, knows how complex it is to maintain, keep consistent and avoid duplication on the long run, across the various projects that may share this configuration.
From a very high level point of view, the workfow PowerStencil
proposes looks like:
PowerStencil
provides development and operations friendly workflows to fully manage the maintenance of a complex shared config in order to generate anything you may want like documentation, static sites, code, configuration for multiple tools..., while avoiding duplication, and manage it like code !
It is a common pattern to introduce a database to manage shared configuration for CI/CD processes, and it is a very bad idea (see F.A.Q.) because it implies extra tooling, and specific developments that de-facto become single point of failure at the heart of your development process. Why would you do that when something as versatile and ubiquitous as Git exists that will integrate smoothly within your existing development and operations workflows, builds, CI/CD processes, while benefiting from all the goodness provided by Git, like branches, decentralization, versioning, tags, code reviews, traceability, simplified fallbacks etc... ?
A PowerStencil
project is composed of a data repository and a pretty standard templating flow to generate whatever is needed by your project or organization and with PowerStencil
everything is maintained in a git repository: data, templates, specific code without sacrificing on data integrity and relational features. No more config loss, no more change applied without a full lineage.
:information_source: With PowerStencil
data can either be managed as documents to edit (Yaml) or fully as code within a powerful shell !!. Choose the method that fits the most to what you are doing.
The PowerStencil
gem provides the power_stencil
executable which is a pluggable CLI to:
- Manage your project relational-integrity-checked data in a versioned, git diff friendly yet flexible format.
- Visualize your data relations with graphviz. Customize generated graphs easily.
- Easily manage templates to generate whatever your project/organization requires.
- Define builds that you can easily embed in your CI/CD process or use them standalone.
- Propose mechanism to temporarily override some data to test changes impact without polluting the official repository data.
- Ensure re-usability with a versatile extension mechanism, including plugins.
Pfff, yet another templating engine...?
Actually PowerStencil
is not a templating engine. It uses already existing and proven templating engines.
PowerStenil
currently embeds two templating engines:
- ERB is the default. This is a pretty standard and very well known engine in the Ruby world and especially for people who already used the Rails framework. This is a generic templating engine which can handle any use-case. Each template will be processed by default using the ERB engine.
- When it comes to generate mark-up documents like XML or HTML, many people prefer to use Haml, as it designed towards generating these kinds of documents and tends to end-up with much clearer templates. Any template whose names ends by
.haml
will be processed by the Haml engine by default instead of ERB.
So PowerStencil
is definitely not a templating engine, it is actually much more than that. It is a framework to manage your templates, design the relational data you can use in your templates, manage the versioning, define the way you build things etc... Read this documentation (and especially the getting started section which is a tutorial in itself).
Installation
You need a working install of the Ruby language (>= 2.3). You should have bundler
installed (gem install bundler
). This is not added as an automatic dependency as updating bundler
may not be what you want. Yet any working install of Ruby should consider having bundler
installed.
:hand: It is strongly advised to have git
installed on your system, as PowerStencil
is integrated with git
to provide a better user experience.
Then you can install the PowerStencil
gem by issuing the usual:
$ gem install power_stencil
If you want to create Graphviz graphs, you probably may want to install it for your system. If you are using an apt-based system like Ubuntu or Debian it may probably be as simple as:
$ sudo apt install graphviz
If you use Windows, install Graphviz using the stable version for Windows and add the bin
directory to the %PATH%
.
Usage
Help
The power_stencil
CLI provides a contextual help. You can begin with:
$ power_stencil --help
That will display a basic help:
PowerStencil is the Swiss-army knife templating workflow for developers and ops.
-- Options ---------------------------------------------------------------------
-v, --verbose Displays extra runtime information.
-h, --help Displays this help.
--program-version, -V, --version Displays program version.
--simulate Will not perform actual actions
--debug Debug mode
--debug-on-err, --debug-on-stderr Sends debugging to SDTERR
--log-level Defines the level of logging (0 to 5)
--log-file Specifies a file to log into
--truncate-log-file Truncates the log file (appends by default)
--project-path Specifies a startup path to use instead of '.'
--auto Bypasses command-line confirmations to the user
--------------------------------------------------------------------------------
Following subcommands exist too:
For more information you can always issue sub_command_name --help...
--------------------------------------------------------------------------------
* init: Initializes a PowerStencil repository ...
* info: Generic information about the repository ...
* plugin: Manipulates plugins ...
* get: Query entities from repository ...
* shell: Opens a shell to interact with entities ...
* adm: Administrative commands ...
* check: Check repository entities consistency ...
* create: Creates entities in the repository ...
* edit: Edit entities from repository ...
* delete: Delete entities from repository ...
* describe: Detailed information about entity types ...
* build: Builds entities ...
The program uses the standard paradigm of sub-commands (à-la-git), and each can have its own options that you can check using:
$ power_stencil <subcommand> --help
Plugins may bring extra subcommands and/or options, so depending on the project you are working in, the output of --help
may differ...
Getting started
See the following topics as a kind of tutorial on PowerStencil
. You should read them in order, as each of them assumes you already read the previous one.
Creating a PowerStencil
project
To create a new project, use the init
sub-command. It works as you would expect. If you run it in an existing directory it will create the project here but you can specify the directory where you want the project to be created (the path will be created provided you have enough rights on the filesystem):
$ power_stencil init /where/you/want/your/project
Once the project created, if you are anywhere within the project tree, you don't need to specify the project path anymore (if needed, any power_stencil
sub-command supports a --project-path
option). If you're used to the Git command line, it works the same way, and you should not feel in uncharted territory...
:information_source: The rest of this documentation will assume you are at the root of this created project.
:information_source: If you have git
installed on your system, **the repository of the newly created project has been automatically turned into a git repository. And any action done through the power_stencil
command-line will be automatically tracked by git
. Only things you will do outside of the power_stencil
command-line (adding or modifying templates, creating or modifying entity types... any manual action) will require a user action to make git
take it in account. You can completely de-activate this behaviour if you want to fully manage things by yourself by adding :no-git: true
in the .ps_project/versioned-config.yaml
, yet there is no good reason for that... Unless you know what you are doing, you should keep the default settings. See Git integration for further information.
PowerStencil
project structure overview
The structure of a brand new PowerStencil
project is the following:
├── .gitignore
└── .ps_project
├── entities
│ └── README.md
├── entity_definitions
│ └── README.md
├── personal-config.yaml
├── plugins
├── templates-templates
│ └── README.md
├── user_entities
│ └── README.md
└── versioned-config.yaml
Other directories will appear once you start working with the project, perform some builds or when you will define templates, but this is the initial state.
Not all the files in this directory should be versioned, and this the reason why there is a preconfigured .gitignore
at the root of the project. It is prefilled so that unless you want to specifically avoid a file to be versioned, you don't need to worry regarding PowerStencil
project files. By default everything that must be versioned actually is and everything that shouldn't isn't.
The project has two distinct config files.
- Keep the official project config in the
.ps_project/versioned-config.yaml
which, as its name suggest, will be git versioned. - But any developer who wants to temporarily override some configuration should do it in the
.ps_project/personal-config.yaml
, which is not to be versioned. The content defined in the latter will override the content of the versioned one.
This is a clean way for anyone who wants to test something to safely do it locally without the risk to pollute the central repo.
There is the same mechanism between .ps_project/entities
(where the project data is stored and which is versioned) and .ps_project/user_entities
(where local data is stored and not versioned), but as opposed to the two aforementioned config files, you should not edit anything directly there, but instead use the CLI or the shell. More information on how to manage entities here.
The plugins
directory can optionally contain project specific plugins. Plugins are a great way to extend PowerStencil
for the needs of a project or organization.
Command line auto-completion
PowerStencil
comes with a powerful command-line auto-completion feature for zsh
. (It is not yet available for bash
).
You can install it by running:
$ power_stencil adm --zsh-completion
It provides a very useful auto-completion for sub-commands, options as well as options parameters...
PowerStencil
uses a two-stages auto-completion mechanism:
Project independent completion is generated in
~/.zsh/completion/_power_stencil
, whereas completion related to new commands or options, coming from plugins used in a project, is generated (and versioned) within the project itself, in.ps_project/.zsh_project_completion
.Meaning that if you want to benefit from the "full" completion from within a project (ie including extra command line options brought by the project itself), you may have to re-run
power_stencil add --zsh-completion
from within any project where you use extra plugins, in order to (re)generate the.ps_project/.zsh_project_completion
file. If you don't, it is not a big deal, but then you will only have access to the "generic" completion, ie completion not taking in account any of the extra commands or options you are using in this project brought by plugins you may use...
Entities
The core of the system is the entity, so you should start by having a look at what entities are, how they are easily managed using the PowerStencil
CLI and shell.
Templates
Then it is important to understand how to use entities within templates. Templates are the way to generate whatever you need (doc, code, descriptors, sites, whatever...) from the data that the entities represent.
Builds
The mechanism that combines entities and templates is called a build. A build is always attached to an entity (you build something). The result of a build is a set of files. Optionally an action can be triggered after the files are generated (could be as simple as calling a script that has been generated).
Plugins
PowerStencil
could stop there, and you would be able to do whatever you want, but there is a whole world beyond. Plugins provide a way to completely extend PowerStencil
and ease cross-projects share and re-usability, further control relations between entities, implement complex post-build actions, add CLI sub-commands and options.
Plugins can be local to the project or coming in the form of standard Ruby Gems ! The icing on the cake...
Git integration
Git is closely integrated at the heart of PowerStencil
. See how works the Git integration.
And of course, you may want to read the F.A.Q.
Project status
Check PowerStencil
status in the F.A.Q..
Contributing
Bug reports and pull requests are welcome on Gitlab at https://gitlab.com/tools4devops/power_stencil. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
License
The gem is available as open source under the terms of the MIT License.
Code of Conduct
Everyone interacting in the PowerStencil project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.