batou - recent improvements and roadmap update

At the recent Plone Alpine City Sprint we took the time to improve batou's documentation a lot. You can read it on https://batou.readthedocs.org. The basic concepts (modelling your application and fitting it into an environment) should be easier to understand and we have made a full guide through all the important features if you want to get started. Also, we are covering API reference and CLI commands almost completely now.

In addition to the documentation there are new features that have been long standing:

Prediction and Quick Check Deployments

You can perform prediction deployments: batou will validate the environment and perform the verification steps of all components in your target environment. It will then predict what changes it would make without actually touching anything. (batou 1.3, will soon be released) If you changed the model and want to get a quick check whether the configuration of a certain environment is consistent, you can perform a "quick check": batou connects to only a single target machine, runs the configuration phase and reports back any errors. This is the even faster version than "dry run" deployments. (batou 1.2)

Supervisor: updated and refactored

The supervisor component is now by default compatible with standard Linux environments and has no additional component dependencies that annoyed people when getting started. This has a few drawbacks for production scenarios as you have to make an explicit configuration for production purposes, but we think it's worthwhile to get you started with a not-too-bad configuration.

Output improvements and error reporting

We improved the output format so it hopefully is now more readable and easier to understand. Specifically error output is now formatted with semantic annotations that should help you understand issues in your deployment faster.

Partial deployments (or "skipping")

The "local" deployment command has been gone for a while which was annoying to people: the new way to deploy locally meant that you had to install all components of an environment even if you were never touching them and just needed them for a consistent configuration. Now we came up with a more flexible way: skipping. Simply prefix hostnames or component names in an environment with "!" (as in the "not" operator) in your environment configuration. batou will then skip those during a deployment run. They still contribute to having a consistent configuration in that environment. And even better: this works with remote deployments, now, too!

Improved secrets management

When managing secrets for many users it will be helpful to use the new commands that allow surveying users' access to environments and adding/removing users for multiple secrets files automatically.

Managing your batou project with git

This has been implemented since last September or so, but as this has been a request we got over and over, I'll repeat it: batou repositories can finally be managed with git (or simply rsync).

Where are we headed?

We also have a bucket list of things we'd love to get implemented in the near future:

Aramaki - batou's Continuous Deployment Server

We have some installations around that use Jenkins to automatically start batou deployments, keep a central log, and allow you to manually promote deployments from various stages. Jenkins is rather hard to manage and we'd love to refactor the little code and features that we actually use into its own small service. We already have a name: Aramiki (who, in Ghost in the Shell, is the chief of Section 9, and thus the supervisor of batou).

More predefined components

Writing batou components is not hard, but we'd like to grow our collection of reusable components. We already have "batou_ext" but it's not documented, yet, and some of the components there need some love. However, this approach will allow us to shrink the batou default library. For example, we think supervisor should no longer be a built-in component as updating batou should not change the version of Supervisor your project uses.

Sharing configuration between environments

Large projects tend to have many environments, sometimes even per developer. This can become hard to update when making structural changes and we'd love to make environments a bit smarter. Also, sometimes if you have multiple hosts with the same configuration an environment becomes bloated. Some macros or templating would be nice here, too.

More documentation

We are happy about the great feedback we got when we improved the documentation. This motivates us to keep moving this forward, too! In case  you are interested in any of those features specifically or have general suggestions - we are open for contributions and we can also work something out to speed up their implementation. We're happy to talk to you - so leave a comment or get in touch!