Django with batou: The very basics

In this first part, we will setup our batou environment and have a brief look at batou itself. This way we can start to batou right from the start. In following parts we will

• deploy a basic Django application,

• provide a configuration for nginx,

• let the the Django application  finally run behind an nginx proxy.

As a later follow up, we plan to have a deeper look on how to use batou to prepare a PostgreSQL database for the Django application, and how to handle maintenance work. Feel free to tell us your thoughts in the comments.

Why batou

At Flying Circus we are using batou practically everywhere. We need a smooth deployment. Compared to other deployment tools batou and its component model is offering a way to ensure we only touch (e.g. restart) parts of the application that really need to be restarted. So if a currently running service is untouched, it will not be restarted and downtimes will be minimized. This is done by splitting up services into its logical components. Of course these can be distributed to a number of servers. All you need is SSH-access to the target server and having Python 2.7 installed. On the top there are two major principles, you might know from other tools:

• Idempotence – with every run of the software the output keeps the same, and

• convergence – only change/update things needs to be done until no action is needed at all as the system is in defined state.

And in fact: We are using it to manage deployments involving 20+ servers with a large number of components like Nginx, HAProxy, Plone, Mailman, MySQL, PostgreSQL, ejabberd etc. A good reason to use it for your deployments, too.

Learning to fly – the basics

We will track everything inside a git repository. Of course, you can skip this step, but we recommend to make usage of a  version control system so your deployments can be reproducible. If you don't like git, you might want to use Mercurial. Start with adding a new folder and a fresh git repository:

 $ mkdir batou_django
 $ cd batou_django
 $ git init ./

We will do all our work during this part on the master branch. You might consider using special branches for current production and maybe staging code to ensure only well tested and prepared code is shipped to production. Also tagging commits on production branch marking the actual deployments is useful in some scenarios. As we will work with Python we will add a .gitignore at the very beginning of the project to discard Python byte-code files. Create a .gitignore with this content:

*.pyc
*.swp

as well as a short README:

# Django-Deployment with batou
A little example deployment for Django with batou

This is your first commit so:

$ git add README .gitignore
$ git commit -m "Init"

Now we need to download and bootstrap batou

$ curl https://bitbucket.org/flyingcircus/batou/raw/tip/src/batou/bootstrap-template -o batou
$ chmod +x batou

Calling batou here will initialize the download with latest release of batou.

$ ./batou

You should see something like this:

$ ./batou 
Preparing virtualenv in .batou ...
Pre-installing batou - this can take a while...
usage: batou [-h] [-d] [-F] {deploy,remote,secrets,init,update} ...
batou: error: too few arguments

Starting from this point, batou will use the same version until you explicit update it. This is to ensure reliable deployments. As batou is adding some local folders, we will need to extend .gitignore a little. Add

.batou-lock
.batou

and commit your changes:

$ git add batou .gitignore
$ git commit -m "Add batou to deployment"

At this point we do have the tool for deployment, but no content to deploy so far.

The first component

Let's add a dummy component to see how it works. We will choose a simple file that will be deployed to your own home directory. Batou is splitting up tasks (more general all things that needs to be deployed) into components. Each component should do one thing and should be atomar in what it is doing. Each component might have sub-components so you can split it up by task. If you like to deploy a web application, as we will do with Django later, you might get components for

• Nginx or Apache,
• Supervisor,
• a component managing the database, and of course
• some component for the web application it self.

You don't need to have a  component for each and every file that needs to get deployed. To give you an impression on how this looks like, we create a new component to deploy a file. All available components are managed within a folder called components and having their own subfolder:

$ mkdir -p components/HelloWorldFile
$ touch components/HelloWorldFile/component.py

Now edit your new file and add this content:

from batou.component import Component
from batou.lib.file import File

class HelloWorldFile(Component):

    def configure(self):
        self += File('hello.txt', content='Hello world!')

It's best practice to have the name of folders and class name in sync. This makes it easier to find the correct component when managing a bigger project. Let's have a look onto this component in detail. The three most important methods of a component are:

• configure(): defines, or refers to, other components and tasks that need to be done to get the component up and running
• verify(): Is used to check, whether an update of the component necessary at the remote system, or if everything is in the defined target state
• update(): Is called only if verify() detected that “something” needs to be done.

The little example is only having a configure()-method. All methods are optional. We don't need to define an extra verify() or update(). Why is that? All work is delegated to a sub-component:

self += File(..)

So File() is a component too, implementing configure(), verify() and update(), to ensure for example content of the file is correct, or the mode is set correctly. And many things more. The operator += is adding the component as sub-component to our HelloWorldFile-component. At this point it becomes part of your component and you do not need to worry about it anymore. Commit your changes to git:

$ git add components/HelloWorldFile/component.py
$ git commit -m "Adding new component: HelloWorldFile"

The first environment

As batou is able to manage different environments to e.g. use the same code on testing as well as on production side, we need to setup those environments. Environment-files are stored inside a folder called environments. They contain information for the specifics of the environment, like

• how to connect to hosts,
• included components, and
• target hosts (servers).

It will also include configuration for the components as e.g. hostnames or usernames. For securely storing passwords, ssl certificates, and other sensitive information there is an additional construct, “secrets”. We will have a look at this in a later part. Create your new environment:

$ mkdir -p environments
$ touch environments/dev.cfg

Let's get a little deeper into the dev.cfg:

[environment]
connect_method = local

Each file is having a general section called [environment]. At this section you can configure general things as for example how to connect to the hosts. In this example it is set to local (we will change that later). A local deployment will install components locally, in the current directory. And you do not need to commit your code or complete any other challenges. The second important part of the configuration file is the [hosts] section. It includes which component shall be deployed on which host. We only need a very simple setup here:

[hosts]
localhost = helloworldfile

The components are written here in all small letters – take care when adding your own components. The complete dev.cfg now looks like this:

[environment]
connect_method = local

[hosts]
localhost = helloworldfile

Commit your changes!

$ git commit -m "Adding dev-environment" environments/dev.cfg

Now you are ready for doing your first deployment.

Deploy!

To do the deployment, run

$ ./batou deploy dev

This will calculate the tree of components needed for that deployment and checks whether all dependencies for the defined components are satisfied: are all required components present, and are all provided components used. This does not include software that needs to be installed on target hosts, unless you model that. Once the model is computed, batou will connect to the target hosts. In our case it does not really talk over a network but deploys in-place. Once the deployment has finished, you can have a look into the work sub-directory. There you will find the hello.txt too. At the very beginning of the blog post we talked about idempotence and convergence: Re-run the deployment! Batou should recognize that nothing is to do. This will change, if you delete the hello.txt and rerun batou deployment. Do so! In the next step on our way to deploy a Django project we will add a component for Django as well as defining a Vagrant environment for easy testing. Stay tuned for part 2! If you would like to stay up to date on what's happening at Flying Circus subscribe to our newsletter.