How To Install Buildbot on Ubuntu 16.04

Buildbot is a continuous integration tool based on Python which automates the build, test and release software cycles. It is built using the Twisted networking engine’s powerful event-driven asynchronous communication that assists Buildbot in the fast deployment of multiple testing builds. Buildbot is very flexible and can be used in projects of any size. In this tutorial, we will show you how to install Buildbot on an Ubuntu 16.04 VPS.

1. Upgrading the System

As usual before beginning to work on your server, make sure all the system packages are up to date:

# apt-get update && apt-get upgrade

2. Installing Buildbot

Installing Buildbot is pretty easy, we will begin by installing pip the Python package manager as it will save us a lot of trouble from finding all the separate Python packages Buildbot needs:

# apt-get install python-pip

Now that we have installed pip we can install Buildbot using pip, execute the following command:

# pip install 'buildbot[bundle]'

The end of the installation should look something like this:

Successfully built zope.interface sqlalchemy Twisted future Tempita MarkupSafe
Installing collected packages: Tempita, decorator, sqlalchemy, sqlparse, six, pbr, sqlalchemy-migrate, txaio, autobahn, MarkupSafe, Jinja2, zope.interface, python-dateutil, constantly, incremental, attrs, Automat, idna, hyperlink, Twisted, PyJWT, future, buildbot-www, buildbot-worker, buildbot-console-view, buildbot-waterfall-view, buildbot-grid-view, buildbot
Successfully installed Automat-0.6.0 Jinja2-2.10 MarkupSafe-1.0 PyJWT-1.6.1 Tempita-0.5.2 Twisted-17.9.0 attrs-17.4.0 autobahn-18.4.1 buildbot-1.1.1 buildbot-console-view-1.1.1 buildbot-grid-view-1.1.1 buildbot-waterfall-view-1.1.1 buildbot-worker-1.1.1 buildbot-www-1.1.1 constantly-15.1.0 decorator-4.3.0 future-0.16.0 hyperlink-18.0.0 idna-2.6 incremental-17.5.0 pbr-4.0.2 python-dateutil-2.7.2 six-1.11.0 sqlalchemy-1.2.7 sqlalchemy-migrate-0.11.0 sqlparse-0.2.4 txaio-2.10.0 zope.interface-4.5.0
You are using pip version 8.1.1, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

If there’s a newer version available from pip, make sure to upgrade to that version using this command:

# pip install --upgrade pip

This should be the output from the upgrade command:

Collecting pip
  Downloading https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl (1.3MB)
    100% |################################| 1.3MB 423kB/s
Installing collected packages: pip
  Found existing installation: pip 8.1.1
    Not uninstalling pip at /usr/lib/python2.7/dist-packages, outside environment /usr
Successfully installed pip-10.0.1

Now check if Buildbot has been correctly installed:

# buildbot --version
Buildbot version: 1.1.1
Twisted version: 17.9.0

Add a system username and group for Buildbot to use:

# addgroup --system buildbot
# adduser buildbot --system --ingroup buildbot --shell /bin/bash

# su buildbot

3. Configuring the Buildbot Master

In this step we will configure the Buildbot master, start by executing the following command from the Buildbot user:

# buildbot create-master ~/master

You should see the following output:

mkdir /home/buildbot/master
creating /home/buildbot/master/master.cfg.sample
creating database (sqlite:///state.sqlite)
buildmaster configured in /home/buildbot/master

Now copy the sample Buildbot configuration to a new file with the following command:

# cp ~/master/master.cfg.sample ~/master/master.cfg

In order for Buildbot’s web interface to be accessible to us, we need to change the BuildbotURL setting in the configuration file, open the file with your favorite editor(we will use nano):

# nano ~/master/master.cfg

c['buildbotURL'] = "http://your_domain_or_ip_address:8010/"

Now at the bottom of the file append the following line to opt-out of sending network usage data to the developers at the end of the file:

c['buildbotNetUsageData'] = None

Save and exit the file and then run the following command to check the configuration:

# buildbot checkconfig ~/master

If everything is ok you should see the following output:

Config file is good!

Now we can start the Buildbot master with the following command:

# buildbot start ~/master

If everything went well you should see the following output:

Following twistd.log until startup finished..
The buildmaster appears to have (re)started correctly.

Navigate to the URL we configured earlier using your browser so we can verify that the web interface is accessible:

http://your_domain_or_ip_address:8010/

4. Configuring a Buildbot Worker

We will install and configure a Buildbot worker on the same server as the master now, type in the following command:

# buildbot-worker create-worker ~/worker localhost rosehosting pass

The command above creates a worker named ‘rosehosting’ with a password ‘pass’ on ‘localhost’ which is our local server, the output should be the following:

mkdir /root/worker
mkdir /root/worker/info
Creating info/admin, you need to edit it appropriately.
Creating info/host, you need to edit it appropriately.
Not creating info/access_uri - add it if you wish
Please edit the files in /root/worker/info appropriately.
worker configured in /root/worker

Open the ‘~/master/master.cfg’ file, find and change ‘example-worker’ in the following lines:

# nano ~/master/master.cf

c['workers'] = [worker.Worker("rosehosting", "pass")]

c['builders'].append(
    util.BuilderConfig(name="runtests",
      workernames=["rosehosting"],
      factory=factory))

Restart the master again:

# buildbot restart ~/master

Now let’s edit the administrator information for the worker:

# nano ~/worker/info/admin

RoseHosting Admin &[email protected]&gt

Do the same for the host information:

# nano ~/worker/info/host

RoseHosting SSD 4 VPS - Buildbot version: 1.1.1 - Twisted version: 17.9.0

5. Restart the Worker

Start the worker using the following command:

# buildbot-worker start ~/worker
Following twistd.log until startup finished..
The buildbot-worker appears to have (re)started correctly.

Finally, navigate to http://your_domain_or_ip_address:8010/ using your browser, from the ‘Build’ menu select ‘Workers’ and then click on the ‘runtests’ builder to force a test build.

Of course you don’t have to install Buildbot on Ubuntu 16.04, if you use one of our managed Ubuntu VPS hosting services, in which case you can simply ask our expert Linux admins to setup Buildbot for you. They are available 24×7 and will take care of your request immediately.

PS. If you liked this post, on how to install Buildbot on Ubuntu 16.04, please share it with your friends on the social networks using the buttons on the left or simply leave a reply below. Thanks.