Deploying debci to your own infrastructure
This document describes how to install debci for production usage. If you are trying to setup a development environment, see How to setup a development environment instead.
Architecture
The following picture represents the debci architecture:
-
The whole system communicates using an AMQP server, in this case
rabbitmq
, as a message queue. -
Test jobs are injected into a queue by the
debci-enqueue
command line tool, or by the web UI/API. -
test jobs are picked up from the test queue by
debci-worker
daemons. They run the tests, and push the results back into a different queue. -
debci-collector
receives the test results, processes them, feeds the database with the results, and produces the HTML pages, atom feeds etc that in the web interface. -
Each component (
debci-collector
,debci-worker
,rabbitmq
, and the command line tools) can live in different hosts if necessary, as long as all of the debci components can connect to therabbitmq
server. -
You can have as many
debci-worker
nodes in the system as you want, but there must be only onedebci-collector
.
Deployment procedure
Install the debci-collector:
$ sudo apt install debci-collector
debci-collector
recommends rabbitmq-server
, and debci-collector will use a locally-installed rabbitmq server by default. If you want to use a remote rabbitmq, you need to add a configuration file with the .conf
extension to /etc/debci/conf.d
with something like this:
debci_amqp_server=amqp://MYRABBITMQSERVER
Note that if MYRABBITMQSERVER
is network accessible, it has to have the proper ACLs configured. Check the rabbitmq documentation for details.
On each worker node, install apt-cacher-ng
to cache package downloads, and debci-worker
itself:
$ sudo apt install apt-cacher-ng debci-worker
As with debci-collector
, debci-worker
will connect to a local rabbitmq by default. To make it connect to a remote rabbitmq-server you can do the same as above.
Note that when first installed, debci-worker
will first build a testbed (a container or a virtual machine image, depending on the selected backend), and only after that is finished the worker will be able to start processing test jobs.
Submitting test jobs
On any host that can connect to rabbitmq-server
, first install debci
:
$ sudo apt install debci
As usual, you will prompted for the address of the AMQP server. If the rabbitmq-server
is on the same host, just leave it blank.
Say you want to run the tests for the ruby-defauts
package. It is as easy as
$ debci enqueue ruby-defaults
Scheduling job submission
By default, debci
will not submit any jobs. You need to decide how, and how often, you want to submit jobs. A simple way of doing that is using cron
.
Example: schedule tests for a set of packages once a day
# /etc/cron.d/debci
0 7 * * * debci debci enqueue rake rubygems-integration ruby-defaults
You can also automate calls to debci enqueue
in any other way you want.
Multiple worker processes per node
If you have worker nodes that have lots of CPUs and a large amount of RAM available you can run multiple worker daemons at once.
TODO
Setting up the LXC backend
TODO