Black&White Project/Compilation Farm/Setup

Organization
The Compilation Farm is installed on the download.kiwix.org server. It is organized as:


 * A buildbot master running as user buildbot on the server.
 * A set of VirtualBox VMs running buildbot slaves.
 * ubuntu32 (192.168.5.50): i686 static
 * ubuntu64 (192.168.5.51): x86_64 static
 * macosx (192.168.5.52): universal (i386+x86_64) dmg with app.
 * win32 (192.168.5.53): i386 windows static
 * win64 (192.168.5.54): x64 windows static
 * fedora (192.168.5.55): i686 sugar static (kiwixdev)
 * arm (no direct IP. ssh on localhost through port 5022 10.0.2.2 192.168.5.56): armel kiwix-serve static
 * debian32 (192.168.5.57): i686 wheezy deb
 * debian64 (192.168.5.58): x86_64 wheezy deb

Usage
A utility script, kbox is present to facilitate start and stop of the VMs.


 * Example connection to ubuntu32 VM:

Master Setup

 * Install required packages
 * Create user buildbot
 * Log-in as user buildbot
 * Edit .bashrc and add at the end
 * Log-out then log-in to enable virtualenvwrapper
 * Create virtualenv
 * Install builbot and create master
 * Deploy configuration
 * Edit the password file in master/secret_pass.py
 * Start the master
 * Create an nginx proxy at /etc/nginx/sites-available/buildbot.kiwix.org

Automatic Startup

 * crontab -e on builbot user

Slave Setup
buildbot slave does not require much: a working python/setuptools environnment.

Linux Slave Setup

 * Create a VirtualBox VM (20GB of storage - dynamic)
 * Install Ubuntu 10.04.4 (Newer versions of Ubuntu uses a recent glibc)
 * Select OpenSSH server during install.
 * Create a regular user (dev)
 * Use whole disk partitioning
 * Install packages:
 * Install python tools

From now, you won't need root privileges anymore.


 * Log in as dev user
 * Edit ~.bashrc and add at the end
 * Log out then log-in again (to activate virtualenvwrapper)
 * Create virtualenv
 * Install buildbot
 * Create the slave
 * Edit files in slave/info/ and put host description and maintainer
 * Start the slave

Automatic Startup

 * crontab -e on dev user

VirtualBox Setup

 * Install the non-free version of Virtual Box:


 * VMs will be stored in /home/buildbot/vms


 * Configure Network


 * Create a HDD Image


 * Create VM (VBoxManage list ostypes for types list)


 * Start VM & Install System using rdesktop

On your computer, connect with:


 * Configure Network (eth1) with a static IP according to mapping above in /etc/network/interfaces


 * Once installed, tweak the VM

From now, connection to the VM will be SSH only.

Post B&W todo

 * Start VM automaticaly/transparently if a compilation is required - otherwise VM is offline.
 * Start chained compilations : only the first one is planned, and all other ones occur one by one afterwards.
 * Bump code with bumpVersion.sh to have a version like svn-[date]
 * Rename bin packages

Tips

 * win32 upload issue: VBoxManage modifyvm win32 --natsettings1 1500,64,64,64,1024