Asko Soukka

Speed up your Plone add-on tests on Travis CI with the Unified Installer

Many thanks for Héctor Verlarde for encouraging us to try out Travis CI for testing our own Plone add-ons. Also, thanks for Godefroid Chapelle for showing me, how to run Selenium tests on a headless server, e.g. on Travis CI.

As you may already know, the main issue in testing Plone add-ons on Travis CI is its strict 15 minute time limit on running your test suite. And as you may also know, 15 minutes is not much time for our dear buildout to gather all the required dependencies of Plone or, and still run our test after the buildout.

As expected, neither did I get far without having issues with the time limit. And for some reason, I couldn’t get the earlier solutions to work for me. Eventually, I found out a new solution, surprisingly, with the help of Plone Unified Installer.

Because Plone Unified Installer comes in a single downloadable package and includes a complete buildout-cache usable also in a test buildout, I realized, that it could speed up my test buildout a lot, and it did. Yet, with Plone 4.3 shipping with Dexterity, I would expect it to speed it up even more.

Update: The method described here is adopted as part of buildout.plonetest, which includes more generic configuration to work with all Plone-versions.

Enough talk. Here’s my setup:


extends =
develop = .
parts = test
versions = versions

recipe = zc.recipe.testrunner
eggs = my_package[test]

Nothing special here. I expect of the tested package to include complete extras_require={'test': ... } with all the required dependencies for testing.

So, on a local machine, python, bin/buildout and bin/test combo should run tests for a freshly cloned package repository just as expected.


extends = buildout.cfg
parts =
eggs-directory = buildout-cache/eggs
download-cache = buildout-cache/downloads

recipe =
url =

recipe = collective.recipe.cmd
on_install = true
cmds = tar jxvf ${download:location}/Plone-4.2.1-UnifiedInstaller/packages/buildout-cache.tar.bz2 1>/dev/null

Here’s the magic for re-using Plone Unified Installer for your test buildout:

  1. At first, download and unpack the installer in [download] part
  2. then extract its buildout-cache in [install] part into the locations defined in [buildout] part.

As you might guessed, after this, buildout needs to download only the extra requirements of the tested package! Long live Plone Unified Installer!


language: python
python: "2.7"
  - mkdir -p buildout-cache/downloads
  - python -c travis.cfg
  - bin/buildout -N -t 3 -c travis.cfg
script: bin/test

Note, how we need to create a buildout-cache-directory for downloads as defined earlier in travis.cfg. The rest should be easy: we just do the bootstrap and run our tests with sane buildout-options, and… that’s all.

.travis.yml for robotsuite

Oh, in the beginning, I mentiond about learning something important from Godefroid. Well, if you have followed me on creating zope.testrunner-compatible Robot Framework -tests with, you only need to add a few extra lines to make your Robot Framework tests runnable on Travis CI:

language: python
python: "2.7"
  - mkdir -p buildout-cache/downloads
  - python -c travis.cfg
  - bin/buildout -N -t 3 -c travis.cfg
  - "export DISPLAY=:99.0"
  - "sh -e /etc/init.d/xvfb start"
script: bin/test

If you think, this is cool, please, give some love for the Travis CI team!