Embedding Robot Framework tests and keywords into Sphinx documentation

October 22, 2012

Robot Framework ships with decent tools for generating reference documentation out of your robot keywords and test data (see libdoc and testdoc). Yet, when Timo Stollenwerk presented Robot Framework as part of his is talk about TDD at PloneConf 2012, the first question from the audience was, would you be able to include tests as examples into narrative documentation of your package.

I’m not sure, how much effort would it require to make Robot Framework support testable documentation (similarly to doctest-module), or would it even make any sense…

The other way around, however, is easy.

Introducing sphinxcontrib-robotdoc

Sphinx is the current state-of-art document generation tool of the Python community. Sphinx is based on Docutils, which makes it very easy to extend its reStructuredText-markup with custom directives.

There’s also a real killer app for it: ReadTheDocs.

So, in the spirit autodoc extension for Sphinx, I wanted to use my sprint time at the PloneConf for starting up a new Sphinx extension for embedding Robot Framework tests and user keywords into narrative package documentation.

This work is now available as: sphinxcontrib-robotdoc.

And robot_-directives

sphinxcontrib-robotdoc introduces two new custom Docutils-directives to be used in Sphinx documentation:

robot_tests and
robot_keywords.

Both directives accept 1) optional regular expression filter and 2) mandatory source-option with a relative path to locate your Robot Framework test data or resource file. In addition, robot_tests-directive accepts also an 3) optional comma separated list of tags to select embedded tests from the parsed test data.

For example:

Embed all tests from a test suite:

.. robot_tests::
    :source: ../src/my_package/tests/acceptance/my_suite.txt

Embed all tests starting with Log in from a test suite:

.. robot_tests:: Log in.*
   :source: ../src/my_package/tests/acceptance/my_suite.txt

Embed all tests tagged with login or logout from a test suite:

.. robot_tests::
   :source: ../src/my_package/tests/acceptance/my_suite.txt
   :tags: login, logout

Embed all user keywords from a test or a resource file:

.. robot_keywords::
   :source: ../src/my_package/tests/acceptance/my_suite.txt

Embed all user keywords starting with Log in from a test or a resource file:

.. robot_keywords:: Log in.*
   :source: ../src/my_package/tests/acceptance/my_suite.txt

When test cases or user keywords contain documentation, it gets parsed with something called nested Docutils parser. This supports also links between keywords and links from narrative to keywords as long as both the link and its target are embedded onto the same Sphinx page.

Enabling for ReadTheDocs

If you are new to ReadTheDocs, you should start with their Getting Started -guide.

ReadTheDocs does support custom Sphinx-plugins (the ones that are not distributed with Sphinx’s main distribution), but there are a few things to know about it:

As usual, you must add the plugin into the extensions list of your Sphinx configuration (usually conf.py). Also, remember to convert dashes in package names to underscores:
```
extensions = ['sphinxcontrib_robotdoc']
```
The required plugin must be published (probably at PyPi) like sphinxcontrib-robotdoc.

You must edit your ReadTheDocs-project through their dashboard to Use virtualenv:

Use virtualenv
[x]  Install your project inside a virtualenv using setup.py install

Your package must include a pip requirements file requiring the Sphinx plugin (and the possibly required minimum version) you are using:
```
sphinxcontrib-robotdoc>=0.3.4
```
The requirements file itself could be made specific for ReadTheDocs by placing it under a package subdirectory, e.g. ./docs/requirements.txt.
Finally, your must edit your ReadTheDocs-project through their dasboard to find your requirement file:
```
Requirements file:
docs/requirements.txt
```

Done. Now, the next ReadTheDocs-build for your documentation should be able to use your custom Sphinx-plugin, e.g. sphinxcontrib-robotdoc.

With a full example

At the PloneConf, I had a presentation with Jukka Ojaniemi on doing AMQP based system integrations for Plone. For the presentation, I wrote a minimal publish-subscribe -example for Plone containing also a pair of acceptance tests written with Robot Framework.

Here goes my

and, finally, the results at ReadTheDocs.

And then what?

So, if you do acceptance driven development, shouldn’t your acceptance criteria be good enough to be embedded as examples of your product’s usage into its narrative documentation?

Actually, I don’t want to argue more on that… I’ll describe a real use case instead:

plone.act is the new acceptance test library for Plone and Plone add-on-developers. It is implemented as an importable resource of Robot Framework user keywords built on top of Robot Framework’s built-in keywords and Selenium2Library-keywords. Of course, it’s still far from complete.

For plone.act, we do need to write a narrative tutorial-like documentation, including descriptions of the available keywords and examples of their use in custom test cases. The best way to do this and keep it in sync with the current implementation, I believe, would be to embed the actual keywords and tests cases into the documentation.

And, I hope, we can do that with sphinxcontrib-robotdoc and enhance it a lot during the process.