Embedding Robot Framework tests and keywords into Sphinx documentation
October 22, 2012
Robot Framework ships with decent tools for generating reference documentation out of your robot keywords and test data (see libdoc and testdoc). Yet, when Timo Stollenwerk presented Robot Framework as part of his is talk about TDD at PloneConf 2012, the first question from the audience was, would you be able to include tests as examples into narrative documentation of your package.
I’m not sure, how much effort would it require to make Robot Framework support testable documentation (similarly to doctest-module), or would it even make any sense...
The other way around, however, is easy.
Sphinx is the current state-of-art document generation tool of the Python community. Sphinx is based on Docutils, which makes it very easy to extend its reStructuredText-markup with custom directives.
There’s also a real killer app for it: ReadTheDocs.
So, in the spirit autodoc extension for Sphinx, I wanted to use my sprint time at the PloneConf for starting up a new Sphinx extension for embedding Robot Framework tests and user keywords into narrative package documentation.
This work is now available as: sphinxcontrib-robotdoc.
sphinxcontrib-robotdoc introduces two new custom Docutils-directives
to be used in Sphinx documentation:
Both directives accept 1) optional regular expression filter and 2)
mandatory source-option with a relative path to locate your Robot
Framework test data or resource file. In addition,
robot_tests-directive accepts also an 3) optional comma separated list
of tags to select embedded tests from the parsed test data.
Embed all tests from a test suite:
robot_tests source ../src/my_package/tests/acceptance/my_suite.txt
Embed all tests starting with Log in from a test suite:
robot_testsLog in.* source ../src/my_package/tests/acceptance/my_suite.txt
Embed all tests tagged with login or logout from a test suite:
robot_tests source ../src/my_package/tests/acceptance/my_suite.txt tags login, logout
Embed all user keywords from a test or a resource file:
robot_keywords source ../src/my_package/tests/acceptance/my_suite.txt
Embed all user keywords starting with Log in from a test or a resource file:
robot_keywordsLog in.* source ../src/my_package/tests/acceptance/my_suite.txt
When test cases or user keywords contain documentation, it gets parsed with something called nested Docutils parser. This supports also links between keywords and links from narrative to keywords as long as both the link and its target are embedded onto the same Sphinx page.
Enabling for ReadTheDocs
ReadTheDocs does support custom Sphinx-plugins (the ones that are not distributed with Sphinx’s main distribution), but there are a few things to know about it:
As usual, you must add the plugin into the extensions list of your Sphinx configuration (usually
conf.py). Also, remember to convert dashes in package names to underscores:
extensions = ['sphinxcontrib_robotdoc']
- The required plugin must be published (probably at PyPi) like sphinxcontrib-robotdoc.
You must edit your ReadTheDocs-project through their dashboard to Use virtualenv:
Use virtualenv [x] Install your project inside a virtualenv using setup.py install
Your package must include a pip requirements file requiring the Sphinx plugin (and the possibly required minimum version) you are using:
- The requirements file itself could be made specific for ReadTheDocs
by placing it under a package subdirectory, e.g.
Finally, your must edit your ReadTheDocs-project through their dasboard to find your requirement file:
Requirements file: docs/requirements.txt
Done. Now, the next ReadTheDocs-build for your documentation should be
able to use your custom Sphinx-plugin, e.g.
With a full example
At the PloneConf, I had a presentation with Jukka Ojaniemi on doing AMQP based system integrations for Plone. For the presentation, I wrote a minimal publish-subscribe -example for Plone containing also a pair of acceptance tests written with Robot Framework.
Here goes my
and, finally, the results at ReadTheDocs.
And then what?
So, if you do acceptance driven development, shouldn’t your acceptance criteria be good enough to be embedded as examples of your product’s usage into its narrative documentation?
Actually, I don’t want to argue more on that... I’ll describe a real use case instead:
plone.act is the new acceptance test library for Plone and Plone add-on-developers. It is implemented as an importable resource of Robot Framework user keywords built on top of Robot Framework’s built-in keywords and Selenium2Library-keywords. Of course, it’s still far from complete.
For plone.act, we do need to write a narrative tutorial-like documentation, including descriptions of the available keywords and examples of their use in custom test cases. The best way to do this and keep it in sync with the current implementation, I believe, would be to embed the actual keywords and tests cases into the documentation.
And, I hope, we can do that with sphinxcontrib-robotdoc and enhance it a lot during the process.