Asko Soukka

Running Robot Framework tests with Google App Engine SDK

For a long time I thought that Selenium was hard to set up and tests for it were hard to write. Well, I couldn’t have been more wrong. With Robot Framework and robotframework-selenium2library even a BDD-style acceptance test case using Selenium can be as simple as:

*** Test Cases ***

Can upload file as an admin
    Given I'm logged in as an admin
     When I go to address '/upload'
      And I select the file 'hello.html' into the field 'file'
      And I click the button 'submit'
     Then the page should contain link 'hello.html'

Let’s see, how that’s possible for a GAE app…

Bootstrap your development environment

I happen to like buildout for keeping my development environments for different projects clean, separated and reproduceable. Today it helps us to install and create scripts for running Robot Framework tests for our GAE app with all the required dependencies. Buildout installs everything under your project directory, so the process keeps your system Python clean and healthy.

  1. create a new project directory called myapp and go there

    $ mkdir myapp
$ cd myapp

  2. create a new directory called src

    $ mkdir src

  3. copy the codebase of your GAE app into that directory

    $ cp -R /path/to/myapp src

  4. create a file called setup.py to allow Python setuptools (or distribute) used by the buildout to find your code

    from setuptools import setup, find_packages

setup(
    name='myapp',
    packages=find_packages('src', exclude=['ez_setup']),
    package_dir={'': 'src'},
    include_package_data=True,
    zip_safe=False,
    install_requires=['setuptools'],
)

  5. create a file called MANIFEST.in to accompany your setup.py

    recursive-include src *.yaml *.py *.txt *.html *.css

  6. create a file called buildout.cfg to configure your buildout to download the SDK and create a Robot Framework test script with both SDK and your code registered into its run-time sys.path

    [buildout]
parts =
    app
    test
unzip = true
develop = .

[app]
recipe = rod.recipe.appengine
zip-packages = False
exclude = tests
url = http://googleappengine.googlecode.com/files/google_appengine_1.6.6.zip
eggs = myapp

[test]
recipe = zc.recipe.egg
eggs =
    robotframework
    robotentrypoints
    robotframework-selenium2library
    myapp
extra-paths = ${buildout:parts-directory}/google_appengine

  7. download the bootstrap.py for bootstrapping your buildout

    $ wget http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py

  8. finally, run the bootstrap and buildout

    $ python boostrap.py
$ bin/buildout

If everything goes as planned, your project directory should now include a script bin/pybot for running Robot Framework with all the requirements in sys.path.

P.S. The above buildout provides also scripts bin/app and bin/appcfg, which allow you to start the development server or upload your app without system wide GAE SDK install. See rod.recipe.appengine for details.

Setup your test fixture

For acceptance tests, it would be nice to have your GAE app running as completely as possible, but with a clean database for every test. That requires every test to have a setup method for launching your app onto background before the test and a teardown method to shut it down after the test.

Unfortunately, GAE SDK won’t ship with much support for automated acceptance test. There exists a project called gaedriver for running SDK’s development server as a Python subprocess while running the tests. Yet, because I was looking for something even simpler, eventually I ended up with something my own.

  1. create a directory src/tests to contain your app’s custom Robot Framework library and the acceptance tests

    $ mkdir src/tests

  2. create a file called src/tests/__init__.txt to define setup and teardown keywords for every Robot Framework test found under the same directory (or under its sub-directories)

    *** Settings ***

Resource  common.txt

Test Setup  Start app and open browser
Test Teardown  Stop app and close all browsers

  3. create an another file called src/tests/common.txt to define the setup and teardown methods in detail and also the libraries including the used keywords

    *** Settings ***

Library  Selenium2Library  timeout=5  implicit_wait=5
Library  tests.AppEngineLibrary

*** Keywords ***

Start app and open browser
    Start app
    Open browser  http://localhost:8080/

Stop app and close all browsers
    Close all browsers
    Stop app

  4. finally, create a file called src/tests/__init__.py to define our custom AppEngineLibrary to provide keywords for starting and stopping our app for every test case

    # -*- coding: utf-8 -*-
"""Google App Engine library for Robot Framework"""

import signal
import os.path


class AppEngineLibrary(object):
    """Provides keywords for starting and shutting down GAE dev_appserver"""

    def __init__(self):

        # initialize variables
        self.robot_pid = None
        self.gae_pid = None

    def start_application(self):

        # save the process id to distinguish it from its children
        self.robot_pid = os.getpid()

        # fork the process for its child to launch the appserver
        self.gae_pid = os.fork()

        if os.getpid() != self.robot_pid:
            # mute logging so that app server doesn't mess with the robot
            import logging
            logging.disable(logging.CRITICAL)

            # import appserver and fix sys.path for GAE modules only after
            # the process has been forked (GAE may monkeypatch things...)
            import dev_appserver
            dev_appserver.fix_sys_path()

            # launch the appserver
            from google.appengine.tools.dev_appserver_main import main
            app_dir = os.path.join(os.path.dirname(__file__), os.path.pardir)
            main([
                os.getcwd(),
                app_dir,
                '--skip_sdk_update_check',
                '--clear_datastore',
            ])

            # make immediate suicide as soon as the appserver has shut down
            os.kill(os.getpid(), signal.SIGKILL)

    def stop_application(self):

        # tell the appserver to terminate
        os.kill(self.gae_pid, signal.SIGTERM)

    start_app = start_application  # alias
    stop_app = stop_application  # alias

Now you should have everything ready and in place for writing acceptance tests for you app.

Write and run your first test case

Create a complete test case file called src/tests/test_upload.txt, including a BDD-style test case and re-usable helper keywords to translate BDD-clauses into more generic Robot Framework tests using the keywords defined in Selenium2Libary

*** Settings ***

Resource  common.txt

*** Variables ***

${ADMIN_EMAIL}  admin@example.com

*** Test Cases ***

Can upload file as an admin
    Given I'm logged in as an admin
     When I go to address '/upload'
      And I select the file 'hello.html' into the field 'file'
      And I click the button 'submit'
     Then the page should contain link 'hello.html'

*** Keywords ***

I'm logged in as an admin
    Go to  http://localhost:8080/upload
    Click link  Login
    Input text  email  ${ADMIN_EMAIL}
    Click button  Login

I go to address '${path}'
    Go to  http://localhost:8080${path}

I select the file '${filename}' into the field '${locator}'
    Choose file  ${locator}  ${CURDIR}/${filename}

I click the button '${locator}'
    Click button  ${locator}

The page should contain link '${locator}'
    Page should contain link  ${locator}

Now this test could be executed with Robot Framework and Selenium just by using the command

$ bin/pybot src/tests

Of course, you should look into Robot Framework User Guide.