Using Camunda with Robocorp Cloud for orchestrating automation

Robocorp Cloud is a commercial automation bot orchestration service by Robocorp. It differentiates itself from the others by promoting license free open source automation solutions based on Robot Framework. I am mostly excited by their contributions to open source Robot Framework ecosystem (e.g. RPA Framework, Xeus-Robot and their training material. Yet, every now and then, I try to update my knowledge on how their main product works.

Right now, most of the Robocorp products and documentation seem to focus on building and running software robots. But I doubt those robots run in isolation. Instead, they probably have their own activites to complete in (more or less) digitized business processes. And that’s why I’ve been working hard to learn BPMN 2.0 and Camunda Platform: I desire to be able to smoothly and transparently integrate automation bots with digitized business processes.

Also Robocorp Cloud composes tasks into higher level thing called workforce “processes”. But those “processes” are just linear series of steps with access to shared process variables (“work item” in RPA terms). For now, it is better to consider Robocorp Cloud “processes” only as a technical configuration concept – not a thing to compare againts BPMN 2.0 modeled business processes.

From a BPMN process…

To showcase Camunda Platform with Robocorp Cloud (especially Robocorp Control Room), we need to have an example process showcasing many of the features available when digitizing process using BPMN. Even better, if the example was easy enough to understand and fun to try out…

Say hello to my automated XKCD Comic Search process:

The process is about searching popular XKCD comic strips with a given search term, fetch the results, and then let a real person do final review for each comic.

The process involves fair amount of BPMN features, including:

• exclusive gateways for conditional process flow
• service tasks for delegating automated tasks for bots
• multi-instance activity for triggering 1:N concurrent subprocesses
• BPMN messaging between the root process and its subprocesses
• business process level exception handling
• alternative process specific end states
• user tasks for manual process activities

BPMN 2.0 can model all this and more. Camunda Platform, on the other hand, is well capable of executing the process. In addition, Camunda provides good enough APIs to fetch and delegate external tasks in the process to available software robots.

…to Robocorp Cloud processes

The example process includes two external tasks to be automated:

• Search for matching comics task in the beginning of the main process expects a robot to do a search for matching XKCD comics and pass along a list for URLs to the candidates.

  
  

• Fetch comic task deep inside Featch and review results activity expects a robot to fetch a single comic and pass it along for a review.

  
  

Configuring Robocorp Cloud bots for these Camunda tasks require the following steps, or something alike:

1. Building and packaging a robot. Or multiple robots. Defining a convenient way to package and distribute Robot Framework robots is one of many achievements by Robocorp. In this case, it makes sense to implement all required steps in a single robot bundle.

2. Building and registering a Robot Workforce Agent. On-premises Camunda deployment, in practice, requires on-premises Robot Framework agents for convenient and secure communication between robots and Camunda. With Camunda Cloud it might be possible to solely use Robocorp Cloud agents instead, but that would be a completely different story…

   

   Note that Robocorp does provide a Dockerfile and instructions for building and customizing these on-premise agents.

3. Defining a Robocorp Cloud workforce “processes” to bind the executable robot tasks from robot packages with the registered workforce runtime environment agents.

   

   One for each external task in our BPMN process.

   

Try to not confuse Camunda processes with Robocorp Cloud workforce “processes”. Camunda executes business process defined in BPMN. Robocorp Cloud workforce “process” are more of a configuration concept for binding specific robot tasks to be executed with specific agent runtime environments.

Workforce “process” should work well allocating and monitoring computing resources for robot usage and configuring allowed concurrency for particular tasks. They could also be used to bridge on-premises workforce agents with cloud ones.

And the glue code to connect their APIs

So, how does all this really work out? Camunda Platform and Robocorp Cloud do not just magically interconnect, yet, but they do have APIs for that. Camunda external Task API can be used to fetch pending tasks, which then can be passed forward to Robocorp Process API. (Note: At the time of writing, Robocorp Cloud’s APIs arenot included in their free tiers.)

Obviously, there are many possible technical ways to actually implement the integration. This is my current approach:

Briefly explained:

1. A custom Camunda external task client fetches and locks pending external tasks from Camunda, schedules their execution at Robocorp Cloud, and keeps polling the Robocorp Process API and the task locked until its execution at Robocorp Cloud has been completed.

2. At Robocorp Cloud, a new workforce “process” run is triggered for each external task. Robocorp Cloud is responsible for choosing an available worker and controlling the execution.

3. Finally, a Robot Framework bot gets just enough external task details from its workforce “process’” “work item” to be able to fetch the task variables (including files) it needs directly from Camunda. The robot also sends the result variables back to Camunda by itself.

For convenience, the whole robot run is wrapped with a special Robot Framework listener library. That manages the completion or failing of the task, and submits its log files directly back to Camunda external task context.

With this approach, log files from automation bot executions are saved into related task contexts and preserved in Camunda process execution history. For both passing and failing executions. This makes the log files also available from Camunda Cockpit, and possibly saves the process owner from need to visit also Robocorp Control Room when in need for any bot execution details.

This work for integrating Camunda Platform with Robocorp Cloud (Control Room) is part of my carrot-executor-project (also available at PyPI). Just now, the documentation is still missing the parts for trying out this particular example, or trying it with Robocorp Cloud, but hopefully that comes soonish…