Having fun at open-source automation playground

Successful business process automation requires real ability to iterate the automation part. During the last year, I went through many ideas and iterations on how to make Camunda Platform BPMN external service task handlers more convenient to develop and execute – especially in Python or Robot Framework. By the end of the year, I surprised myself by settling down on a quickish solution based on Robocorp RCC: a lightweight wrapper concept I named carrot-rcc.

Standing on the shoulders…

After struggling in mapping many Camunda and BPMN concepts to Robot Framework, I seem to have learned the best practise to be simply: do not.

What if one could just go to Camunda provided documentation to learn about process modelling and Robocorp provided documentation to learn about RPA and automation (or just “glue coding” in general). Then use their optimized tools to iterate on both, and eventually everything would just work.

That’s what carrot-rcc is about. Carrot-rcc works as a bridge between Camunda Platform external service tasks and Robocorp style Python or Robot Framework robot packages. It converts Camunda Platform process task state into Robocorp style work item, and then uses Robocorp RCC to execute the robot package for it (by matching external task topic to robot task name). Finally, the external task state is updated from the saved work item, and then the task is either completed or marked failed at the Camunda Platform process.

Carrot-rcc is not really a lot of code. So it could also be taken as a concept to be re-implemented with the favourite language.

The playground

Open-source mashup on top of Camunda Platform and Robocorp toolchain provides extremely flexible automation foundation. The downside is that, because that’s not a single polished product, putting it all together requires some effort.

Fortunately, carrot-rcc approach can be tried out using prepared playground implemented as a Vagrant provisioned virtual machine:

Screencast of developing Hello World process with a Robot Framework bot with the carrot-rcc playground

Vagrant is a popular developer tool for managing development environments. And it is able to start a new local playground with just:

vagrant init datakurre/carrot-rcc
vagrant up

That will setup a virtualized desktop environment with everything required, ready to use, especially:

• Camunda Platform Community Edition with minimal history plugin and configuration to load and migrate process definitions from the desktop folder named BPMN

• carrot-rcc with configuration to load zipped Robocorp robot packages from the desktop folder named Robots

• Camunda Modeler with recommended plugins for BPMN process modelling

• VSCode with Robocorp extensions for RPA and generic automation code development

• the current working directory mounted onto the desktop as folder named Shared.

Of course, both Vagrant and supported virtualization environment like VirtualBox must be installed beforehand. (Alternatively also libvirt should be supported and hyperv could be later.)

To be even more honest, a configuration file with better defaults is recommended for better experience. That means a file named Vagrantfile with the following content:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.box = "datakurre/carrot-rcc"

  config.vm.provision "rebuild", type: "shell",
    inline: "sudo nixos-rebuild switch --max-jobs 1"

  config.vm.provider "virtualbox" do |vb, override|
    vb.gui = true
    vb.memory = 4096
    vb.cpus = 4
    override.vm.provision "remount", type: "shell",
      inline: "sudo mount -t vboxsf vagrant /vagrant -o umask=0022,gid=1000,uid=1000"
  end
end

With that file in place, a new playground can be started with simply:

$ vagrant up --no-provision

And upgrade in-place at any time to the latest available version with:

$ vagrant provision

In-place upgrades with vagrant provision are possible, because the playground virtual machine has been built on top of NixOS (check the repository for the full virtual machine configuration in Nix).

Finally, the playground may be removed with:

$ vagrant destroy

A walkthrough

The playground starts with a minimal Xfce desktop environment with all required applications, web services and special folders (BPMN, Robots and Shared) directly on the desktop. Terminal and other usual Xfce features are available using the right click context menu.

Playground packages Camunda Modeler with many useful plugins, like BPMN linter and token simulation, preinstalled.

VSCode comes packaged with Robocorp Code and Robot Framework LSP extensions, which together provide top of the line Robot Framework bot development experience. This includes support for mock data and interactive debugging.

Another cool feature in Robocorp Code is Jypyter Notebook -inspired interactive console to support more exploratory development style.

Because everything in the playground is set up to work well together, processes may be deployed and started directly from the modeler, out of the box.

That said, the recommended way to deploy simple models on the playground is saving them to BPMN-named folder on the desktop. Because the playground runs Micronaut-powered Camunda with minimal startup time, it has been possible to configure it to deploy models directly from the filesystem with restart and re-deploy. Also, during re-deploy, all running processes are automatically migrated to the latest BPMN versions on when possible.

Packaged Camunda Platform build also comes with my minimal history plugin to make inspecting completed tasks and processes more convenient with the open-source version of the platform.

And if something does not seem to be running right, both Camunda and carrot-rcc services stream their logs into system(d) journal, which can be viewed either from terminals or by clicking Journal icon on the desktop.

The only thing that is missing its own UI is, how to package developed version of a bot into a package recognized by carrot-rcc service. For now, it must be done manually on a terminal with running rcc robot wrap -z /var/lib/carrot-rcc/RobotName.zip while inside the folder of a developed robot.

Examples, examples…

Personally, I feel the playground to be a low barrier environment to experiment with.

Fun and easy enough that my intention is to stream how to use the playground every now and then during this spring with Markus Stahl. For example, the last Friday, we converted the latest Robocorp course: Work data management to run on the playground, replacing Robocorp Cloud orchestration with playground’s Camunda Platform.The result, and hopefully more on later, is available at our “GitHub show notes”.

Fun fact: moving to Camunda reduced the lines of robot code (by moving some logic into BPMN).