Setak: A Framework for Stepwise Deterministic Testing of Akka Actors

Welcome to the Setak project homepage. Setak is a testing framework for Akka actor programs. It is similar to an earlier testing framework for Scala actors called Setac. Setak is especially suited for test cases that find bugs related to non-determinism. It allows programmers to specify the order of messages during the execution of a test. Also it makes it easy to check test assertions that require actors to be in a stable state, i.e., no actor is processing any message and no message can be processed (until new messages are delivered). One big advantage of Setak is that controlling the non-determinism is completely decentralized, while Setac has a centralized scheduler. In the future, we plan to enable Setak to run test cases in a distributed environment.

New Feature: Support for Non-Stable Systems

There might be the case that the system does not get stable, i.e. the actor receives the messages frequently. We have added a feature in Setak to support these cases. Please check how to use by example page for more details.

On This Page

• Overview
• Downloads
• How to Use?
• Publication

Overviewback to the top

In Setak, the programmer can define relevant messages in the program under test as test message envelops for controlling their orders and checking assertions. Specifically, these are messages whose delivery/processing status is important for the purpose of testing. Controlling the order between some messages brings an element of determinism to test execution, hence a part of the name for Setak. Assertions should only be checked when the results are ready. In actor programs this usually happens when the individual actors process all the messages they can, i.e., the system reaches a stable state where no message can be processed (until new messages are delivered). This point of stability can be viewed as the end of a step in the test execution. Indeed, Setak allows the programmer to break the entire execution of the test into multiple steps. Each step starts by delivering some messages, then lets the program run, and finally checks assertions.

These features in Setak make it usable for both unit testing and inetegration testing of actor programs. It can also be used for white-box and black-box testing. Setak has been integrated with JUnit and ScalaTest.

Compared to the current approaches for writing actor tests, Setak provides several advantages:

• It allows the programmer to easier write tests with some explicit constraints on the message schedule. The schedule is the order of messages delivered to the actors. The constraints do not need to enforce order of all message from a test execution but can enforce ordering constraints only on a selected set of test messages. Allowing the programmer to enforce some partial constraints avoids two extreme situations: (1) the programmer has no control over the schedule, or (2) the programmer should specify all the details of the schedule although the order of some messages does not matter.
• It makes it easy to check assertions in the stable state, which is usually the appropriate time for checking assertions. Without Setak, checking that the system is in a stable state is not straightforward in most cases.
• It allows accessing the delivery/processing status of test message envelops which enables new types of assertions to be succinctly encoded. Additionally, providing a way to define test messages eliminates the need to change the program under test to keep track of the messages.
• It reduces the cost of changing the program for the purpose of testing: the only change needed to use Setak is to use Setak factory for creating actors. This change can be easily automated by dynamically replacing the current factory in Akka library with Setak factory.

Downloadsback to the top

Prequisites

• Scala version 2.9.1 or higher
• Akka version 1.2

You can find the latest version of Setak jar file, source code, and documentation at http://mir.cs.illinois.edu/setak/snapshots/.

Using Setak with Sbt

Add the dependency using the following lines:

resolvers += "setak" at "http://mir.cs.illinois.edu/setak/snapshots/"

libraryDependencies += "edu.illinois" %% "setak" % "1.0-SNAPSHOT"

Running from the source ode

The source code of Setak is hosted in Github. You can compile and run it by following the instructions in the README file.

How to Use?back to the top

In order to see how to use Setak, see how to use by example page that describes the usage of Setak by an example along with a summary of Setak features.

Publicationback to the top

Our previous version of testing framework for Scala actors was presented at Scala Days 2011.

Setac: A Framework for Phased Deterministic Testing of Scala Actor Programs

By Samira Tasharofi, Milos Gligoric, Darko Marinov, and Ralph Johnson.
Second Scala Workshop (Scala Days 2011)
Stanford, CA, June 2011. [pdf]. [ppt].