Functional Testing made easy with SWTBot for Developers and Testers

Bonitasoft is a long-time user of SWTBot framework. The framework has been used to test UI since the beginning of Bonitasoft, 5 years ago. It was first used exclusively by "Eclipse-knowledgeable" developers because of its maintainability and stabilization difficulties; and also with the limitations on the testable components. Based on experience and evolutions, it is now used by Developers and QA Team in a smoother way.

1. Test components you wish

SWTBot supports various components. Of course, all SWT components are supported but it also provides tools for advanced components such as GEF, Eclipse Forms or Nebula Widget.

Are you using a specific component?

No problem! Adding support for a custom component is quite easy. For instance, we implemented support for Nebula Gallery and Nebula TableCombo. You can find the general idea of how to implement yours in this wiki page.

Nebula Gallery support has been integrated into the SWTBot project and will be available with the next release (thanks to Gerrit and SWTBot Project Lead Mickaël Istria).

2. Let Testers write Functional tests

At the beginning, writing SWTBot tests was an Eclipse experts' affair. As mentioned on the project description page, "SWTBot provides APIs that are simple to read and write". That’s totally true... for "Java developers".

The first tests are quite straightforward to implement, the developers have an easy access to the source code that they just wrote and the SWTBot API is really easy to use when you have minimal Eclipse/SWT skills.

It becomes tricky when you consider the following statements:

• You have to find the correct widget in old source code - Who is able to tell the difference between a Combo and a CCombo at first sight?
• The execution of the test requires that you start the whole Platform - For us, in debug mode, it can take between 30 seconds and 1 minute.
• The test requires that there be no interaction with the UI during its execution - Don't touch the keyboard or move your mouse! (The exception is that you are using Linux and have configured the tests to launch on another Display as explained here.)

Considering these issues, watching a test failing because a ".click()" is missing or there is a misspelled button label can be frustrating! In addition, each new test was resulting in - if not duplicating - a lot of similar code.

With experience we start gathering some of the most recurrent actions in static methods (e.g : SWTBotTestUtil.createNewDiagram(bot)). That was a little better but not very satisfying as a lot of these methods were emerging here and there.

Up to this point only developers were writing tests. One objective of our Engineering Team was to enable the automation of functional tests for our QA Team. To be able to do this we decided to develop an in-house framework on top of SWTBot using the Page Pattern (see Martin Fowler's PageObject article and Selenium wiki). This pattern was already used internally for our web component.

This new framework provides a fluent API, for example:

  
  BotDataDialog botDataDialog =
  myBotApplication.newDiagram()
.getPropertiesView().
selectDataTab().addData();
botDataDialog .setName(“userName”)
.setType(“Text”)
.setDefaultValue(“john”).finish();

This snippet will add a variable in a process and fill out this following wizard page.

An API like this opens up the SWTBot testing capabilities to users with less technical profiles because of its Application Oriented Language. It is easier to develop, read and maintain. Even as a developer, you will spend less time writing a new "PageObject" and reuse it than writing specific SWTBot code for your test.

3. Ensure repeatable tests

Writing repeatable tests is not completely trivial as we are dealing with asynchronous code (the UI is executed in a dedicated thread).

Let’s illustrate with a little story. All of our SWTBot tests were running smoothly but slowly on our Continuous Integration (CI) server. A great effort has been made to improve performance of the whole CI infrastructure. One phase consisted of updating hardware. The test execution was so much faster than before!!!... that it broke a lot of our UI tests. Wait what? While going faster, the SWTBot test thread is going too fast for the UI one.

What can you do to make sure you have a robust test suite?

You can follow these little guidelines:

• Use Wait Condition intensively to ensure that the UI thread is where you expect it to be. For instance, wait for a shell to be opened, wait until a validation ends its run and enables a "Finish" button.
• Close remaining opened shells after a test has finished to avoid failing all upcoming tests of the test suite. You can use a JUnit RunListener as explained in this blog post or a common abstract class as explained here for a plain SWT application.
• Use the Internationalized String and not the hardcoded value of text displayed in the UI (even if hardcoding seems simpler at test writing time). For instance, while using NLS framework, you can simply export the package containing your NLS class as x-friends for your test plugin. You can check this article for more details.

What you should remember

Following some simple rules, developers are able to provide an easy-to-use and reliable Test Framework on top of their application. This framework eases writing and maintaining Functional Tests for Testers AND Developers.

What's next?

Now that we have a clean Application Oriented API, it will be much easier to get stakeholders and user advocates to join the party using some Business Driven Testing technology. This talk planned for Project Quality Day at EclipseCon Europe might provide some inspiration