Jubula Functional Testing Tool

The project has been created. Please visit the project page.

Jubula Functional Testing Tool

The Jubula Functional Testing Tool project is a proposed open source project under the Eclipse Technology Project.

This proposal is in the Project Proposal Phase (as defined in the Eclipse Development Process) and is written to declare its intent and scope. We solicit additional participation and input from the Eclipse community. Please send all feedback to the Eclipse Proposals Forum.

Background

Existing functional testing tools work on the premise that test scripts are saved and managed in the form of code. Depending on the approach, the test automation may involve recording, generation or straight writing of code.

However, the introduction of code to acceptance testing has various negative side-effects:

  1. Recorded code is full of duplications and redundancies, as well as being closely linked to the implementation details. A recorded script cannot be used as a flexible, maintainable automated test until it has been refactored. This refactoring work (parameterization, abstraction, removal of duplications) must be done by someone who is an accomplished programmer.
  2. The work involved with writing or adapting code to be fit for automated testing is often underestimated. Ensuring that the test code is maintainable, understandable and flexible for all scenarios (on multiple systems) is painstaking work which benefits neither the testing nor the development directly.
  3. If the test base is code, then many testers will not be able to work with it. This removes the control over the tests from the testers and places it in the development team, where it must be dealt with alongside other high priority tasks. This introduces unnecessary delays in testing and hinders the test team in writing tests completely from the user perspective.
  4. Recording code involves the presence of a working application, but waiting until an application is ready and functional means that the tests lag behind the development and so errors are present in the software for longer before they can be found and resolved.
  5. The dependency on a working application means that recorded tests cannot be meaningfully used in agile processes, where acceptance testing must happen in parallel to development, or even beforehand if the development is being acceptance-test-driven.

The Jubula functional testing tool is based on the premise that automated acceptance tests are just as important as the project code, and should adhere to the same best practices (modularity, reusability, and readability) without requiring that any code be produced. This places the power of testing in the hands of the testers and improves accessibility for customers who may want to monitor the tests. The code-free approach keeps test maintenance to a minimum and allows acceptance tests to be written from the user perspective. By focusing on how the user will see the software (as a black-box) , the team can gain valuable quality information which often slips through when only JUnit tests are executed.

Test creation in Jubula is achieved using a library of actions which can be combined using drag and drop. This library has been successfully used in diverse projects and already consists of the vast majority of actions necessary to write acceptance tests.

The Jubula library is platform and application independent, which means that test creation can begin before the software is available. This is a must for agile processes, where acceptance tests must be created alongside or even before development. Traditional processes also benefit from the earlier involvement of the test team to clear up misunderstandings more quickly. Working with the Jubula library means that tests can be executed as soon as code is committed reducing the time spent between introducing an error and catching it via the tests.

Scope

Objectives of the Jubula project:

  • Provide tooling for testers and developers to do automated GUI testing on Java and HTML based applications.
  • Specify and maintain a model for test specifications and a persistance layer for the model elements.
  • Provide API for interested projects to generate test scenarios, execute automated tests and get results from these tests.
  • Integrate with other projects to get information to create test scenarios and submit results. This includes tracing changes made by other projects to their data and mirroring these changes back to the test scenarios.
  • Description

    Jubula will provide functional GUI testing support for Java and HTML applications. It will further try to be an anchor point for a broader testing scope including requirements analysis, code coverage of Java applications and test metrics.

    The initial components will be
    • Plug-ins which provide the GUI for specifying and maintaining test scenarios.
    • Plug-ins which will provide a persistance layer for the models defined above.
    • Plug-ins for executing automated tests based on the defined scenarios.
    • Plug-ins to test Swing, SWT, RCP, GEF and HTML based applications
    • Server to control test targets on remote platforms.
    • Java agents to control the test targets and provide the test environment.
    • Integration in the JDT and WDT project to enable developers to define and use automated tests.
    • Wrapping the plug-ins into a standalone application to enable testers to define and use automated tests.
    • Tools to use test in batch builds.
    • Extensive online help and external documentation for the tool user.
    • Documentation on extending various parts of the tool.

    Relationship with other Eclipse projects

    Jubala will use EclipseLink as a persistance layer.

    It will integrate with JDT and WDT.

    It integrates with Mylyn.

    It supports testing of SWT, RCP and GEF components. Testing RAP is currently under evaluation.

    We will check on how TPTP and Jubula might benefit from each other. There are areas, for instance code coverage, where the tools complement each other. To our knowledge there is no current development in TPTP on GUI tests or other funtional tests.

    SWTbot is aiming at programmed jUnit tests and will probably be used mostly by programmers. Jubula on the other hand is focused on non-programmers specifying test scenarios.

    Initial Contribution

    BREDEX GmbH is offering code from their commercial tool GUIdancer as the intial contribution. This offering consists of the core functionality of the GUIdancer testing tool owned by BREDEX. The code base consists of 2000+ Java classes with 350,000+ lines of code. Some code developed using third-party libraries or developed under a NDA will be removed prior to the contribution.

    Committers

    BREDEX will continue working on the project. The current development team of the product version will continue working on the Jubula project.

    The following individuals are proposed as initial committers to the project:

    Achim Lörke, BREDEX GmbH, Project lead
    Oussama Bouchhioua, BREDEX GmbH
    Zeb Ford-Reitz, BREDEX GmbH
    Dennis Grabow, BREDEX GmbH
    Alexandra Imrie, BREDEX GmbH
    Tim Winselmann, BREDEX GmbH
    Markus Tiede, BREDEX GmbH

    We welcome additional committers and contributions.

    Mentors

    The following Architecture Council members will mentor this project:

    • Bernd Kolb
    • Tom Schindl

    Interested Parties

    The following individuals, organisations, companies and projects have expressed interest in this project:

    • Prof. Dr. Karin Vosseberg, Hochschule Bremerhaven (University), Department of Computer Science
    • Björn Schindler, TU Clausthal (Technical University), Department of Informatics
    • Stefan Kastigen, novaObjects GmbH
    • Dr. Frank Gerhardt, Gerhardt Informatics Kft.
    • Holger Staudacher, EclipseSource
    • Werner Keil, emergn
    • Benjamin Muskalla, Tasktop Technologies
    • Ashitha MS, SAP

    Project Scheduling

    Scheduled for Action
    Q4 2010 Working on internal changes to make the code base fit for the initial contibution.
    Q1 2011 Initial Contribution, Parallel IP, Build

    Changes to this Document

    Date Change
    14-October-2010 Document created
    20-October-2010 Added mentor, fixed typos
    21-October-2010 Final mentors
    26-October-2010 added some interested parties, mentioned consideration for RAP testing
    05-November-2010 added interested parties