Component Behaviour Evaluation Project (ComponentBee)
This proposal has been withdrawn.
Component Behaviour Evaluation (ComponentBee) is a proposed open source project under the Eclipse Technology Project.
This proposal is in the Project Proposal Phase (as defined in the Eclipse Development Process) and is written to declare its intent and scope. This proposal is written to solicit additional participation and input from the Eclipse community. Please send all feedback to the eclipse.componentbee newsgroup.
ComponentBee project provides an extensibile frawework for tools that are used to evaluate the dynamic behaviour of software components in unit and integration tests. In brief, such tools are able to log the dynamic behaviour of software components, extract behaviour patterns from the collected log, and finally calculate reliability (or other quality) values for the tested components. ComponentBee also faciliates the creation of visual test model editors.
The construction of reliable component-based SW systems requires methods to facilitate the SW integrators to ensure that the SW architecture, selected components, and finally the constructed SW system meet the desired quality requirements. Component Behaviour Evaluation (ComponentBee) framework supports the reliability testing of software components and integrates these testing features to support architectural-level reliability analysis of software systems.
The ComponentBee project develops an extensible framework and exemplary tools to evaluate the reliability and quality of software components that are executed in multiple threads. The project does not focus on dynamic behaviour monitoring but utilises existing TPTP tools in trace and raw log data recording.
The aim of the project is to develop a framework for tools being capable of evaluating dynamic behaviour of software components from trace and raw log data. The framework provides interfaces for: 1) the trace and raw log data and 2) for processors that will process and compose new presentations for trace and raw log data. The trace and raw log data interfaces provide an access to data structures that present the dynamic behaviour of software components. The processors will utilise these trace and raw log data interfaces and compose different kinds of presentations (data structures) for dynamic behaviour of software components. More precisely, the processors will: 1) compose an abstract tree presentation for dynamic behaviour of software components performing concurrent processing in different threads, then 2) extract behaviour patterns from the abstract tree presentation, and finally 3) evaluate the extracted behaviour patterns and calculate quality and reliability values for the evaluated software components. In addition, the ComponentBee framework provides a domain-specific-language (DSL), which supports the creation of visual test model editors.
ComponentBee project also provides exemplary tools that are currently capable of evaluating dynamic behaviour of multithreaded Java programs. The provided tools are model-based tools that are integrated into Eclipse. They include a visual editor, which can be used to create generic and reusable test models and, then, to refine these test models to support the testing of specific Java components.
- Support evaluation of concurrent behaviours executed in various threads. ComponentBee makes it possible to takee message sequences executed in different threads as an input, recognize how these different threads interact, compose an overall presentation (a behaviour tree) for the concurrent behaviours, and finally extracs the behaviour patterns from the behaviour tree.
- Support for reusable and configurable plugins. The input data sources can be configured and thus the ComponentBee plugins can be adapted to support behaviour testing of different kinds of component-based applications.
- Support for both generic and domain and application-specific test model creation. ComponentBee provides an extensible language and framework for creating visual test model editors. The editors are based on the Graphical Modeling Framework (GMF). The resulting test models are Eclipse Modeling Framework (EMF) models. The ComponentBee framework supports refining more abstract test models with domain or application-specific test models.
- Dynamic instrumentation. The ProbeKit platform of TPTP enables integrators to insert Java code fragments called probes into a program to collect information about its dynamic behaviour without adding test code to the actual software components. ComponentBee takes the test model as an input and generates probes to record raw log information about the input attributes and return values of methods and the states of the components to XML-based behaviour logs.
- Utilisation of information embedded in source code. The ComponentBee uses JDT for obtaining information about Java interfaces and classes.
- Support for developing, debugging and running Evaluator plug-ins within one workbench. Reliability evaluation with ComponentBee requires development of Evaluator plug-ins (by extending abstract plug-ins provided by the framework). ComponentBee package includes a supplementary component, which aims to make the development of these plug-ins more fluent. By default, Eclipse PDE requires a user to start another (run-time) instance of the Eclipse workbench before a newly implemented plug-in can be tested. However, starting a run-time workbench is a time- and memory-consuming task. Therefore, we developed an additional component which makes it possible to develop, debug and run Evaluator plug-ins within one workbench. This component considerably increases usability of the ComponentBee framework and tools (but they can also be used without it).
Relationship with other projects
Dependences on other Eclipse projects
The ComponentBee framework is based on components from several Eclipse projects. The most important relationships are the following.
- The dynamic behaviour logging of software components is based on the Eclipse TPTP probes
- Eclipse Java Development Tools (JDT) enables the ComponentBee to access information embedded in source code of Java software components.
- The visual editor of the ComponentBee is based on EMF and the GEF.
Dependency on JavaCC
- Java Compiler Compiler (JavaCC) is an essential part of ComponentBee.
- JavaCC is distributed under the Berkeley Software Distribution (BSD) License (without the advertising clause).
- ComponentBee uses JavaCC as follows:
- first generates a BNF (Backus-Naur Form) grammar that defines production rules for the behaviours that are described in the test model
- then calls the JavaCC to generate a BehaviourParser for the grammar
- finally calls the generated BehaviourParser to extract behaviour patterns from the pre-processed raw log data
- Java CC is currenlty included in the ComponentBee installation package. This is convenient, but not strickly necessary. Users could alternatively download it from the JavaCC homepage and then configure ComponentBee to use the seperately installed library. This would obviouly complicate deployment.
The current version of the ComponentBee consists of four main packages:
- The test execution package provides components capable of producing information from the dynamic behaviour of components and finally writing a test report of the observed behaviours.
- The behaviour presentation package provides various kinds of presentation components capable of presenting dynamic behaviour of components.
- The test model package provides components enabling software integrators to define various kinds of test models supporting R&A testing of OS components.
- The test creation package provides tool components to help software integrators to create test models for the R&A testing.
In order to increase the level of modularity, the components of the packages communicate with each other via predefined interfaces. Each package is divided into core and implementation parts. The core part provides interfaces for the components of the package whereas the implementation part provides the reference implementations for the core interfaces. In addition, predefined extension points enable developers to build on ComponentBee.
The ComponentBee framework provides extension points for plug-ins that can support test creation, utilisation of UML models in testing, or recording and evaluation of raw and trace data about dynamic behaviour of SW components. Two kinds of extension points exist in the ComponentBee: static (ie. Eclipse) extension points and dynamic extension points. The Plug-in Development Environment (PDE) provides tools to create, develop, test, debug, build and deploy Eclipse plug-ins by using the static extension points. The dynamic extension points provide a mechanism to extend the ComponentBee at run-time with the plug-ins being developed in the workspace of Eclipse. The current set of extension points are listed in the following table.
Organization of the development work
VTT Technical Research Centre of Finland is offering the current version of the ComponentBee as an initial codebase (see http://opensource.erve.vtt.fi/componentBEE/main.html). The ComponentBee model and tools have been built in accordance with the proposed concept. The current code base consist of 22146 SLOC hand-written code (152 classes/interfaces) and a GMF-generated visual editor with 360 classes. The current code base has been made availabe under the Eclipse Public Lisence (EPL)
Proposed initial committers
- Marko Palviainen, VTT (firstname.lastname@example.org, proposed as a project administrator)
- Antti Evesti, VTT (email@example.com)
- Katja Henttonen, VTT (firstname.lastname@example.org)
The following parties (from outside of VTT) have expressed interest in the ComponentBee project and allowed using their name in this proposal:
- Hans Petter Dahle, ICT-Norway (Hans.Petter.Dahle@fornebuconsulting.com)
- Mattsson Anders, Combitech (Anders.Mattsson@combitech.se)
- Fransisco Yuste, Telvent (email@example.com)
- Frank van der Linden, Philips Healthcare (firstname.lastname@example.org)
- Carlos Acuña, Telefonica (email@example.com)
- José María, Telefonica (firstname.lastname@example.org)
- Rauno Saarnio, Meritie (email@example.com)
The given parties are our partners in the research project (ITEA-COSI) which has financed the development of the ComponentBee. We have not yet tried to promote the ComponentBee tool outside of the research team.
The further development of the ComponentBee tool is still on-going work. We would greatly appreciate improvement suggestions and feedback from the Eclipse community. The proposed initial contributors are there to develop the tool into the direction desired by the community. In our opinions, following improvements are needed, for example: (1) more plug-ins (UMLImporters) capable of generating test model templates of UML models (e.g. of sequence diagrams), (2) new Eclipse editors enabling testers to define extension elements refining the test model templates for the actual reliability tests and (3) new plug-ins supporting evaluation of other quality attributes of concurrent Java programs (e.g. performance attributes).