Eclipse Test & Performance Tools Platform (TPTP) Project
4.3 Plan (Approved)

Last revised 10/23/2006 09:23 PM Pacific Time ( marks interesting changes since September 2006)

    Please send comments about this plan to the tptp-pmc@eclipse.org PMC mailing list.

This document lays out the feature and API set for the TPTP 4.3 release.

The first part of this plan deals with the important matters of release deliverables, release milestones, target operating environments, and release-to-release compatibility.  These are all things that need to be clear for any release, even if no features were to change. 

The remainder of the plan consists of plan items for the four projects under the TPTP Top-Level Project.  Each plan item covers a feature or API that is to be added to TPTP, or some aspect of TPTP that is to be improved.  Each plan item has its own entry in the TPTP bugzilla database, with a title and a concise summary (usually a single paragraph) that explains the work item at a suitably high enough level so that everyone can readily understand what the work item is without having to understand the nitty-gritty detail.

Not all plan items represent the same amount of work; some may be quite large, others, quite small. Some plan items may involve work that is localized to a single component; others may involve coordinated changes to several components; other may pervade the entire project.

With the previous release as the starting point, this is the plan for how we will enhance and improve it.  Fixing bugs, improving test coverage, documentation, examples, performance tuning, usability, etc. are considered routine ongoing maintenance activities and are not included in this plan unless they would also involve a significant change to the API or feature set, or involve a significant amount of work. The intent of the plan is to account for all interesting feature work.

Release Deliverables

The following release deliverables are provided:

Release Milestones

The TPTP 4.3 release is targeted for general availability on 27-Oct-2006.  All release deliverables will be available for download as soon as the release has been tested and validated in the target operating configurations.  The first iteration is dedicated to defect fixing, automation of regression testing, and measuring and increasing test coverage for public API. There will be no enhancements committed to CVS HEAD stream until end of iteration #1, except new features that were made available as technology preview in TPTP 4.2 release. In iteration #2, 4.2.1 release branch will be created in CVS and enhancements in plan for 4.3 release will be allowed to be committed to CVS HEAD stream. Iteration #3 is for shutting down development and get ready to release 4.3 GA.

Release Milestones

Milestone Date Description
Iteration 1 Friday, 04-Aug-06 Stable build - bug fixes only (CVS stream common with 4.2.1)
Iteration 2 Friday, 22-Sep-06 Stable build - enhancements and bug fixes
Iteration 3 Friday, 17-Nov-06 Stable build - shut down
  Monday, 04-Dec-06 General Availability

For a detailed development schedule of TPTP 4.3 release, click here.

Target Operating Environments

In order to remain current, each TPTP release targets reasonably current versions of the underlying operating environments.

Most of the TPTP SDK is "pure" Java™ code and has no direct dependence on the underlying operating system.  The chief dependence is therefore on the Java 2 Platform itself.  The TPTP 4.3 release is written and compiled against version 1.4 of the Java 2 Platform APIs, and targeted to run on version 1.4 of the Java 2 Runtime Environment, Standard Edition.

There are many different implementations of the Java 2 Platform running atop a variety of operating systems. We focus TPTP testing on a handful of popular combinations of operating system and Java 2 Platform; these are our reference platforms.  TPTP undoubtedly runs fine in many operating environments beyond the reference platforms we test.  However, since we do not systematically test them we cannot vouch for them.  Problems encountered when running TPTP on non-reference platforms that cannot be recreated on any reference platform will be given lower priority than problems with running TPTP on a reference platform.

TPTP SDK 4.3 is tested and validated on the following target reference platforms (this list may be updated over the course of the release cycle):

TPTP Agent Controller Reference Platforms
Processor architecture Operating system
Intel IA32 Red Hat Linux v7.1, v7.2, v7.3, v8.0
Intel IA32 Red Hat Linux Advanced Server v2.1
Intel IA32 SuSE Linux v7.2, v7.3
Intel IA32 SuSE Linux Enterprise Server (SLES) v7, v8
Intel IA32 Windows 2000 Advanced Server (service pack 2)
Intel IA32 Windows 2000 Professional (service pack 2)
Intel IA32 Windows 2000 Server (service pack 2)
Intel IA32 Windows NT 4.0 (service pack 6a)
Intel IA32 Windows Server 2003
Intel IA32 Windows XP Professional
Intel EM64T Red Hat Linux Advanced Server v3
Intel EM64T Windows 20003 Server (service pack 1)
Intel IPF Red Hat Advanced Server v3
Intel IPF Windows 20003 Server (service pack 1)
iSeries OS/400 V5R1, V5R2
PA-RISC HP-UX v11.0, v11i
RS/6000 AIX v4.4.0, v5.1, v5.2
SPARC Sun Solaris v8, v9
zSeries z/OS V1R7
zSeries SuSE Linux Enterprise Server (SLES) v8
PowerPC/64-bit Red Hat Enterprise Linux AS release 3

Although untested, TPTP should work fine on other OSes that support the same operating system kernel and version.

Internationalization

TPTP is designed as the basis for internationalized products. The user interface elements provided by the TPTP SDK components, including dialogs and error messages, are externalized. The English strings are provided as the default resource bundles.

Latin-1 locales are supported by the TPTP SDK on all of the above operating environments; DBCS locales are supported by the TPTP SDK on the Windows, GTK, and Motif window systems; BIDI locales are supported by the TPTP SDK only on Windows operating environments.

The TPTP SDK supports GB 18030, the new Chinese code page standard, on Windows XP and 2000, and Linux.

TPTP supports ICU4J starting in 4.2 release. This will significantly increase the number of supportable locales. Products needing to localize to newer locales are enabled. German, Traditional Chinese, and Arabic are tested. 

Compatibility with Previous Releases

TPTP 4.3 will be compatible with TPTP 4.2.  The following specifies details of the various aspects of release compatibility.

Themes

The TPTP PMC adopted and specialized the following Eclipse themes which represent the key focus areas for TPTP enhancements in the year ahead.

Projects

The TPTP top-level project is is comprised of four projects, managed in a coordinated fashion, across which the plans items are allocated.  TPTP projects include:

Features

Plan items targeted for this release represent the addition of new features or areas where existing features will be significantly reworked or enhanced.  Plan items are allocated to themes and projects indicated above.

TPTP Platform Project Plan Items
Status Description
In Plan Support existing RAC clients and agents on the Linux IA32 platform using the new Agent Controller technology through compatibility layers(125103).
In Plan TPTP profiler (based on JVMTI) will be further improved in this release cycle with a heap profiler (147653) thread profiler (147654), and improved performance summary information (147658)
In Plan This requirement is to make the Log Analyzer UI format-agnostic when displaying Common Base Events as the event format will change and evolve over time. Any view, dialog, or editor that currently displays Common Base Events should display the events in a consistent manner with consistent naming. 71682
In Plan Provide the Log Analyzer functionality available in TPTP as an RCP application. The TPTP Log Analyzer will be restructured so that it can be packaged as a Rich Client Platform application. The RCP application will provide log visualization, analysis and correlation. 134531
In Plan The TPTP log analyzer provides a set of useful services to analyze log files. Most of these services are tightly tied to the log and trace analyzer user interface. It is required to decouple these services from the UI so that consuming products can reuse the above functionality. This enhancement is used to define a set of interfaces to expose the log analysis and correlation services available in the TPTP Log Analyzer function. 143766
Helpwanted
We are investing in automating post-build steps in deploying and executing regression tests. This has been dire need for the project and should positively impact quality and productivity. Automate the deployment and set up of Agent Controller on all platforms (144958)
Helpwanted This enhancement will improve the usability of dynamic probekit feature by automatically deploying the support class files that contain byte code for the user defined probes (143251)
Helpwanted Web document for creating custom profilers with Probekit 147450
Helpwanted Explore design approaches to improve scalability and responsiveness of EMF-based data models for trace and test tools.[Trace Model 147656] [Test Model 144950]
Helpwanted Provide capability in the agent framework to dynamically determine if the data collection services are available to an agent. The data collection services include agent controller and file transfer service. 142742

 

TPTP Testing Tools Project Plan Items
Status Description
In Plan Improve JUnit integration between TPTP and JDT. Allow the JUnit user to have the same experience, once TPTP is installed, as he/she used to have in Eclipse SDK, with the possibility of executing his/her JUnit tests in the TPTP framework. Any JUnit test would be edited and created as usual, and still it would be integrated with other types of TPTP tests, and could be executed in the TPTP framework. (90628) [Theme: Appealing to the Broader Community, Simple to Use]
In Plan Provide a generic Java API recorder Based on the generic recording facility provided in TPTP v4.1+ (see enhancement #74926), provide a generic recorder to record Java API invocations to create a test. The generic recorder could leverage the TPTP Probe Kit to instrument target APIs to capture invocations based on predefined criteria. (119688) [Theme: Design for Extensibility: Be a Better Platform]
In Plan Provide the concept of an object mine inside each test suite The object mine will keep track of a single id of an object irregardless of the number of times it occurs in the test cases owned by the test suite. The user will also be able to include external object mines (i.e. object mines owned by other test suites) and contribute any newly detected objects to an external object mine. (144763) [Theme: Appealing to the Broader Community, Simple to Use]
Helpwanted Providing better means for the user to specify a workspace location An extra page specific to the auto GUI test suite should be included in the launch configuration dialog to allow the user to specify a workspace and the plug-ins that they wish to include in the launched workbench. (109880) [Theme: Appealing to the Broader Community, Simple to Use]
Helpwanted Automated Documentation Generation The feature being proposed it to utilize the GUItester to create SIMPLE documentation (in HTML format, or another generic output of the users choosing) that outlines the (1) name of the UI test; (2) the series of steps taken in the UI test; (3) screenshots at every possible step during the test. (110108) [Theme: Appealing to the Broader Community, Simple to Use]
Helpwanted Support RCP applications for recording and playing back GUI test cases A common feature that users ask for is to have support in recording and playing back test cases in an RCP application. (114159) [Theme: Rich Client Platform]
Helpwanted Generic Recorder Framework improvements. Support for launching and controlling two or more recorders or recording agents in parallel. Support for launching Java applications for recording without creating/using an Eclipse launch configuration. Support remote recording via the Agent Controller. Provide an extension point for user-defined annotations to a recording session. Support (e.g. model loading) for recorders to emit standard trace model events. (145146) [Theme: Appealing to the Broader Community, Simple to Use]
Helpwanted The test launch configuration needs to have the means for users to specify JVM argument. Support for a standard way for users to specify JVM arguments for test execution. Provide an UI for the user to specify zero or more JVM arguments and an API for JUnit-based test runners to access these JVM arguments for launching the Java test process. (143121) [Theme: Design for Extensibility: Be a Better Platform]

 

TPTP Tracing And Profiling Tools Project Plan Items
Status Description
In Plan Providing better analysis views for analyzing execution statistics The following enhancements should make analysis of execution statistics easier: Easily identify the method that has had the maximum base/cumulative time. Quick hover explanation in the view or better naming for the base or cumulative time. The method statistics view should simply should show the stack for a particular method invocation. (147930) [Theme: Appealing to the Broader Community, Simple to Use]
In Plan  Provide better abstraction and navigation means in the Trace views The following feature tries to outline some improvements that may help to abstract and navigate the profiling data: Provide a find operation under the execution time statistics, memory statistics, and code coverage view. Ability to bring deltas to the top or only showing deltas between refreshes Showing what’s changed when deltas are displayed Doing post filtration in the views Doing pagination in the tabular views Filtering methods that have a base/cumulative time that is less than a given threshold (147931) [Theme: Appealing to the Broader Community, Simple to Use]
In Plan  Using virtualization in stat views Virtualization can be used to reduce the amount of time required to show a large number of tree items. This can significantly help improve the performance of the execution stat view when it is viewed at the method level. (147933) [Theme: Scaling Up]
In Plan  Create Extension Point for ARM Factories Create a new extension point to allow user to define there own Application Resource Measurement Factories. The extension point would include transaction, metric, and reporting. (148048) [Theme: Design for Extensibility: Be a Better Platform]
Helpwanted Trace-related views should make use of view linking service Currently some views listen for selection events on other views and make use of this information to update their own selections to something related to the original selection. For example, if you have the log view and log interactions view open at the same time, and select a log record in one of them, the other view will automatically update and select that same log record, if it's there. All the views should use a standard view linking service to do this. (87605) [Theme: Design for Extensibility:Be a Better Platform]
Helpwanted Providing better analysis views for detecting memory leaks The following enhancements need to be made to make memory leak detection easier; Objects that are garbage collected need to be removed from the object reference view. Integrate the object reference view with the memory statistics view. Provide a dynamic chart that is updated when the heap size changes Enhanced look & feel of the object reference view. (147929) [Theme: Appealing to the Broader Community, Simple to Use]
Helpwanted Add a thread analysis and a call graph view A thread analysis view will allow users to resolve deadlock problems, thread starvation or other thread related issues. A call graph view will help to easily visualize application flow and critical paths that maybe the cause of a bottleneck. (147932) [Theme: Appealing to the Broader Community, Simple to Use]

 

TPTP Monitoring Tools Project Plan Items
Status Description
In Plan Improve usability of Stand-alone Generic Log Adapter package. Applications can package and make use of GLA run-time and adapter configuration files to parse log files. However, to use the packaged adapter files they must be modified. For example, the adapter configuration must specify the location and name of the log file to parse. Currently, the only way to do this programmatically is by using XML parsing libraries to read and modify these XML files. To simplify adapter configuration modification, this feature will extend the current org.eclipse.hyades.logging.adapter.Adapter API to provide methods to retrieve and modify the configuration before it is executed by GLA. Also, to allow the packaged adapter configuration files to be used more easily in various applications, multiple outputters will be added to them, which can them be enabled or disabled depending on the application. (96433 and 126653) [Theme: Design for Extensibility: Be a Better Platform]
In Plan Provide sensor/extractor filtering capability in TPTP Generic Log Adapter log parsers. In TPTP 4.2 support was added to the Generic Log Adapter run-time for filtering at the sensor and extractor component level. This filtering capability will be exploited in the current TPTP provided log parsers to allow users to improve the performance of log parsing when only a subset of the log records is of interest. (141840) [Theme: Scaling Up]
In Plan Provide Rich Client Platform versions of Log Analysis tooling. Some users want stand-alone tools for log analysis. The TPTP Symptom Editor will be restructured so that it can be packaged as a Rich Client Platorm application. Also, some portions of the Log Analyzer will be restructured so it can be packaged as a Rich Client Platform application. (134532 and 134798) [Theme: Rich Client Platform]
In Plan Improve the Managed Resource Explorer. In TPTP 4.2 a technical preview was introduced for a generic Managed Resource Explorer. This tool will be improved in 4.3 so that it can become a full TPTP component. (145056, 145057, 145058, 150259, 150385) [Theme: Design for Extensibility: Be a Better Platform]
In Plan Add tooling to simplify creation of WSDM managed agents. In TPTP 4.2, a Managed Resource Explorer was added that allowed the developer to manipulate resources that sit behind a manageability endpoint. The Managed Agent explorer has plugins for JMX and WSDM. This feature for 4.3 would allow a developer to easily create WSDM based managed agents (referred to in WSDM as manageability endpoints) that are compliant with the WSDM 1.1 specification from OASIS. (142543) [Theme: Design for Extensibility: Be a Better Platform]
In Plan Improve the Monitoring Instrumentation tooling. In TPTP 4.2 a technical preview was introduced for tooling to instrument existing Java applications for monitoring. This support will be improved in 4.3 so that it can become a full TPTP component. ( 146272, 148042, 148044) [Theme: Design for Extensibility: Be a Better Platform]

Defects

In addition to the targeted features for this release, we plan to reduce the defect backlog. Defects are prioritized based on severity, age, and resource availability. We encourage users to report defects and we will do our best to fix them in priority order. The goal is clear backlog of major/critical/blocker defects and make reasonable progress on fixing as many as possible.

See TPTP 4.3 Defects for a listing of already fixed defects, current defect targets and backlog.

Select "4.3 [Completed | nil| I1 | I2 | I3] bugs" tabs.