10/23/2006 09:23 PM
Pacific Time (
marks interesting changes since September 2006)
Please send comments about this plan to the firstname.lastname@example.org PMC mailing list.
This document lays out the feature and API set for the TPTP 4.3 release.
The first part of this plan deals with the important matters of release deliverables, release milestones, target operating environments, and release-to-release compatibility. These are all things that need to be clear for any release, even if no features were to change.
The remainder of the plan consists of plan items for the four projects under the TPTP Top-Level Project. Each plan item covers a feature or API that is to be added to TPTP, or some aspect of TPTP that is to be improved. Each plan item has its own entry in the TPTP bugzilla database, with a title and a concise summary (usually a single paragraph) that explains the work item at a suitably high enough level so that everyone can readily understand what the work item is without having to understand the nitty-gritty detail.
Not all plan items represent the same amount of work; some may be quite large, others, quite small. Some plan items may involve work that is localized to a single component; others may involve coordinated changes to several components; other may pervade the entire project.
With the previous release as the starting point, this is the plan for how we will enhance and improve it. Fixing bugs, improving test coverage, documentation, examples, performance tuning, usability, etc. are considered routine ongoing maintenance activities and are not included in this plan unless they would also involve a significant change to the API or feature set, or involve a significant amount of work. The intent of the plan is to account for all interesting feature work.
The following release deliverables are provided:
The TPTP 4.3 release is targeted for general availability on 27-Oct-2006. All release deliverables will be available for download as soon as the release has been tested and validated in the target operating configurations. The first iteration is dedicated to defect fixing, automation of regression testing, and measuring and increasing test coverage for public API. There will be no enhancements committed to CVS HEAD stream until end of iteration #1, except new features that were made available as technology preview in TPTP 4.2 release. In iteration #2, 4.2.1 release branch will be created in CVS and enhancements in plan for 4.3 release will be allowed to be committed to CVS HEAD stream. Iteration #3 is for shutting down development and get ready to release 4.3 GA.
|Iteration 1||Friday, 04-Aug-06||Stable build - bug fixes only (CVS stream common with 4.2.1)|
|Iteration 2||Friday, 22-Sep-06||Stable build - enhancements and bug fixes|
|Iteration 3||Friday, 17-Nov-06||Stable build - shut down|
|Monday, 04-Dec-06||General Availability|
For a detailed development schedule of TPTP 4.3 release, click here.
In order to remain current, each TPTP release targets reasonably current versions of the underlying operating environments.
Most of the TPTP SDK is "pure" Java™ code and has no direct dependence on the underlying operating system. The chief dependence is therefore on the Java 2 Platform itself. The TPTP 4.3 release is written and compiled against version 1.4 of the Java 2 Platform APIs, and targeted to run on version 1.4 of the Java 2 Runtime Environment, Standard Edition.
There are many different implementations of the Java 2 Platform running atop a variety of operating systems. We focus TPTP testing on a handful of popular combinations of operating system and Java 2 Platform; these are our reference platforms. TPTP undoubtedly runs fine in many operating environments beyond the reference platforms we test. However, since we do not systematically test them we cannot vouch for them. Problems encountered when running TPTP on non-reference platforms that cannot be recreated on any reference platform will be given lower priority than problems with running TPTP on a reference platform.
TPTP SDK 4.3 is tested and validated on the following target reference platforms (this list may be updated over the course of the release cycle):
TPTP Agent Controller Reference Platforms
|Processor architecture||Operating system|
|Intel IA32||Red Hat Linux v7.1, v7.2, v7.3, v8.0|
|Intel IA32||Red Hat Linux Advanced Server v2.1|
|Intel IA32||SuSE Linux v7.2, v7.3|
|Intel IA32||SuSE Linux Enterprise Server (SLES) v7, v8|
|Intel IA32||Windows 2000 Advanced Server (service pack 2)|
|Intel IA32||Windows 2000 Professional (service pack 2)|
|Intel IA32||Windows 2000 Server (service pack 2)|
|Intel IA32||Windows NT 4.0 (service pack 6a)|
|Intel IA32||Windows Server 2003|
|Intel IA32||Windows XP Professional|
|Intel EM64T||Red Hat Linux Advanced Server v3|
|Intel EM64T||Windows 20003 Server (service pack 1)|
|Intel IPF||Red Hat Advanced Server v3|
|Intel IPF||Windows 20003 Server (service pack 1)|
|iSeries||OS/400 V5R1, V5R2|
|PA-RISC||HP-UX v11.0, v11i|
|RS/6000||AIX v4.4.0, v5.1, v5.2|
|SPARC||Sun Solaris v8, v9|
|zSeries||SuSE Linux Enterprise Server (SLES) v8|
|PowerPC/64-bit||Red Hat Enterprise Linux AS release 3|
Although untested, TPTP should work fine on other OSes that support the same operating system kernel and version.
TPTP is designed as the basis for internationalized products. The user interface elements provided by the TPTP SDK components, including dialogs and error messages, are externalized. The English strings are provided as the default resource bundles.
Latin-1 locales are supported by the TPTP SDK on all of the above operating environments; DBCS locales are supported by the TPTP SDK on the Windows, GTK, and Motif window systems; BIDI locales are supported by the TPTP SDK only on Windows operating environments.
The TPTP SDK supports GB 18030, the new Chinese code page standard, on Windows XP and 2000, and Linux.
TPTP supports ICU4J starting in 4.2 release. This will significantly increase the number of supportable locales. Products needing to localize to newer locales are enabled. German, Traditional Chinese, and Arabic are tested.
TPTP 4.3 will be compatible with TPTP 4.2. The following specifies details of the various aspects of release compatibility.
API Contract Compatibility: Refer to Evolving Java-based APIs for a discussion of the kinds of API changes that maintain contract compatibility.TPTP SDK 4.3 will be upwards contract-compatible with TPTP SDK 4.2 or lower. Downward contract compatibility is not supported. There is no guarantee that compliance with TPTP SDK 4.3 APIs would ensure compliance with TPTP SDK 4.0 APIs.
Binary (plug-in) Compatibility: TPTP SDK 4.3 will be upwards binary-compatible with TPTP SDK 4.2. Downward plug-in compatibility is not supported. Plug-ins for TPTP SDK 4.3 will not be usable in TPTP SDK 4.2. Refer to Evolving Java-based APIs for a discussion of the kinds of API changes that maintain binary compatibility.
Source Compatibility: TPTP SDK 4.3 will be upwards source-compatible with TPTP SDK 4.2. This means that source files written to use TPTP SDK 4.2 APIs might successfully compile and run against TPTP SDK 4.3 APIs, although this is not guaranteed. Downward source compatibility is not supported. If source files use new TPTP SDK APIs, they will not be usable with an earlier version of the TPTP SDK.
Workspace Compatibility: TPTP SDK 4.3 will be upwards workspace-compatible with TPTP SDK 4.2 unless noted. This means that workspaces and projects created with TPTP SDK 4.2 can be successfully opened by TPTP SDK 4.3 and upgraded to a 4.3 workspace. This includes both hidden metadata, which is localized to a particular workspace, as well as metadata files found within a workspace project (e.g., the .project file), which may propagate between workspaces via file copying or team repositories. Downward workspace compatibility is not supported. A workspace created (or opened) by a product based on TPTP 4.2 will be unusable with a product based an earlier version of TPTP. Visible metadata files created (or overwritten) by TPTP 4.3 will generally be unusable with earlier versions of TPTP.
The TPTP PMC adopted and specialized the following Eclipse themes which represent the key focus areas for TPTP enhancements in the year ahead.
Scaling Up - TPTP will work to enhance the support of large data volumes and processing rates in areas such as data collection, user interface and in the persistence of trace, log and statistical models and execution histories.
Enterprise Ready - Hooks will be provided within the TPTP infrastructure to link testing tools to requirements tracking tools and defect tracking tools, thus embedding them effectively in enterprise development cycles. Changes to the data collection layers will increase interoperability with enterprise security infrastructure. In addition, there will be progressive adoption of the TPTP tools and infrastructure as a test platform for the project itself, which is in turn likely to drive refinements into the tools. An increased focus on whole-project integration testing will ensure effective interoperability amongst all TPTP components and the rest of the Eclipse environment.
Design for Extensibility: Be a Better Platform - There will be a wide range of activities within TPTP to externalize APIs and define extension points, making the infrastructure more flexible, and more generic in application. A good example of this is integration of TPTP with WTP and BIRT for web application testing, profiling and generation of customized reports of results.
Embedded Development - TPTP target execution environment and remote data collection framework provide capabilities that are adapted for high-end embedded systems. TPTP will seek contributions to add support for embedded systems. We are promoting use of TPTP native logging capabilities on a number of embedded target systems.
Rich Client Platform - TPTP will use RCP for building manual test client, log analyzer and other GUI-based clients in target environments.
Simple to Use - The existing TPTP tools were conceived as samples, rather than as exemplary, they are deficient in many areas of usability and in some cases lacking in function. The plan is that within the domains which they target they will provide a high-quality user experience out of the box. We will focus on ease of use through enhanced user documentation, tutorials, white papers, demonstrations, and a wide range of enhancements to the user interface to streamline basic processes and clarify concepts and terminology. We are focused on improving as much as possible in Release 4.3, and expect need for continuing this focus beyond 4.3.
Enable Consistent Multi-language Support - In TPTP a significant effort will be applied in extending coverage of the trace models to represent C/C++ programs and to handle protocol activity (specifically HTTP) consistently with program activity. There will also be C/C++ APIs provided to the data collection and control layers.
Appealing to the Broader Community - A range of initiatives will be taken to broaden the community of potential and actual users of TPTP. Technically this will include additional integration of open source test tool technologies based on JUnit, and the various hooks to JUnit in the JDT, more data collection agents – particularly focusing on open source technologies, and additional operating system and hardware platforms from which data can be collected. There will be additional marketing and an extensive outreach program to the Eclipse community for additional contribution and adoption.
The TPTP top-level project is is comprised of four projects, managed in a coordinated fashion, across which the plans items are allocated. TPTP projects include:
TPTP Platform Project - Provides common infrastructure in the areas of user interface, EMF based data models, data collection and communications control, as well as remote execution environments. Additionally, the Platform provides extension points for leveraging or extending these capabilities in solution specific tooling or runtimes. This includes Eclipse workbench plug-ins as well as runtime plug-ins on a target and optionally remote system.
Testing Tools Project - Provides specializations of the TPTP Platform for testing (e.g. test editors, trace/test conversion support), and exemplary extensible tools for specific testing environments. Initially this includes three test environments: JUnit, manual, and URL testing. These specializations provide optimized editing and reporting experiences for these use cases. In the cases where a unique runtime or an implementation of a testability interface is required, it is also developed in the project. For example, the manual test execution environment provides a remotely managed user interface specifically for collecting manual test progress. This manual user interface is unique from the common execution environment for JUnit and URL testing. .
Tracing & Profiling Tools Project - Extends the TPTP Platform with specific data collection for Java and distributed applications that populate the common trace model, additional language and protocol support is anticipated. There are also viewers and analysis services that draw data from the common trace model. Capabilities are provided to collect and analyze heap and stack information as well as generic toolkits for instrumenting running applications..
Monitoring Tools Project - Extends the TPTP Platform for collecting, analyzing, aggregating, and visualizing data that can be captured in the log and statistical models. The typical examples are the collection of system or application resources such as CPU or memory utilization and support for the viewing, aggregation, and analysis of that data. Logs can also be transformed into a common format and model allowing for symptom and pattern analysis. The correlation of the data in these models is of particular interest when it is associated with other model instances of statistical or log data as well as traces and tests..
Plan items targeted for this release represent the addition of new features or areas where existing features will be significantly reworked or enhanced. Plan items are allocated to themes and projects indicated above.
TPTP Platform Project Plan Items
|In Plan||Support existing RAC clients and agents on the Linux IA32 platform using the new Agent Controller technology through compatibility layers(125103).|
|In Plan||TPTP profiler (based on JVMTI) will be further improved in this release cycle with a heap profiler (147653) thread profiler (147654), and improved performance summary information (147658)|
|In Plan||This requirement is to make the Log Analyzer UI format-agnostic when displaying Common Base Events as the event format will change and evolve over time. Any view, dialog, or editor that currently displays Common Base Events should display the events in a consistent manner with consistent naming. 71682|
|In Plan||Provide the Log Analyzer functionality available in TPTP as an RCP application. The TPTP Log Analyzer will be restructured so that it can be packaged as a Rich Client Platform application. The RCP application will provide log visualization, analysis and correlation. 134531|
|In Plan||The TPTP log analyzer provides a set of useful services to analyze log files. Most of these services are tightly tied to the log and trace analyzer user interface. It is required to decouple these services from the UI so that consuming products can reuse the above functionality. This enhancement is used to define a set of interfaces to expose the log analysis and correlation services available in the TPTP Log Analyzer function. 143766|
||We are investing in automating post-build steps in deploying and executing regression tests. This has been dire need for the project and should positively impact quality and productivity. Automate the deployment and set up of Agent Controller on all platforms (144958)|
|Helpwanted||This enhancement will improve the usability of dynamic probekit feature by automatically deploying the support class files that contain byte code for the user defined probes (143251)|
|Helpwanted||Web document for creating custom profilers with Probekit 147450|
|Helpwanted||Explore design approaches to improve scalability and responsiveness of EMF-based data models for trace and test tools.[Trace Model 147656] [Test Model 144950]|
|Helpwanted||Provide capability in the agent framework to dynamically determine if the data collection services are available to an agent. The data collection services include agent controller and file transfer service. 142742|
TPTP Testing Tools Project Plan Items
|In Plan||Improve JUnit integration between TPTP and JDT. Allow the JUnit user to have the same experience, once TPTP is installed, as he/she used to have in Eclipse SDK, with the possibility of executing his/her JUnit tests in the TPTP framework. Any JUnit test would be edited and created as usual, and still it would be integrated with other types of TPTP tests, and could be executed in the TPTP framework. (90628) [Theme: Appealing to the Broader Community, Simple to Use]|
|In Plan||Provide a generic Java API recorder Based on the generic recording facility provided in TPTP v4.1+ (see enhancement #74926), provide a generic recorder to record Java API invocations to create a test. The generic recorder could leverage the TPTP Probe Kit to instrument target APIs to capture invocations based on predefined criteria. (119688) [Theme: Design for Extensibility: Be a Better Platform]|
|In Plan||Provide the concept of an object mine inside each test suite The object mine will keep track of a single id of an object irregardless of the number of times it occurs in the test cases owned by the test suite. The user will also be able to include external object mines (i.e. object mines owned by other test suites) and contribute any newly detected objects to an external object mine. (144763) [Theme: Appealing to the Broader Community, Simple to Use]|
|Helpwanted||Providing better means for the user to specify a workspace location An extra page specific to the auto GUI test suite should be included in the launch configuration dialog to allow the user to specify a workspace and the plug-ins that they wish to include in the launched workbench. (109880) [Theme: Appealing to the Broader Community, Simple to Use]|
|Helpwanted||Automated Documentation Generation The feature being proposed it to utilize the GUItester to create SIMPLE documentation (in HTML format, or another generic output of the users choosing) that outlines the (1) name of the UI test; (2) the series of steps taken in the UI test; (3) screenshots at every possible step during the test. (110108) [Theme: Appealing to the Broader Community, Simple to Use]|
|Helpwanted||Support RCP applications for recording and playing back GUI test cases A common feature that users ask for is to have support in recording and playing back test cases in an RCP application. (114159) [Theme: Rich Client Platform]|
|Helpwanted||Generic Recorder Framework improvements. Support for launching and controlling two or more recorders or recording agents in parallel. Support for launching Java applications for recording without creating/using an Eclipse launch configuration. Support remote recording via the Agent Controller. Provide an extension point for user-defined annotations to a recording session. Support (e.g. model loading) for recorders to emit standard trace model events. (145146) [Theme: Appealing to the Broader Community, Simple to Use]|
|Helpwanted||The test launch configuration needs to have the means for users to specify JVM argument. Support for a standard way for users to specify JVM arguments for test execution. Provide an UI for the user to specify zero or more JVM arguments and an API for JUnit-based test runners to access these JVM arguments for launching the Java test process. (143121) [Theme: Design for Extensibility: Be a Better Platform]|
TPTP Tracing And Profiling Tools Project Plan Items
|In Plan||Providing better analysis views for analyzing execution statistics The following enhancements should make analysis of execution statistics easier: Easily identify the method that has had the maximum base/cumulative time. Quick hover explanation in the view or better naming for the base or cumulative time. The method statistics view should simply should show the stack for a particular method invocation. (147930) [Theme: Appealing to the Broader Community, Simple to Use]|
|In Plan||Provide better abstraction and navigation means in the Trace views The following feature tries to outline some improvements that may help to abstract and navigate the profiling data: Provide a find operation under the execution time statistics, memory statistics, and code coverage view. Ability to bring deltas to the top or only showing deltas between refreshes Showing what’s changed when deltas are displayed Doing post filtration in the views Doing pagination in the tabular views Filtering methods that have a base/cumulative time that is less than a given threshold (147931) [Theme: Appealing to the Broader Community, Simple to Use]|
|In Plan||Using virtualization in stat views Virtualization can be used to reduce the amount of time required to show a large number of tree items. This can significantly help improve the performance of the execution stat view when it is viewed at the method level. (147933) [Theme: Scaling Up]|
|In Plan||Create Extension Point for ARM Factories Create a new extension point to allow user to define there own Application Resource Measurement Factories. The extension point would include transaction, metric, and reporting. (148048) [Theme: Design for Extensibility: Be a Better Platform]|
|Helpwanted||Trace-related views should make use of view linking service Currently some views listen for selection events on other views and make use of this information to update their own selections to something related to the original selection. For example, if you have the log view and log interactions view open at the same time, and select a log record in one of them, the other view will automatically update and select that same log record, if it's there. All the views should use a standard view linking service to do this. (87605) [Theme: Design for Extensibility:Be a Better Platform]|
|Helpwanted||Providing better analysis views for detecting memory leaks The following enhancements need to be made to make memory leak detection easier; Objects that are garbage collected need to be removed from the object reference view. Integrate the object reference view with the memory statistics view. Provide a dynamic chart that is updated when the heap size changes Enhanced look & feel of the object reference view. (147929) [Theme: Appealing to the Broader Community, Simple to Use]|
|Helpwanted||Add a thread analysis and a call graph view A thread analysis view will allow users to resolve deadlock problems, thread starvation or other thread related issues. A call graph view will help to easily visualize application flow and critical paths that maybe the cause of a bottleneck. (147932) [Theme: Appealing to the Broader Community, Simple to Use]|
TPTP Monitoring Tools Project Plan Items
|In Plan||Improve usability of Stand-alone Generic Log Adapter package. Applications can package and make use of GLA run-time and adapter configuration files to parse log files. However, to use the packaged adapter files they must be modified. For example, the adapter configuration must specify the location and name of the log file to parse. Currently, the only way to do this programmatically is by using XML parsing libraries to read and modify these XML files. To simplify adapter configuration modification, this feature will extend the current org.eclipse.hyades.logging.adapter.Adapter API to provide methods to retrieve and modify the configuration before it is executed by GLA. Also, to allow the packaged adapter configuration files to be used more easily in various applications, multiple outputters will be added to them, which can them be enabled or disabled depending on the application. (96433 and 126653) [Theme: Design for Extensibility: Be a Better Platform]|
|In Plan||Provide sensor/extractor filtering capability in TPTP Generic Log Adapter log parsers. In TPTP 4.2 support was added to the Generic Log Adapter run-time for filtering at the sensor and extractor component level. This filtering capability will be exploited in the current TPTP provided log parsers to allow users to improve the performance of log parsing when only a subset of the log records is of interest. (141840) [Theme: Scaling Up]|
|In Plan||Provide Rich Client Platform versions of Log Analysis tooling. Some users want stand-alone tools for log analysis. The TPTP Symptom Editor will be restructured so that it can be packaged as a Rich Client Platorm application. Also, some portions of the Log Analyzer will be restructured so it can be packaged as a Rich Client Platform application. (134532 and 134798) [Theme: Rich Client Platform]|
|In Plan||Improve the Managed Resource Explorer. In TPTP 4.2 a technical preview was introduced for a generic Managed Resource Explorer. This tool will be improved in 4.3 so that it can become a full TPTP component. (145056, 145057, 145058, 150259, 150385) [Theme: Design for Extensibility: Be a Better Platform]|
|In Plan||Add tooling to simplify creation of WSDM managed agents. In TPTP 4.2, a Managed Resource Explorer was added that allowed the developer to manipulate resources that sit behind a manageability endpoint. The Managed Agent explorer has plugins for JMX and WSDM. This feature for 4.3 would allow a developer to easily create WSDM based managed agents (referred to in WSDM as manageability endpoints) that are compliant with the WSDM 1.1 specification from OASIS. (142543) [Theme: Design for Extensibility: Be a Better Platform]|
|In Plan||Improve the Monitoring Instrumentation tooling. In TPTP 4.2 a technical preview was introduced for tooling to instrument existing Java applications for monitoring. This support will be improved in 4.3 so that it can become a full TPTP component. ( 146272, 148042, 148044) [Theme: Design for Extensibility: Be a Better Platform]|
In addition to the targeted features for this release, we plan to reduce the defect backlog. Defects are prioritized based on severity, age, and resource availability. We encourage users to report defects and we will do our best to fix them in priority order. The goal is clear backlog of major/critical/blocker defects and make reasonable progress on fixing as many as possible.
See TPTP 4.3 Defects for a listing of already fixed defects, current defect targets and backlog.
Select "4.3 [Completed | nil| I1 | I2 | I3] bugs" tabs.