Eclipse Test & Performance Tools Platform (TPTP) Project
4.4 Plan (Approved)
Last revised 03/20/2007 01:30 AMEastern
Time (
marks interesting changes
since March 15 2007)
Please send comments about this plan to the tptp-pmc@eclipse.org PMC mailing
list.
This document lays out the feature and API set for the TPTP 4.4 release.
- Release
Deliverables
- Release
Milestones
- Target Operating Environments
- Compatibility
with Previous Releases
- Themes
- Projects
- Features
- Defects
The first part of this plan deals with the important matters of release deliverables, release milestones, target operating environments, and release-to-release compatibility. These are all things that need to be clear for any release, even if no features were to change.
The remainder of the plan consists of plan items for the four projects under the TPTP Top-Level Project. Each plan item covers a feature or API that is to be added to TPTP, or some aspect of TPTP that is to be improved. Each plan item has its own entry in the TPTP bugzilla database, with a title and a concise summary (usually a single paragraph) that explains the work item at a suitably high enough level so that everyone can readily understand what the work item is without having to understand the nitty-gritty detail.
Not all plan items represent the same amount of work; some may be quite large, others, quite small. Some plan items may involve work that is localized to a single component; others may involve coordinated changes to several components; other may pervade the entire project.
With the previous release as the starting point, this is the plan for how we will enhance and improve it. Fixing bugs, improving test coverage, documentation, examples, performance tuning, usability, etc. are considered routine ongoing maintenance activities and are not included in this plan unless they would also involve a significant change to the API or feature set, or involve a significant amount of work. The intent of the plan is to account for all interesting feature work.
Release Deliverables
The following release deliverables are provided:
- Runtime
- Source
- Examples
- Component Tests
- Data Collection Engine for
Windows (
- Data Collection Engine for
Windows (XP, Server 2003) 32-bit (x86)/64-bit (x86_64)Runtime
- Data Collection Engine for
Windows Server 2003 Itanium Runtime
- Data Collection Engine for
Linux 32-bit (x86)/64-bit (x86_64)Runtime
- Data Collection Engine for
Linux Itanium Runtime
- Native Logging Implementation
(All platforms)
- Plugin Translatability Log
It is important to notice that this list of platforms has been significantly reduced from previous releases. Based on consumer feedback, several platforms have been stabilized at the TPTP 4.3 level, and will remain there with no enhancement plans until there is a larger community demand backed by committer resources.
Release Milestones
The TPTP 4.4 release is targeted for general availability on
28-June-2007. All release deliverables will be available for download as
soon as the release has been tested and validated in the target operating
configurations. There is a significant allocation of resource to defect
removal in all iterations. The first 2 iterations will also contain the bulk of
any enhancement work. Iteration #3 and beyond are used to stabilize and shut
down the release. Every iteration will provide a
milestone driver to the Europa release stream.
|
Release Milestones |
||
|
Milestone |
Date |
Description |
|
Iteration 1 |
Fri 02/23/2007 |
Stable build - enhancements and
bug fixes |
|
Iteration 2 |
Fri 04/06/2007 |
Stable build - enhancements and
bug fixes |
|
Iteration 3 |
Fri 05/18/2007 |
Stable build |
|
Iteration 4 |
Fri 06/15/2007 |
shut down and General Availability
June 29 2007 |
For a detailed development schedule of TPTP 4.4 release, click here.
Target Operating Environments
In order to remain current, each TPTP release targets reasonably current versions of the underlying operating environments.
- Java runtime (JRE) or Java
development kit (JDK) 1.4 and 1.5
- Eclipse SDK 3.3
for Linux (GTK) , Linux (Motif), or Windows
- Eclipse Modeling Framework
(EMF) SDK 2.2
Note 2.3 is required if using WTP. TPTP 4.4 is compatible with both.
- XML Schema Infoset Model (XSD)
SDK 2.2Note
2.3 is required if using WTP. TPTP 4.4 is compatible with both.
Most of the TPTP SDK is "pure" Java™ code and has no direct dependence on the underlying operating system. The chief dependence is therefore on the Java 2 Platform itself. The TPTP 4.4 release is written and compiled against version 1.4 of the Java 2 Platform APIs, and targeted to run on version 1.4 or newer of the Java 2 Runtime Environment, Standard Edition.
There are many different implementations of the Java 2 Platform running atop a variety of operating systems. We focus TPTP testing on a handful of popular combinations of operating system and Java 2 Platform; these are our reference platforms. TPTP undoubtedly runs fine in many operating environments beyond the reference platforms we test. However, since we do not systematically test them we cannot vouch for them. Problems encountered when running TPTP on non-reference platforms that cannot be recreated on any reference platform will be given lower priority than problems with running TPTP on a reference platform.
TPTP SDK 4.4 is tested and validated on the following target reference platforms (this list may be updated over the course of the release cycle):
|
TPTP
Agent Controller Reference Platforms |
|
|
Processor architecture |
Operating system |
|
Intel IA32 |
Red Hat Linux v7.1, v7.2, v7.3,
v8.0 |
|
Intel IA32 |
Red Hat Linux Advanced Server v2.1 |
|
Intel IA32 |
SuSE Linux v7.2, v7.3 |
|
Intel IA32 |
SuSE Linux Enterprise Server
(SLES) v7, v8 |
|
Intel IA32 |
Windows |
|
Intel IA32 |
Windows Server 2003 |
|
Intel IA32 |
Windows XP Professional |
|
Intel EM64T |
Red Hat Linux Advanced Server v3 |
|
Intel EM64T |
Windows 2003 Server (service pack
1) |
|
Intel IPF |
Red Hat Advanced Server v3 |
|
Intel IPF |
Windows 2003 Server (service pack
1) |
Although untested, TPTP should work fine on other OSes that support the same operating system kernel and version.
Internationalization
TPTP is designed as the basis for internationalized products. The user interface elements provided by the TPTP SDK components, including dialogs and error messages, are externalized. The English strings are provided as the default resource bundles.
Latin-1 locales are supported by the TPTP SDK on all of the above operating environments; DBCS locales are supported by the TPTP SDK on the Windows, GTK, and Motif window systems; BIDI locales are supported by the TPTP SDK only on Windows operating environments.
The TPTP SDK supports GB 18030, the new Chinese code page standard, on Windows XP and Linux.
TPTP supports ICU4J starting in 4.2 release. This will significantly increase the number of supportable locales. Products needing to localize to newer locales are enabled. German, Traditional Chinese, and Arabic are tested.
Note that although TPTP support all these languages, only the English versions of the strings are provided.
Compatibility with Previous Releases
TPTP 4.4 will be compatible with TPTP 4.3. The following specifies details of the various aspects of release compatibility.
· API Contract Compatibility: Refer to Evolving Java-based APIs for a discussion of the kinds of API changes that maintain contract compatibility.TPTP SDK 4.4 will be upwards contract-compatible with TPTP SDK 4.3 or lower. Downward contract compatibility is not supported. There is no guarantee that compliance with TPTP SDK 4.4 APIs would ensure compliance with TPTP SDK 4.0 APIs.
· Binary (plug-in) Compatibility: TPTP SDK 4.4 will be upwards binary-compatible with TPTP SDK 4.3. Downward plug-in compatibility is not supported. Plug-ins for TPTP SDK 4.4 will not be usable in TPTP SDK 4.3. Refer to Evolving Java-based APIs for a discussion of the kinds of API changes that maintain binary compatibility.
· Source Compatibility: TPTP SDK 4.4 will be upwards source-compatible with TPTP SDK 4.3. This means that source files written to use TPTP SDK 4.3 APIs might successfully compile and run against TPTP SDK 4.4 APIs, although this is not guaranteed. Downward source compatibility is not supported. If source files use new TPTP SDK APIs, they will not be usable with an earlier version of the TPTP SDK.
· Workspace Compatibility: TPTP SDK 4.4 will be upwards workspace-compatible with TPTP SDK 4.3 unless noted. This means that workspaces and projects created with TPTP SDK 4.3 can be successfully opened by TPTP SDK 4.4 and upgraded to a 4.4 workspace. This includes both hidden metadata, which is localized to a particular workspace, as well as metadata files found within a workspace project (e.g., the .project file), which may propagate between workspaces via file copying or team repositories. Downward workspace compatibility is not supported. A workspace created (or opened) by a product based on TPTP 4.3 will be unusable with a product based an earlier version of TPTP. Visible metadata files created (or overwritten) by TPTP 4.4 will generally be unusable with earlier versions of TPTP.
- Non-compliant usage of API's: All non-API methods and
classes, and certainly everything in a package with "internal"
in its name, are considered implementation details which may vary between
operating environment and are subject to change without notice. Client
plug-ins that directly depend on anything other than what is specified in
the TPTP SDK API are inherently unsupportable and receive no guarantees
about compatibility within a single release much less with an earlier
releases. Refer to How
to Use the Eclipse API for information about how to write
compliant plug-ins.
Themes
The TPTP PMC adopted and specialized the following Eclipse themes which represent the key focus areas for TPTP enhancements in the year ahead.
· Scaling Up - - TPTP continues to enhance the support of large data volumes and processing rates in areas such as data collection, user interface and in the persistence of trace, log and statistical models and execution histories. In 4.4 the test execution logs are the focus.
·
· Design for Extensibility: Be a Better Platform - There will be a wide range of activities within TPTP to externalize APIs and define extension points, making the infrastructure more flexible and more generic in application. A good example of this is integration of TPTP with WTP and BIRT for web application testing, profiling and generation of customized reports of results. In the 4.4 release all dependencies on "negotiated" api were removed unless the needed api was promoted to public status.
· Embedded Development - TPTP target execution environment and remote data collection framework provide capabilities that are adapted for high-end embedded systems. TPTP will seek contributions to add support for embedded systems. We are promoting use of TPTP native logging capabilities on a number of embedded target systems.
· Rich Client Platform - TPTP will use RCP for building manual test client, log analyzer and other GUI-based clients in target environments. TPTP is looking for contributions to add RCP support to the UI testing tools.
· Simple to Use - The existing TPTP tools were conceived as samples, rather than as exemplary, they are deficient in many areas of usability and in some cases lacking in function. The plan is that within the domains which they target they will provide a high-quality user experience out of the box. We will focus on ease of use through enhanced user documentation, tutorials, white papers, demonstrations, and a wide range of enhancements to the user interface to streamline basic processes and clarify concepts and terminology. We are focused on improving as much as possible in Release 4.4, and expect need for continuing this focus beyond 4.4. Specifically in 4.4 the Java profiler infrastructure has been completely replaced along with an upgrade to the JVM TI interface.
· Enable Consistent Multi-language Support - In TPTP a significant effort will be applied in extending coverage of the trace models to represent C/C++ programs and to handle protocol activity (specifically HTTP) consistently with program activity. There will also be C/C++ APIs provided to the data collection and control layers.
· Appealing to the Broader Community - A range of initiatives will be taken to broaden the community of potential and actual users of TPTP. Technically this will include additional integration of open source test tool technologies based on JUnit, and the various hooks to JUnit in the JDT, more data collection agents – particularly focusing on open source technologies, and additional operating system and hardware platforms from which data can be collected. There will be additional marketing and an extensive outreach program to the Eclipse community for additional contribution and adoption.
Projects
The TPTP top-level project is comprised of four projects, managed in a coordinated fashion, across which the plans items are allocated. TPTP projects include:
· TPTP Platform Project - Provides common infrastructure in the areas of user interface, EMF based data models, data collection and communications control, as well as remote execution environments. Additionally, the Platform provides extension points for leveraging or extending these capabilities in solution specific tooling or runtimes. This includes Eclipse workbench plug-ins as well as runtime plug-ins on a target and optionally remote system.
· Testing Tools Project - Provides specializations of the TPTP Platform for testing (e.g. test editors, trace/test conversion support), and exemplary extensible tools for specific testing environments. Initially this includes three test environments: JUnit, manual, and URL testing. These specializations provide optimized editing and reporting experiences for these use cases. In the cases where a unique runtime or an implementation of a testability interface is required, it is also developed in the project. For example, the manual test execution environment provides a remotely managed user interface specifically for collecting manual test progress. This manual user interface is unique from the common execution environment for JUnit and URL testing. .
· Tracing & Profiling Tools Project - Extends the TPTP Platform with specific data collection for Java and distributed applications that populate the common trace model, additional language and protocol support is anticipated. There are also viewers and analysis services that draw data from the common trace model. Capabilities are provided to collect and analyze heap and stack information as well as generic toolkits for instrumenting running applications.
· Monitoring Tools Project - Extends the TPTP Platform for collecting, analyzing, aggregating, and visualizing data that can be captured in the log and statistical models. The typical examples are the collection of system or application resources such as CPU or memory utilization and support for the viewing, aggregation, and analysis of that data. Logs can also be transformed into a common format and model allowing for symptom and pattern analysis. The correlation of the data in these models is of particular interest when it is associated with other model instances of statistical or log data as well as traces and tests.
Features
Plan items targeted for this release represent the addition of new features
or areas where existing features will be significantly reworked or
enhanced. Plan items are allocated to themes and projects indicated
above. Even though the 4.4 release has a top priority theme to address defect
backlogs, there are several critical requested enhancements being enabled.
|
TPTP
Platform Project Plan Items |
|
|
Status |
Description |
|
In Plan |
In general TPTP 4.4 will move all agent
control support to the newer code base. This includes standalone agent
controller processes as well as the Integrated Agent Controller. Only subsets
of the platforms historically supported by TPTP are being carried forward in
4.4. All other platforms are now stabilized in the 4.3 release stream.
Several enhancements are being used to track this work. (108584). (121658). (125103). (173330). |
|
In Plan |
TPTP profiler (based on JVMTI)
will be the prime Java profiler in 4.4. The previous PI based profiler will
be maintained for Java 1.4 and older support. The following enhancements are
being used to track this work. (165579) (167356) (167994) (167997) (168375) (170207) (174574) (170208) |
|
In Plan |
In the continued effort to move to
an out of memory model structure in the TPTP platform, this release will
enhance the test execution history persistence model with RDB backed storage.
This continues on the pattern established for log files and the large log
support provided in previous releases. 169353 |
|
In Plan |
In order to make the new RCP based
Log Analyzer extensible some additional provisional API has been requested. 163144 |
|
In Plan |
TPTP 4.4 will support |
|
In plan |
In addition to these other
significant enhancements we also intend to refine the static BIRT reports
provided in TPTP as well as add additional client support for handling
standard err/out. 160342 166596 |
|
In Plan |
With the move to a TI based JVM connection
the BCI engine of Probekit needs to be moved to that infrastructure. This
enhancement is being used to track that work. 141540 |
|
In Plan |
As a continual effort to automate and
self host, in 4.4 TPTP continues to replace manual tests with automation, and
will automate the creation of reports. 147907 174643 |
|
TPTP
Testing Tools Project Plan Items |
|
|
Status |
Description |
|
In Plan |
The only enhancement to the
testing project is the additional support for RDB backed execution histories
and that work is managed as part of the platform enhancement since that is
where the bulk of the work is. 169353 |
|
TPTP
Tracing And Profiling Tools Project Plan Items |
|
|
Status |
Description |
|
In Plan |
As part of the move to JVMTI as
the technology supporting our enhanced profiler there is addition support for
Heap analysis. (167995) |
|
Help Wanted |
ARM as a data collection service
provides much more that correlated trace events as currently supported and
exploited in TPTP. There are also event for statistical data reporting and
this request is for the enabling of such support. (166692) |
|
TPTP
Monitoring Tools Project Plan Items |
|
|
Status |
Description |
|
In Plan |
A series of enhancements have been
requested to improve the tech preview support in the TPTP WSDM tooling. This
includes usability changes, updating of runtime dependencies and improvements
to the WSDL, schema and meta data support. These enhancements are being track
with the following bugzilla entries. (150385, 165544, 165546, 165568) |
|
In Plan |
The managed resource explorer can
be populated with a large number of resources. Filtering and sorting support
will make this more usable by users. 165565 |
|
In Plan |
There is a need to make the log
navigator extensible to allow custom configuration per log type/source. An
example is to allow a consumer to extend for rlog support. 156527 |
|
In Plan |
The tech preview support for probe
based instrumentation for JMX, ARM and logging has been a well received
capability by end users. With the community validation of the function, this
support is being moved to the GA stream of TPTP. 159634 |
|
In Plan |
A series of enhancements have been
requested to improve the monitoring instrumentation support in TPTP. This
includes adding support for launching applications instrumented with multiple
instrumentation types, support for adding instrumentation by directly editing
the source code, and support for enabling the JMX monitoring notification
service in applications instrumented with JMX. These enhancements are being
track with the following bugzilla entries. (146267, 148044, 157215) |
|
In Plan |
Add tooling for instrumenting .NET
applications with ARM and logging. This tooling will be provided as a tech
preview. 158063
|
Defects
In addition to the targeted features for this release, we plan to reduce the defect backlog. Defects are prioritized based on severity, age, and resource availability. We encourage users to report defects and we will do our best to fix them in priority order. The goal is clear backlog of major/critical/blocker defects and make reasonable progress on fixing as many as possible.
See TPTP 4.4 Defects for a listing of already fixed defects, current defect targets and backlog.
Select "4.4 [Completed | nil| I1 | I2 | I3] bugs" tabs.