Project FAQs
| eclipse test & performance tools platform project Project FAQs |
|
This project proposal is in the Proposal Phase and is posted here to solicit additional project participation and ways the project can be leveraged from the Eclipse membership-at-large. You are invited to comment on and/or join the project. Please send all feedback to the eclipse.test-and-performance newsgroup or the test-and-performance-proposal mailing list.
| Frequently Asked Questions | |
|
The following is the official Eclipse Test & Performance Tools Platform Project Announcement FAQ. Last Modified: July 23, 2004 |
|
| Table of contents: |
| Overview: |  top
|
The Eclipse Foundation is proposing a new Top-Level Project to develop a Test and Performance Tools Platform: an open development platform supplying frameworks and services for test and performance tools that are used throughout the software lifecycle (e.g., testing, tracing/profiling, tuning, logging, monitoring, analysis, autonomics, administration, etc.)
Intel Corporation has also offered to lead the project and is making a
major resource contribution to the Eclipse Foundation and to the development
of software within the project
The Test & Performance Tools Platform Project is based on and extends
the Eclipse community's Hyades Project that was created in late 2002.
The Test & Performance Tools Platform Project is managed by a Project
Management Committee (PMC). The work of the Top Level Project is organized
into Projects, each with its own Project Lead who is responsible for its
success.
The scope of the original Hyades project has been partitioned into four
new projects: The Hyades Core Platform, provides core elements and interoperability,
and three others which leverage that infrastructure and relate to specific
types of activity: Hyades Testing, Hyades Tracing and Profiling, and Hyades
Monitoring. Additional subprojects may be added, and new features and requirements
are being added to each of the four projects. Test & Performance was
chosen as a descriptive name for the top-level project, and its scope leaves
open the possibility of additional projects being created which do not directly
leverage the Hyades Core Platform.
The project provides an open development platform supplying frameworks and services for test and performance tools that are used throughout the software lifecycle (e.g., testing, tracing/profiling, tuning, logging, monitoring, analysis, autonomics, administration, etc.) Included in this platform are data collection services and user interfaces for tracing, test case execution, logging and statistical behavior in local or remote execution environments.
In most cases such tools, even if they are sold by the same company, were developed independently. This means that the interoperability is retro-fitted, and tends not to operate at a particularly fine-grained level either for user interface or for the data assets the tools work with (tests, traces etc.). By providing a common infrastructure upon which tools can be built, where possible leveraging existing standards, the Test and Performance Project can drive interoperability into existing tooling, and allow new tooling to interoperate from inception, significantly enhancing the experience of users of the tools. In addition, by allowing flexible combinations of asset type and user interface, the project may offer the possibility of vendors (or even the project itself) providing brand new types of tools function.
The integration with the Eclipse IDE allows test and performance to be
driven early in the software lifecycle, enhancing the quality of software
being developed inside Eclipse, and as Eclipse expands into Rich Clients
and additional new projects, the Test & Performance project will ensure
these applications can easily be subjected to effective testing and performance
analysis.
In an open source model, the cost of building the common infrastructure
is shared amongst the participants in the project, and the diversity of
intellectual inputs amongst Open Source Committers in open collaboration
with their peers ensures that it is naturally based on Open Standards and
can be best of breed. The focus on a shared infrastructure means the incremental
cost of adding new features or supporting new platforms and environments
is small, so the project itself can be nimble, and vendors leveraging the
framework can accelerate the rate at which they add value to their own tools,
thereby moving the overall Eclipse Ecosystem forward.
The following organizations are currently committed to participating in
the development of the project:
The Test and Performance Tools Platform is concerned with the performance
of systems in general, although some special support is provided for systems
built in Eclipse and for the Eclipse workbench itself. For example, the
tests used to validate the Hyades infrastructure and tooling are stored
in the Eclipse.org CVS repository in ways that can be managed and executed
by tools that Hyades itself supplies. At this stage, however, the definition
and testing of Eclipse builds, and certification of plug-ins against those
builds, is not being addressed by the project.
Yes. The project leverages core Eclipse subsystems such as SWT for user
interfaces and EMF for data persistence, but there are elements of the Hyades
infrastructure which operate in target environments which can be remote
to the workbench, some of which are written in native code, rather than
Java. User interfaces can be built as workbench clients, rich clients, and
in some cases the infrastructure can operate without a user interface.
As is true in general for Eclipse, the project builds both infrastructure
and Exemplary Exensible Tools. The open source distribution contains tools
that will cover much of the basic activities of tracing, profiling, monitoring
and testing, although this will probably primarily apply to small-scale
systems, open source systems, and development environments rather than production
systems. Individual vendors (both contributing Eclipse members, non-contributing
members and non-members) may provide additional functionality through interfacing
to the infrastructure, extending sample tools etc. They will also tend to
provide support for the combined offering under commercial licensing arrangements.
The user interface components of the Hyades Infrastructure can run on any
Eclipse-supported platform. The various test execution, monitoring and tracing
components typically run on a broad range of infrastructure and interface
with a broad range of systems. One notable deficiency at present is that
the various trace facilities are currently Java-specific, although the project
has a goal to extend additional language support in due course. For the
other agents there is a reasonable coverage implemented or planned for popular
operating systems, application servers, web servers etc, with a particular
focus on providing support for other open source products.
The key Test and Performance Tools Platform capabilities for the Eclipse
release 3.0 we delivered as part of the Hyades 3.0 release and are:
The Test and Performance Tools Platform Project Requirements, Architecture, and Planning Groups guides our project. Each of these groups are aligned with their respective Eclipse Foundation Committees.
Requirements are gathered from Eclipse member companies, research organizations,
independent projects and projects hosted by the Eclipse Foundation. Feedback
from these groups and investment in our meritocracy help prioritize implementation
of the new technology in the Test and Performance Tools Platform as we take
a generic view of how the frameworks could be reused. Ultimately, however,
contributing organizations and individuals need to make long-term commitments
to developing, documenting, testing and enhancing elements of the infrastructure,
and new contributors are always welcome.
The projects that are hosted by Eclipse make all distributions available
under licenses that are certified by the Open
Software Initiative. Test and Performance Tools Platform Project distributions
are made available under the Eclipse
Public License. It makes the Test and Performance Tools Platform available
for commercial re-distribution without royalty or the obligation to donate
software back to Eclipse. See the license for more detail.
The Hyades Core Platform project covers the common infrastructure in the areas of user interface; EMF based data models, data collection and communications control, as well as remote execution environments. In addition, the platform provides the extension points for leveraging or extending the common infrastructure in solution specific tooling or runtime. This includes Eclipse workbench plugins as well as runtime plugins on a target and optionally remote system.
The Hyades Testing project provides specializations of the platform for testing (e.g. test editors, trace/test conversion support), and exemplary extensible tools for specific testing environments. Initially this includes 3 test environments: JUnit, manual and URL testing. These specializations provide optimized editing and reporting experiences for these use cases.
The Hyades Tracing and Profiling project extends the platform with specific data collection for Java and distributed applications that populate the common trace model, additional language and protocol support is anticipated. There are also viewers and analysis services that draw data from the common trace model. Capabilities are provided to collect and analyze both heap and stack information as well as generic toolkits for instrumenting running applications.
The Hyades Monitoring project collects, analyzes, aggregates and visualizes
data that can be captured in the log and statistical models. The typical
examples are the collection of system or application resources such as CPU
or memory utilization and support for the viewing ,aggregation and analysis
of that data. Logs can also be transformed into the common format and model
allowing for symptom and pattern analysis. The correlation of the data in
these models is of particular interest when it is associated with other
model instances of statistical or log data as well as traces and tests.
The Data Models are abstract descriptions in UML of the types of assets
(tests, traces, logs etc.) that the project deals with, and they are provided
with a concrete implementation through the Eclipse Modeling Framework (EMF).
This means that tools inside the Eclipse Workbench can manipulate objects
which correspond to those abstract descriptions, which are managed and persisted
inside the eclipse asset model and can be stored in projects, subjected
to version control, etc. When they actually hit a filesystem they are in
an open XMI format, but individual tools vendors don't have to go through
the effort of manually constructing code to map that XMI into and out of
an object format they can usefully manipulate, they merely speak to the
EMF-generated API. Furthermore if two tools both speak to that API they
can interoperate through shared use of the stored assets inside the asset
management model provided by Eclipse.
The Test data model is an implementation of the UML 2 Test Profile (U2TP)
defined by the Object Management Group. The Log Model is derived directly
from a standard currently proposed at OASIS known as Common Base Event.
The Trace Model is ad-hoc but maps quite closely to the data formats of
JVMPI and JVMTI. The statistical model is also ad-hoc, but maps well onto
the formats used by JMX and the Microsoft PerfMon counters.
The Test and Performance Top-Level Project Proposal and associated documents
are at /proposals/eclipse-tptp/index.html
The Hyades Project Website is at /Hyades
until it merges into the proposed new structure.