Runtime Analysis Tools (RAT)

The project has been created.

Runtime Analysis Tools (RAT)

Introduction

The Runtime Analysis Tools is a proposed open source project under the Eclipse Tools Project. This proposal is in the Project Proposal Phase (as defined in the Eclipse Development Process document) and is written to declare its intent and scope. The proposal is written to solicit additional participation and input from the Eclipse community. You are invited to comment on and/or join the project.

Description

The Runtime Analysis Tools is a tool for performance analysis and finding CPU/Allocation and other bottlenecks in Java applications. It is useful to any Java software engineer. An application or more complex application systems like an Eclipse Installation or an application server can be closely analyzed. The analysis may be performed online or offline basing on existing CPU and Memory snapshots. Such snapshots may be written on termination of the Java Virtual Machine or any arbitrary point in time through profiler agent.

Background

The motivation for this project is to provide a tool to analyze performance failures of critical development and production systems that suffer from performance problems. These problems are often hard to analyze usually and may require careful analysis which is hard to perform without sophisticated tools.

Before starting to develop these tools we investigated and tried to use a lot of commercial and open source profilers. Some of them were good in certain domain areas like high level profiling, but lacked stability or scalability or suffered from performance overhead. Finally we came to conclusion that performance and functionality of existing tools could not match the demands and started the project.

Profiling overhead and scalability for programs having millions of traces was a big problem. The performance problems were solved with a use of sophisticated data structures, highly tuned caches and carefully tuned algorithms. Later efforts went more and more into the area of usability and functionality. Overall goal was to make finding and resolving performance problems as simple as possible for end user.

Each issue or usage complexity reported by end user was a source of inspiration to add new functionality, so the project evolved in a practical and useful tool for diagnosing performance problems in application servers and standalone Java applications like Eclipse-based products.

In the end of 2006 the project was initiated by Instantiations as an effort to create world leading Java performance analysis suite, Pavel Petrochenko developed the specification and started developing the project prototype with an active help of Ilya Seleznev. In April 2008, OnPositive was chosen by Instantiations as a primary consulting service provider and development force for Profiler project. In the August 2010 project was transferred to Google as a part of Instantiations acquisition. Later Google decided to contribute the project to Eclipse community, and chose OnPositive developers as primary project committers.

Project Scope

Project scope is to create a framework for building profilers to provide solid performance and memory analysis tooling, which will play well with Eclipse platform. Also the project intends to provide an exemplary implementation of profiler for Java language. However basing on this implementation project may serve as a foundation for building profiling tools for other platforms and programming languages.

Relationship with other Eclipse Projects

It seems that this project may be synergetic to the Memory Analyzer project which provides state of art heap analysis tools for Eclipse platform(for example in further releases we are planning to integrate MAT tooling to serve instead of current memory analyzer implementation). Other potential points for collaboration are seen with DLTK and PDT projects that may benefit from common core and UI, which may serve as a good foundation for building profiling tools integrated with existing IDEs for PHP, Python and other programming languages.

Relationship to TPTP: project is planned to serve as a replacement for archived TPTP project

Initial committers

The proposed initial committers are:

Pavel Petrochenko (OnPositive Technologies) - Project lead
Ilya Seleznev (OnPositive Technologies)
Denis Denisenko (OnPositive Technologies)
Dmitry Karpenko (OnPositive Technologies)

Pavel and Ilya was in this project from start. Denis and Dmitry contributed to it on 2008 to 2010 time frame.

Contributions

The initial contributions will be focus on evolving and stabilizing current tool functionality and implementation, as well on separation of project core and Java specific project functionality which will allow reusing existing functionality with other programming languages and platforms. After the separation it would be even possible to replace the whole UI or distribute the tool with a whole different set of memory queries.

The committers will continue to improve UI and make it cover new core functionality. Further generic memory queries will be added when identified.

Contributions in all areas are welcome.

The project's agile development process will follow eclipse.org's standards for openness and transparency. The goal is to provide an even more useful tool and framework. The tool would greatly benefit from the community. There are many different contributions that could be made to this tool:

  • Improve probes/triggers framework to support more sophisticated profiling trigger conditions and actions.
  • Improve high level profiling functionality to allow more detailed/clear profiling of high level operations (for example memory cache access or JDO operations).
  • Provide advisor framework which will allow to heuristically detect common performance problems/bugs at runtime and will suggest solutions for them (common case, which may be easily detected is searching in big list-based collections or highly fragmented IO access)

Developer community

The project expects to extend the initial set of committers by actively supporting a developer community. The goal is to have the initial developer community harden the core of the basic inspection and expert system platform to the point where it is easy for any other extension to leverage on the technology. The Runtime Analysis Tools project will encourage all active contributors to become committers. The project will make it easy for interested parties to take an active role in the project by making the planning and process transparent and remotely accessible.

User community

The Runtime Analysis Tools project is aiming to provide a state of art open source profiling tool which may be useful to a large community of Java developers and may help to improve quality of Java applications in general. As such, supporting and soliciting feedback from a large user community of commercial developers is critical to create an even more useful tool in the future. The project plans on doing this by using the standard eclipse.org mechanisms of supporting an open project and a community of early adopters.

Mentors

The following Architecture Council members will mentor this project:

  • Ed Merks
  • Gunnar Wagenknecht
  • Chris Aniszczyk
  • Kim Moir

Legal issues:

Currently some parts of initial code base depend on or are derived from LGPL libraries. These parts will be rewritten to use EPL alternatives:

  • Charting part of the project is based on JFreeChart library rewritten to SWT (LGPL).

    Switching to BIRT looks like a viable approach for replacing this library. However, more analysis is required here because using BIRT might introduce some technical issues, including performance issues with real-time charting

Changes to this Document

Date Change
01-Dec-2010 Document created
15-Dec-2010 Document released for public review
11-March-2011 Minor changes covering relationship with MAT and TPTP projects. Added mentors.