Eclipse IDE Meta-tooling Platform ("The Eclipse IMP")
The Eclipse IDE Meta-tooling Platform is a proposed project under the
Eclipse Technology Project.
This proposal is in the
Proposal Phase (as defined in the
Development Process document) and is written to declare its intent and scope. This
proposal is written to solicit additional participation and input from the Eclipse
community. You are invited to comment on and/or join the project. Please send all
feedback to the
Eclipse provides a very high-functionality IDE for Java, as well as for several other
languages (e.g. C/C++, Python, Ruby). At
this point, however, the cost of developing an IDE for a new language remains
prohibitively high. This is due largely to the fact that although Eclipse offers a
rich suite of building blocks for constructing a wide variety of applications, the set
of APIs intended specifically for building IDEs is largely tied to specific
languages. This precludes code reuse even where common concepts and language
processing idioms already exist.
Moreover, building an IDE for Eclipse is an extremely knowledge-intensive task,
requiring intimate understanding of hundreds or even thousands of interfaces and
implementation classes. As a result, the learning curve is daunting, and there is not
nearly enough guidance available to direct the IDE developer to the relevant
interfaces and reusable implementation classes.
The Eclipse IMP's purpose is to develop a set of extensible frameworks and exemplary
tools built on the Eclipse platform that ease the creation of IDEs for programming
languages (new or otherwise) that do not yet enjoy full-featured Eclipse support. Our
ultimate goal is to assist language implementers in creating IDEs with a richness and
usability on par with the Eclipse Java
Development Toolkit (JDT). In the near term, however, we intend to provide support
for a significant portion (but not all) of the JDT's functionality. It is our belief
that this is a natural next step in the evolution of the Eclipse - doing for
programming IDEs what RCP
does for rich UI applications.
By accelerating the process of developing new IDEs, we also hope to expand the use of
Eclipse to new programming domains, further enhancing the appeal of the Eclipse
platform. In fact, we hope to encourage language design experimentation by making it
viable for language implementers and their communities to produce full-featured IDEs
with more reasonable effort than is presently possible.
- Service Generators. A set of code-generating wizards and templates that
generate stub/initial implementations for various language-specific IDE services,
along with any necessary meta-data (e.g. extensions of IMP-defined extension points).
- IDE Runtime. A class library that encapsulates behavior and appearance common
to many IDEs and languages, with hooks (via extension points or abstract methods) for
providing language-specific customizations. Example sub-components include views such
as the source editor, the outline view, or the program explorer, a dependency
tracking mechanism for incremental building, and so on.
- Language Service Extension Points. A set of extension points that encapsulate
the various language-specific IDE service implementations (e.g. parsers, syntax
highlighters, outliners, formatters, etc.).
- Declarative Languages. A set of domain-specific languages that will allow IDE
developers to specify such language-specific concepts as language syntax (e.g., for
syntax highlighting or outline contents), language semantics (used in providing
various other language services), appearance in the IDE of program elements, program
transformations (e.g. refactorings), various program analyses, and so on. This will
include full-featured IDE support for the declarative languages themselves.
- Parser Generator IDEs. Since much of IDE development activity revolves around
the language grammar, this component provides IDE support for various existing
grammar specification languages and parser generators, such as LPG, Antlr, or JavaCC.
- Compiler/Parser Adapters. IMP will not require any particular scanning,
parsing, or AST technology. In particular, IMP must be capable of reusing existing
compiler front-ends. This has the dual benefit that IDE developers can avoid
gratuitously rewriting existing code, and that the IDE and compiler are in complete
agreement as to the structure and meaning of every construct. To that end, some
adapters will be required to permit IMP components to work with existing
parser/compiler generators, such as LPG, Antlr, or JavaCC.
- AST-based Tools. A key part of refactoring is transforming existing
code. Currently, in the JDT this is done by writing verbose Java code to manipulate
ASTs and build new AST nodes. A language-independent AST pattern/transformation
language would help speed the development of new transformations and make it easier
to reason about their correctness. Such a pattern language would also be useful for
specifying how to populate search indexes, specifying candidate "quick fixes", as
well as for providing the basis for a user-level AST-based search mechanism. We
intend to develop a suite of domain-specific languages for accomplishing these and
other tasks, all making use of a common AST pattern language.
- Static Analysis. A set of reusable analysis components (type inference,
pointer analysis, call-graph construction, escape analysis, etc.) to be used in
constructing refactorings and other tools that rely on deep semantic knowledge of the
program. While static analysis is by definition language-specific, the analysis
algorithms themselves can share much code, including data structures, graph
algorithms, generic solvers, etc. Probably to be built on top of an existing static
analysis framework such as WALA.
- Debug Framework. This component would provide, e.g., a specification language
to define the debug-time presentation of the runtime platform (e.g. backtraces,
variables, breakpoints, etc), along with a framework to map this specification into
implementations of the necessary Eclipse Debug APIs.
- Process Documentation. Cheat sheets and other forms of documentation to guide
the IDE developer through the process.
- IDE-building Perspective. A perspective helps focus the user's attention on
the relevant IDE-building views, actions and commands by making them appear in the
toolbar, top-level menus, and so on.
Relationship with Other Eclipse Projects
The Eclipse IMP will be built on top of the Eclipse Platform. Since
the JDT represents the desired level of
functionality, we intend to work with the JDT team to identify existing components in
the JDT that can be generalized and made part of the IMP.
The LTK (Language ToolKit)
project provides runtime language support components, which is used for example by the
IMP to provide refactoring support. We will investigate which IMP components could be
moved to the LTK, which can be used even by IDE developers who do not build their IDE
using the IMP meta-tooling.
The Web Tools Platform's
Structure Source Editor may also provide components useful for the source editing
portion of the IMP runtime.
The DLTK is a related effort, but requires
the use of a specific AST class hierarchy to drive much of its functionality, which is
not suitable for arbitrary languages, nor for situations in which a compiler front-end
already exists. Nevertheless, we intend to investigate what portions of the DLTK's
infrastructure can be reused or adapted.
Ed Merks of the EMF project
believes that EMF may benefit from enhancing IMP to permit the design and
implementation of concrete textual language syntax from existing EMF models. This is
based on the observation that an EMF model can be viewed as abstract syntax for a
language. The result is that rather than using generic list- and tree-based EMF editor
views for both the model and model instances, one can instead use IMP-based
language-specific (i.e. model-specific) textual source editors, with all the
affordances that IMP provides for textual languages (e.g. syntax highlighting, content
assistance, navigation, hover help, etc.).
We propose this project should be undertaken as a Technology project rather than as
part of the Eclipse Platform. The technology is not yet well defined enough in either
specification or implementation to be ready for general consumption. As a Technology
project, we will have the opportunity for eager adopters to use, critique, and improve
the software in a non-disruptive manner before its use in mainline Eclipse Platform
Finally, since Technology sub-projects are not meant to be ongoing, we foresee three
possible evolutionary paths for the subproject:
- The project is merged back into the mainline Eclipse Platform top-level project.
- The project is moved to a permanent state as a Tools Platform project.
- The technology proves to be insufficient for Eclipse and the work is set aside.
In all likelihood, some combination of outcomes (1) and (2) will make the most sense
for sub-portions of the components developed within this project.
- Rich Gronback, Borland
- Ed Merks, IBM
Proposed initial committers
IBM Research is offering to contribute its Eclipse IMP software (previously released
as open-source here) as the
- Borland, Inc. - Artem.Tikhomirov@borland.com
- Eclipse JDT team - Martin Aeschlimann, Philippe Mulet
- Oxford University - Oege.de.Moor@comlab.ox.ac.uk
- Centrum voor Wiskunde in Informatica (CWI) - Paul.Klint@cwi.nl, Jurgen.Vinju@cwi.nl
- Cornell University - Andrew Myers
- EPFL, Switzerland - Martin Odersky
- Delft University - Eelco Visser
- Eclipse CDT team
- IBM Rational - Chris Laffra
- Initial Eclipse.org presence in Oct 2007
- CVS repository, seeded with source code from current (non-public), if IBM source code
contributions are accepted
- Bugzilla repository, seeded with bug reports from current (non-public) Bugzilla
repository, if IBM's code contributions are accepted.
- Initial Eclipse.org release, v1.0: Dec 2007, including:
- IMP Runtime library including language-independent editor
- Initial version of meta-tooling wizards, with support for scanning, parsing, syntax
highlighting, outlining, source folding, hover help, hyperlinking, content assist,
- Meta-tooling support and IDE support for LPG-based grammar and scanner
specifications, with APIs that permit integration with other parser/scanner
- Exemplar IMP-based IDEs for X10, the LPG grammar specification language, Java and
- Cheat sheets guiding/describing IDE development process
- Initial installation and use documentation, taken from existing (non-public) web
- Second Eclipse.org release, v1.1: June 2008, possibly including:
- Language embedding
- Integration with the JAstAdd compiler framework
- Integration with other parser generators such as ANTLR and/or JavaCup
- Improved support in meta-tooling and code templates for language inheritance