Summary
Eclipse requires that the contents of each project be stored in a single directory on disk. Every file and folder in that directory tree on disk must belong to the project in the workspace. This restriction makes it difficult to use Eclipse in conjunction with tools that have specific layout requirements for their files, or with users who have legacy file base layouts that they need to maintain. We would like to improve the situation for the 2.1 release of Eclipse. This document is the third of a trilogy. The first described the nature of the problem, and the second mapped out the various options for addressing it. This third document describes the concrete proposed solution in detail.
Last Modified: 18:30 October 24, 2002
Analysis of the problem identified three characteristics of Eclipse projects that made them difficult to use in conjunction with existing project layouts. Namely, Eclipse projects are currently single-rooted, all-inclusive, and non-overlapping. The proposed solution includes a new mechanism in the Platform core for allowing projects to be multi-rooted, a new JDT facility for interacting with a selective subset of a project, and a relaxation of the restriction that resources cannot overlap in the file system. For readers of the previous document that mapped out the solution space, these are solutions P-2, Q-1, M-2, S-2, V-1 and V-2.
Relaxing these restrictions is necessarily a breaking change for many tools built on the Eclipse platform. Some plug-ins may be able to adapt to the new behavior in future releases, but some may never work in the presence of projects that relax these layout restrictions. To minimize the impact, the proposed solution adheres to the following principles:
These solutions will not fundamentally change the workspace resource model. The workspace still maintains a faithful representation of corresponding files on the local disk. Local refresh will still cause the workspace to be updated to accurately reflect the changed contents on disk. Operations on resources in the workspace will continue to have direct impact on files in the file system -- deletion from the workspace will cause deletion of files from disk, etc. Directories on disk that are associated with resources in the workspace will continue to "belong" to the workspace, and this association will still be recursively all-inclusive. Projects will still have a single, non-overlapping root directory. The only significant change is that a project can now attach contents that are not in its root directory. This allows a single project to bring together contents from several different directories on disk.
Two categories of problems were identified in relation to the single-rootedness of Eclipse projects. The most common complaint is that it is impossible to locate the Java builder's output folder in another location. The other complaint is that users wanted to draw together sources from several locations into a single project. This second case arises commonly for users of the Eclipse JDT, but also for users of the CDT (C/C++ tools project). In general, this problem boils down to allowing the project to contain folders located elsewhere in the file system.
Part of the proposed solution is to add new API in the Platform core to allow linking external resources into projects (solution M-2 from the previous document). This will allow clients of the workspace API to logically connect a resource sub-tree located outside of the project's content area into a project.
The link operation is invoked on an IFile or IFolder handle in a fashion similar
to the create method. That is, it must be called on a resource that does
not yet exist, and the result of the operation is that the resource (and possibly child
resources) exists. No files or folders in the file system are moved or modified as
a result of creating the link. The link operation essentially just creates resources in the
Eclipse workspace tree corresponding to all files at the given file system location.
Similarly, deletion of a link removes the corresponding resources from
the workspace tree, but leaves the files in the file system untouched. The new terms
linked folder and linked file (generally, linked resource) will be used
to describe these new entities. This document will also use the term link location
to refer to the file system path corresponding to a given linked resource. Children of linked
folders are not referred to as linked resources.
Support will only be added for linking resources as direct children of a project. Restricting links to only the first level makes the model easier to understand for clients and users. Essentially, the project becomes a logical container that can have arbitrary resources linked in, and the semantics of other files and folders remain the same. None of the other IDEs surveyed allowed linking external resources at arbitrary depths. Finally, supporting links at arbitrary depths would add significant complexity to the implementation, and would affect the performance of most "deep" workspace operations such as deletion, local refresh, move, etc.
Links will have similar semantics to soft symbolic links in Unix. Copy and move semantics are as follows:
delete operation.
A new validation method (IWorkspace.validateLinkLocation) will be introduced
to allow clients to check if a proposed file system location is an acceptable location to link into the
workspace. This method will return a status with severity IStatus.ERROR if the
proposed location is not acceptable (for example, if it contains characters that Eclipse does not
allow in filenames, or if it overlaps the workspace metadata area). The method will return a
status with severity IStatus.WARNING if the proposed location overlaps
another resource location in the workspace. The section on overlapping resources will
discuss this in more detail.
Link locations will be represented by entries in the .project file. Modifying the .project file can cause links to be created or deleted. A new kind of Core path variables will be introduced, and link locations can be relative to these variables. Resources will be created for all links described in the .project file, even if they cannot be accessed (for example if based on an undefined variable or location that is not accessible).
There may be some plug-ins that, for whatever reason, do not want to tolerate links in projects they operate on. For example, a web site management tool may rely on the file system structure of a project being the same as the structure in the resource tree. A mechanism will be introduced to allow plug-ins to prevent links from being added to a given project.
An attribute will be added to project natures that specifies whether the nature allows links. This will default to being true (links allowed), if the nature does not specify it. If any nature installed on a project does not allow links, then attempts to create links will fail. Attempting to install a nature that does not allow links on a project that already has links will fail.
Additionally, a hook will be added so that team providers have an opportunity to disallow links. Team providers based on 2.0 will not allow links on projects that they manage. Team providers based on 2.1 or greater must specify that they tolerate links.
If either of these mechanisms decides to veto links, users will not be able to override this decision (except by removing the intolerant plug-in).
Generally, links will be ignored by team providers. The assumption is that these files are generally taken from some location outside the workspace because they are already managed by an external VCM tool. The java output folder is also usually of no interest to VCM tools. Team providers that have been upgraded to Eclipse 2.1 will be expected to silently tolerate and ignore links. This includes handling of links in the team move/delete hook. If team providers wish to provide more support for linked resources, they may do so. As mentioned earlier, team providers based on Eclipse 2.0 will not tolerate the presence of links on projects they manage.
The Platform UI will add new operations to allow users to create links as top level folders and files. It may be useful to provide a warning in the UI when linking is first attempted to warn users that this may cause problems for other plugins (A warning dialog with the option, "Don't tell me again" would be appropriate). Linked resources should have some visual annotation in the resource navigator to make it easy to distinguish linked resources from normal resources in the UI.
The Platform UI will also add a preference page and dialogs for adding, removing, and editing the platform core variables used in link locations. This will be similar to the current UI for java classpath variables.
JDT core will need to revisit compiler and search infrastructure that currently relies on direct navigation of resources in the file system. Since it is no longer guaranteed that all source folders are children of the project's local directory, they will need to consult the workspace API to find the correct locations.
New flexibility will also be added for java output folders, specifically:
Allowing multiple projects to share an output folder introduces a new problem. Currently, each java builder assumes that it has complete control of the output folder. On full build, it currently deletes the entire output folder, which will obviously not be acceptable in a shared scenario. If the builder does not delete the output folder before full builds, then any extra .class files in the output folder will influence the classpath of that project. This can have unexpected results at runtime if stale classes in the output folder obscure legitimate classes in other projects. Neither approach (scrub everything or scrub nothing) is entirely appealing. However, a flag will be introduced on output folders to allow either of those two options. If the user turns scrubbing off, they will have to deal with the problem of performing "make clean" before full builds by themselves.
In some of the identified problems, users wanted to operate on only a selected subset of the resources in a project. There are two different angles to this problem:
The first problem is more difficult to address, and would cause more grief for tools that rely on the all-inclusive nature of the workbench. The difficulty in supporting complete exclusion boils down to tension between plug-ins that have different needs. One plug-in may wish to exclude certain resources, while another plug-in operating on the same resource tree may not want to exclude those same resources. Furthermore, external tools that may operate on resources in the workspace (such as Netscape™ or Ant scripts), will not be aware of any resource exclusion mechanism introduced in the platform, and would thus be fundamentally at odds with such a mechanism.
For these reasons, support for exclusion will only be introduced at the JDT level. JDT Core will provide a new facility for adding exclusion rules to source classpath entries. These exclusion rules will support matching of path prefixes, e.g., exclude com/xyz/package1/*. Excluded resources will be omitted completely from the Java model, and will not be considered for compilation. These exclusion rules will be stored in the .classpath file, and thus automatically shared with other team members. This approach is equivalent to solution S-2 from the previous document.
The proposed solution is to have almost no restrictions on overlapping link locations. Existing restrictions on non-overlapping project locations will remain. Linked resources will only have a small number of restrictions to prevent situations impossible to support. This breaks down as:
The platform core change will introduce some new behavior and wrinkles
in how resources behave. If two resources correspond to the same underlying
file, then changing one resource will incur a resource delta on both. Also, if a
refreshLocal is invoked on an overlapping portion of the workspace,
then several different resources may be refreshed as a result. Because of the potential
dangers of having overlapping resources (generally that changing one will affect all duplicates),
a warning will be presented when a user tries to create a linked resource whose location
overlaps another resource in the workspace. Calculation of this warning will be done
by the new validateLinkLocation method.
This change has ramifications for the following Platform Core methods:
IFile IWorkspaceRoot.getFileForLocation(IPath location);
IContainer IWorkspaceRoot.getContainerForLocation(IPath location);
The specification of these methods will be clarified to indicate that they will
ignore linked resources. They will continue to only look in each project's root
location for the file system path in question. If a given file exists under both a
project's root directory and under a link location, it will always return the resource
under the project's root directory. Since project root directories cannot overlap,
there is no ambiguity to this solution. New link-aware API methods will be introduced for finding
the set of resources corresponding to a given file system location.
JDT Core will add the ability to introduce nested source folders, subject to the condition that the nested portion is excluded from the parent folder. Users will not be allowed to link the same folder twice to create two different source folders at the same location in a given project.
To show that the proposed solution is sufficient to solve many of the problems described in the problem definition, we will revisit the four example workspaces from the problem statement and show how they can be supported by the proposals in this document.
Example 1:
|- AllProducts
|- Product1
|- JavaSourceFiles
|- Product2
|- JavaSourceFiles
This configuration can be supported using only the new linked resources.
Projects would be left in the default workspace content area, and the source folders
would be linked into the monolithic directory tree. Eclipse artifacts such as .project,
along with build output, would remain in the default area, thus causing no pollution
of the source tree with transient or Eclipse-specific files. This solution assumes that
the resources are not being managed by an eclipse team provider:
/Product1 at default location
/src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles
/Product2 at default location
/src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles
If such a monolithic resource tree needs to use an Eclipse team provider, then an extra project can be used whose location is the "AllProducts" folder. This extra project would be a simple project without a java builder, and would be used for source control purposes only. All other projects would not have a team provider.
/MasterProject at file://AllProducts (shared with team provider)
/Product1 at default location (no team provider)
/src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles
/Product2 at default location (no team provider)
/src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles
Example 2:
|- AllJavaSourceFiles
|- com
|- xyz
|- product1
|- P1Main.java
|- product2
|- P2Main.java
This common configuration requires support from all three proposal areas. Again projects would remain in the default content area, one project per product. All projects would link the same external folder as a source folder, AllJavaSourceFiles. Each project's source folder would then exclude all other product packages from its build classpath.
/Product1 at default location (no team provider)
/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.
/Product2 at default location (no team provider)
/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.
If such a monolithic resource tree needs to make use of an Eclipse team provider, it can be arranged in a similar fashion to example 1. An extra project would correspond to AllJavaSourceFiles, and would have an Eclipse team provider installed.
/MasterProject at file://AllJavaSourceFiles (shared with team provider)
/Product1 at default location (no team provider)
/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.
/Product2 at default location (no team provider)
/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.
Example 3:
|- Product1
|- JavaSourcesFiles
|- com
|- xyz
|- product1
|- P1Main.java
|- tests
|- com
|- xyz
|- product1
|- tests
|-
P1Test.java
This directory structure could be supported in several ways, but the most likely solution is as follows. A single project would be created. The path Product1/JavaSourceFiles/ would be specified as a source folder. This folder would have an exclusion rule to omit the "tests" sub-directory. The path Product1/JavaSourceFiles/tests would be specified as a second source folder, with no exclusion rules. If desired, the two source folders could be placed in separate Eclipse projects to provide greater separation between application code and tests.
/Product1 at default location
/JavaSourceFiles - java source folder excluding the "tests" sub-directory.
/tests - java source folder
Example 4:
|- CommonFramework
|- JavaSourceFiles
|- Product1
|- JavaSourceFiles
|- Product2
|- JavaSourceFiles
Again this structure can be supported in several ways. The most likely is to create three projects, mapping to CommonFramework, Product1 and Product2. Product1 and Product2 would create a linked source folder CommonFramework/JavaSourceFiles. In this case CommonFramework would not need to be a java project, as it mainly just acts as the VCM container for that set of files.
/CommonFramework at file://CommonFramework
/Product1 at file://Product1
/JavaSourceFiles - java source folder
/src-common - java source folder linked at file://CommonFramework/JavaSourceFiles
/Product2 at file://Product2
/JavaSourceFiles - java source folder
/src-common - java source folder linked at file://CommonFramework/JavaSourceFiles