Omnibus Resource Improvements

Last revised 10:00 Wednesday February 27, 2002

This note summarizes a series of proposed additions and changes to Eclipse to improve the resource story and address several Eclipse 2.0 plan items.

Reify project description as project resource

A project's project description contains important information about the project.

The proposal is to reify the project description in a file, named ".project", in the root of each project's content area. The contents of the file would be a well-specified XML format, and contain Core project description information: project name, project description, list of project nature ids, list of required project names, and the project build spec.

The .project file in the root folder would be considered the master copy. IProject.setDescription would overwrite this file. If a team provider (or a user) changes the .project file through means other than IProject.setDescription, this eventually affects the project much as if IProject.setDescription had been called. The one exception is changes to the project name, which would be disallowed [alternative: rename the project]. After the fact resource change events are reported on the .project file resource as well as being flagged as a description change on the project resource.

Note that the location of the project's content directory is not represented in the .project file. Rather, the .project file is found in the project's content directory.

As a consequence of this change, Core should drop its internal .prj file.

For open projects, core would keep the project description in memory at all times. IProject.getDescription uses the in-memory information - it does not access the local file system. Core would also remember a timestamp for the .project file. When refreshLocal picks up a change to a .project file, Core reads the modified .project file and changes the in-memory project description. Resource change events are reported much as if the project description had been changed via IProject.setDescription.

Problems addressed:

New wrinkles created:

Derived resources

Derived resources are regular files and folders in a project that are created in the course of translating, compiling, copying, or otherwise processing other files. Derived resources are not original data, and can be recreated from other resources.

It is commonplace to exclude derived resources from VCM because they clutter the repository with ever-changing files that each user appears to modify regularly. Each team provider typically provides the user with a way to include certain files under VCM and exclude others. If plug-ins that create derived resources mark them as such, the team provider can do a better job of excluding by default those resources that any user is unlikely to want in the repository.

The proposal is there be a way for non-Team-clients to provide hints that certain resource subtrees are likely to contain derived resources. This would be done by setting a mark bit on a folder or file; the mark would mean that the resource, and any present or future descendents, should be considered derived resources by default. Note that this is only a default. A user should still be able to place any resource under management even if marked as, or sits below, a derived resource.

Example: JDT would set this bit on the project's output folder (or on individual generated *.class files when the output folder coincides with the project root).

Example: An editor or utility that creates a backup copy of a file in the same folder should mark the copy as a derived file.

Derived marks would be recorded in the resource tree, and saved to disk along with the in-memory resource tree. Derived marks would not be recorded in a Team repository.

To mark (or unmark) a resource as derived, the resource would need to exist. There would be no resource delta for setting or resetting this bit; default is off (i.e., not derived). However, the fact that the bit would be retained in memory at all times means that a Team provider can quickly discover whether a plug-in has marked a resource or one of its ancestors as derived.

The user would be able see and change a resource's derived flag via the Properties dialogs. It would not warrant anything more obvious since users are unlikely to ever be interested in this property. However, letting the user mark a resource as derived would allow them to indicate that certain files (such as certain metadata files) are not prime candidates for version management.

In addition to this basic support for derived resources, the Team component will maintain a list of excluded file and folder name patterns. Any files or folders in the project with names that match these patterns would be considered unmanaged (or unmanaged by default).

Problems addressed:

Team-private resources

Some Team providers have extra files and folders that are created in the project by the team provider to satisfy certain external requirements of that provider. For example, CVS uses a CVS/ subfolder in each managed folder to store CVS-specific information. Although these files are needed by the team provider, they are of no interest to other plug-ins. Indeed, other plug-ins should not even know of their existence.

The proposal is to provide a way for a Team provider to hide certain resources in the resource tree so that they are not seen by other plug-ins. This is achieved with a new team-private member flag in the in-memory resource tree. When a file or folder is marked as a team-private member, it is omitted from the list of resources returned from IContainer.members (a variant method includes team-private members upon request). In all other regards, a team-private member is like any other file or folder resource that exists in the workspace.

Team-private member marks are recorded in the resource tree, and saved to disk along with the in-memory resource tree. Team-private member marks are not be recorded in a Team repository. To mark (or unmark) a resource as a team-private member, the resource needs to exist. The bit defaults to off (i.e., not team-private).

Creations, deletions, and changes to team-private member resources are reflected in resource deltas. Like IContainer.members, the deltas seen by normal delta clients omit all mention of team-private member resources (the unfiltered list of child deltas is available on request). A change to the team-private member flag of a resource is reflected in the delta (indicated as a change to the resource's team-private member property), and will be seen by all delta clients (because the resource was a non-team-private member either before or after the change).

The mechanism is intended to be used by Team providers. Team providers are in the unique position of being able to set this flag to hide any extra files or folders they might create. Team providers can do this without needing to store additional metadata in the repository.

Example: the CVS team provider would set this bit on the extra "CVS" folders that it creates in most folders. By doing so at the time a CVS folder first enters the workspace, the folder does not show up in the normal resource delta, and therefore slips in to the workspace without other plug-ins noticing. Since the CVS folders contain special metadata, the CVS team provider may need to protect these files from being moved or deleted; the new move/delete hook (described in a separate proposal) is useful in this regard.

Note that other plug-ins are not is a similar position because team-private member marks are not recorded in team repositories. If a non-Team provider plug-in was to set this flag, the resource does disappear from general view in the workspace. The hiding is only for that workspace, however. When a resource propagates to another workspace via a Team repository, the team-private member flag does not propagate along with it. Thus, this mechanism cannot be used by other plug-ins to hide their resources.

The workbench resource navigator does not show team-private members by default, although the user can ask to see team-private members if they want (similar to the way ".*" metadata resources are hidden from the user by default). The user can see (but not change) a resource's team-private member flag via the Properties dialogs. It does not warrant anything more obvious as users are unlikely to ever be interested in this property. The user is not allowed to change the setting is this flag.

Problems addressed:

Java builder resource file copying

The Java incremental project builder copies resource files from source folders to the Java project's output folder. This allows the runtime classpath to be shorter, and makes the output folder more closely resemble a typical JAR contents used when deploying a Java program.

However, this resource copying can have undesirable consequences when it copies files willy-nilly to the output folder, and can create a situation that confounds the user who peers into the project's output folder.

The proposal is for JDT to provide a mechanism for excluding certain files from being copied. This would take the form of a list of file name patterns to be excluded. This would make it easy to exclude files with certain names (e.g., "*") or certain types of files (e.g., "*.antlr"). The list would be primed with the pattern ".*", which would exclude all folders and files matching the common Eclipse metadata naming convention. Through the UI, users would be able to remove this pattern if it was too strong, and add additional patterns to exclude other files.

Note that the derived bit is unsuitable for controlling JDT copying behavior (a derived resource file might very well need to be copied to the output folder).

A global list of exclusion patterns would be persisted by JDT core. This information would remembered with the workspace as JDT Core metadata (it would not be in the .classpath file, and would not be shareable across workspaces).

Problems addressed:

Other issues

Project archive files

Given the project description is reified as a file in the project, there is no longer as great a need for a special form of project archive. Users can create a zip archive containing the project's resources, which would now include its .project file. After unzipping into the local file system, the project can be added to a different workspace. 

Read-only projects

No changes are planned to support read-only projects.

The notion of a read-only Eclipse project is somewhat of an oxymoron. An Eclipse project is for doing work, and must be located in a read-write file system. The binary library projects employed by Java development tooling are an admitted abberation to facilitate people working on large-scale products spread across several related Eclipse projects. The idea is that it is easy for the user to replace a binary library project in their workspace with the real, source code project from a team repository if for some reason they decide to do so.

PDE uses binary library projects for plug-ins. For self-hosting convenience, PDE provides the option to locate an Eclipse plug-in project directly in the directory where the plug-in is installed. Doing so requires creating additional files in that directory (including JDT's .classpath file), and can only be done when the plug-ins are installed in a read-write directory. For shared plug-in installs on, or plug-in installs on read-only drives, there is no option but to have PDE create projects elsewhere and copy over the necessary files from the plug-in.