Hi Walter,
I forgot all about the absolute paths issue with linked
resources. I'll update the wiki.
I'm a bit confused about your comment about this not
being a project setup issue. We have our own builder as well, and it
will happily build whatever the build description says, whether those files
are under the project root or not. We even have our own project explorer
view which shows a logical representation of the project rather than the
physical file system layout. But we still run into a lot of issues when
files are not under the project root - that is, when you can't get an IFile
for them.
We have a wide range of user types from small application
developers to large system developers. In many cases, a users code base
consists of hundreds of directories with thousands of source files. In
such a source base, there are many hundreds of build artifacts and almost as
many "logical projects". It is a huge problem for these users to be able
to create projects currently. They will typically have a few projects
going at a time, but many times the natural project root for all of them will
be the same. We've found ways to work around some of the other issues,
but not this one. It sounds like perhaps you guys have found a
way. Could you elaborate on how you setup your
projects?
Thanks,
Warren
Hi Warren,
FWIW, you did not mention anything about linked
resources and absolute paths these persist in the .project file by default.
Again a big issue around linked resources in combination with sharing project
within a team (even without team support), and one more reason why they appear
to be so cumbersome to handle. To me it seems many times one has
to unsell linked resources to users: Whereas linked resources
are (kind of!) nice for evaluation purposes (because, yes, in this case
you might not want to pollute your sources), as soon as you start serious
development, you run into all kind of troubles. The hurdle to get everything
right from the beginning is overwhelming for novices (e.g. its not possible to
change a linked resource to use a variable later). Sorry, I don't know how to
add this to the Wiki page...
Having said that, the scenario you describe is
really about having the flexibility around build and indexer setup, not around
project setup, IMO.
It's rather classic: users have common
code they want to reuse in multiple applications - so they create one or
a set of libraries out of it, within one or a set of projects. Of course,
indexing should be able to handle only code going into these libraries, and
optionally ignore the rest. Then, they create their application projects, which
use the binary artifacts of the library project(s). Now it would be great if
they would have automatic support for application linkage specification, i.e.
some nice wizard or UI allowing to select the library binaries of other
projects to be linked in, without the need to specify it manually in the
linker options. And probably also desired: during application code
development, the public API's of all used library projects should be the only
thing they see WRT code completion, etc. I guess, some UI would be needed for
this as well.
And now think of all developers in the world.
Wouldn't it be great to give as many of them the freedom to choose how to
achieve this? Either everything in one project, or one project per build
artifact, or one project per module/application/product, or with nested
projects... its possible. Our commercial IDE based on CDT supports all this,
and we did not have to provide some EFS or work with linked resources.
Well, we had to override the build system, and this is IMO the place to solve
this in CDT as well.
Again, I don't see anything specific to project
setup. The issue around having the source tree polluted with project files - I
don't think this is the big thing. I would not leave the Eclipse path in this
area at all and allow to separate the project file from the project location.
Its a very general paradigm of Eclipse, and I am pretty sure doing everything
differently will generate lots and lots of problems in all kind of areas
(probably much more than you already identified), unless you make it a new
Eclipse way (add/change this in the platform, not in CDT, that
is).
Just my 2 or 3 cents again,
Walter
I've updated the Wiki page http://wiki.eclipse.org/CDT:Flexible_Project_Structure with
some more thoughts on the issue. It would be great to get feedback
from other CDT users - both those shipping C/C++ IDE's and end users.
You'll see that I'm not convinced that the linked resources route is a
viable option. Maybe we can get the platform team involved in the
discussion to help find the best route forward.
Thanks,
Warren
I guess what my investigation has shown me that the EFS
solution and linked resources are pretty much identical. I really
noticed this when trying to figure out how to persist the adds and found
myself wishing I could add that to the .project file just like linked
resource are. And they are....
I think all the issues that we have with linked resources would be equally as bad with the EFS solution,
possibly worse because the EFS adds are hidden. The CVS one is a great
example. I really doubt CVS would work with the EFS solution either. And I
don't want us to think EFS would be better since it's not in the platform
where we'll have a battle getting changes. Any platform changes required to
make linked resource work correctly would also need to be done for
EFS.
So my hope is to save the effort at implementing
the add/remove functionality since I believe that's already there with
linked/hidden resources. We can then focus on making linked resources
work where we need them and improving the workflows. But this
really needs to start now.
So, Warren, you've somewhat started a list of
workflows that we'd like to support with this solution. This is a great
place to start. I've created a Wiki page to start capturing these. Please
feel free to add more information, especially to the workflow section. When
we have that we may get a better idea of which of the two solutions will
work best.
Thanks,
Doug
We've been working on Eclipse/CDT based products for
about three years now. I'm sorry to say that the project model is
still not satisfactory for our purposes. We've tried many angles, but
are still stuck with some pretty serious limitations. I've volunteered
to investigate the EFS route to see if it will help at all. Based on
this thread I'm assuming it won't.
Let me give you a brief overview of how our users work,
and then discuss the problem we've run into. I don't think any of this
is specific to our users BTW.
Most of our users have existing code bases. They
simply want to "import" it into the IDE. Others will create new
projects from our templates. The new projects are created in the
workspace. Imported projects could be anywhere in the file
system. Often times they will import several projects from the same
source tree. This is where our biggest problem is. Let's say the
source base looks like this:
C:\MyProjects\Project1\...
C:\MyProjects\Project2\...
C:\MyProjects\Common\...
Because both projects
share code in the Common directory, the logical root project directory for
both Project1 and Project 2 is C:\MyProjects\. But in Eclipse you
can't have two projects with the same root. This is where the .project
and .cproject files are created. So currently our users would import
Project1 with the natural root (C:\MyProjects\), but Project2 has to be
rooted at C:\MyProjects\Project2\. This means that any source/headers
from the common directory are not under Project2. This means those
files are not in the project explorer for that project, are not indexed,
etc.. We logged this against the platform - https://bugs.eclipse.org/bugs/show_bug.cgi?id=78438.
Basically if you put the .project file anywhere, but have a project root
attribute, this would cease to be a
problem.
Our first product actually always created the
.project in the workspace, and for imported projects, would create links to
files and folders. We ran into so many issues with this that we had to
change the model. I don't recall all of the issues, but here's a list
of some:
- Version control simply didn't work at
all
- You can't make file system changes with
links. For example, if you want to rename a file or folder, or move a
file around, you can't do this with linked resources. It only changes
the link itself, not the underlying
resource.
- Creating new resources in a project with links is
confusing at best. Let's say you have a project with a linked folder
and file at the root. If you create a new file or folder at the root,
it is created in the workspace, not where the other folder/file are in the
file system. But if you create a new file under the linked
folder, it gets created where you'd
expect.
- The location of the .project/.cproject files are
problematic. Some users will want to keep these in version control,
while others won't. Those that do want them created in the source
tree, but those that don't want them elsewhere, like the workspace. I
forget now why this was a problem with linked resources, but there was
something weird going on
there.
I suppose it's worth noting that the last time we
really looked at this was in Eclipse 3.2, so some of this may have been
fixed by now. But I doubt it. In general linked resources are
second class citizens. Some IResource API's just don't work for linked
resources. Just search for references to IResource#isLinked for
"special handling". I suspect that we'll run into similar issues with
EFS though - see getLocation vs
getLocationURI.
We also have the same issue that Doug is trying to
address (hiding some branches in a source tree). This is much less of
an issue for us though. You can already reduce the scope of the
indexer and the build. The only real issue for us is for a very large
source tree, you're going to get IResource's for everything, which slows
things down quite a bit. There is actually somewhat of a problem in
reducing the indexer scope - see https://bugs.eclipse.org/bugs/show_bug.cgi?id=178159.
The hidden attribute addition sounds promising for
hiding resources under the project root, but doesn't really do anything to
add flexibility to the contents of a project. EFS sounds like it would
though. What I mean by that is, having resources under a project that
are real resources, not linked, but that don't live under the project root
in the file system. I've just started looking into EFS, so maybe it's
a bit of wishful thinking at this point, but I'm hoping we could create a
project anywhere, and when we create it we pass the URI location from our
own EFS. Then when asked for the children, we could return URI's for
files from anywhere in the file system, or on other machines even.
This would seem to hold the potential for working around the issues listed
above. We'd basically have an EFS map from what we want under a
project to the actual file
system.
So hopefully some of the experts can chime in
here. Is my hope for EFS unrealistic? Is there a different
approach we should look at?
Thanks,
Warren
Hi,
after reading this rather long thread, I'll
decided to throw in my personal opinion.
I consider this approach to work against one of
the most general Eclipse platform paradigms, where a project is defined
to be a root directory and everything in it. IMO, the more workarounds are
introduced against this paradigm, the more problems will be faced, and the
more incompatibilities (or at least unawarenesses)
created.
Isn't the whole problem you try to solve here
rather about what files should go into the build (and probably into the
indexer) than what files are part of a project? I understand that CDT has no
separation of what a project and what the build input is (well, IIRC one can
exclude specific files from the build, but in general, the project content
defines the build input, right?).
In our commercial IDE, we separated this. This not
only introduced much more powerful build setup capabilities in general, but
especially enabled users to setup build artifacts with arbitrary
contents (think of sources being compilable with different compiler flags
for different build artifacts, build input exclusion patterns, build input
from all over the workspace, multiple build artifacts within the same
project, reusable build artifacts accross project boundaries, etc., etc.,
etc.). BTW, we call this build system flexible managed build - because
that's what it is:-)
Of course, one can setup CDT projects as of today
to exactly contain what is desired (with the help of linked resources).
However, I find linked resources to be cumbersome and error prone, though
many of our customers start out with them during evaluation as well, mostly
because they are looking for a way to achieve what they did in the past with
other non-Eclipse based IDEs, but sooner or later I know of lots of
them realizing its much easier to use the features of our flexible build
system instead, especially if projects need to be shared in a team.
And now think of virtual file systems, the potential complexity of
these, hidden assumptions, restrictions, etc. Sounds worse than linked
resources to me.
I guess, the point I am trying to make is:
whatever you decide to do, make it understandable
and transparent (and of course as simple as possible to use) for
the user.
As said, just my 2 cents,
Walter
Jogging through the code, it really looks like the
HIDDEN feature is what I was looking for. What I haven't found yet is UI
to make a resource hidden or a navigator filter to show hidden resources
(in case you want to bring them back). Is this
planned?
Assuming we have the core features available to link
in and hide resource, I think we still have workflow issues that need to
be addressed. I like Ken's idea of a file that controls the
linking/hiding. We could have an import/export mechanism for setting up
projects based on this file. A nice wizard for creating the file would
also be good, similar to the way the way the export file system wizard
works.
Given this, we may be further along than we thought
(BTW, the hidden stuff seems to have been added in 3.4
M4).
Cheers,
Doug
Thanks Michael/Szymon,
Is there a bug describing the isHidden
feature?
Doug
Doug et al,
Szymon is really the person you want to bug
on this but I'll throw in my 2 cents ;-) First, I have to say that a
solution at the IResource level (e.g. using linked resources and the new
hidden folder support) is infinitely better from a repository provider
perspective than an EFS based solution. You may not get all the Team
support you want at the IResource level but a solution at the EFS level
would certainly break the existing CVS client since the CVS client isn't
EFS aware to any great extent. For instance, if you tried to hide a folder
using EFS, the CVS client would probably try and recreate it the next time
you performed a Team>Update. It is also important to note that the
Platform does not provide all the hooks required by repository providers
and I know of at least one provider that has resorted to using it's own
EFS implementation under projects that are mapped to that provider to get
the capabilities it requires. I think it is important that tooling in
Eclipse stick to using the IResource layer as the layer they operate on
and let the repository provider (or any other tooling whose responsibility
it is to manage the available files) control the underlying file system.
If there are shortcomings or enhancements required then you should push to
get them in at the IResource level.
As for the current state of
Team support for linked resources, I think the best approach is to
enumerate some specific scenarios of how you see linked resources and
exclusions working with descriptions of what you need to do today to get
Team support and what you would like to see. It is also important to know
if you expect all the links to come from the same repository (or at least
repository type) or whether a project could contain content from different
repository types (obviously the later would be more difficult to
accommodate than the former).
Hope this
helps,
Michael