Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [platform-core-dev] RE: [cdt-dev] First gotchawithadd/excludechildren of FFS

Title: Re: [platform-core-dev] RE: [cdt-dev] First gotchawithadd/excludechildren of FFS

Any time Szymon can make it, I’ll be there. How about just adding it on to the end of the CDT BOF?

- Ken

From: "ext Schaefer, Doug" <Doug.Schaefer@xxxxxxxxxxxxx>
Reply-To: "CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Date: Thu, 13 Mar 2008 10:58:45 -0700
To: "CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Cc: "Eclipse Platform Core component developers list." <platform-core-dev@xxxxxxxxxxx>
Subject: RE: FW: [platform-core-dev] RE: [cdt-dev] First gotchawithadd/excludechildren of FFS

OK. Someone suggest a time them. I'll see if it's something I can drop. Or we can do it at a lunch time.


From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Sergey Prigogin
Sent: Thursday, March 13, 2008 1:39 PM
To: CDT General developers list.
Cc: Eclipse Platform Core component developers list.
Subject: Re: FW: [platform-core-dev] RE: [cdt-dev] First gotchawithadd/excludechildren of FFS

I'm for separating this meeting from CDT BOF. I'd like to spend CDT BOF talking about other issues (binding resolution, context assist, etc).


On Thu, Mar 13, 2008 at 10:06 AM, Schaefer, Doug <Doug.Schaefer@xxxxxxxxxxxxx> wrote:
Well  I'll be there and I want to talk about flexible file systems. Szymon can you  make this time?

We can also use lunch times, I guess.


-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx  [mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of Chris Recoskie
Sent: Thursday, March 13, 2008 12:55 PM
To: CDT  General developers list.
Subject: RE: [platform-core-dev] RE: [cdt-dev] First  gotchawith add/excludechildren of FFS

I guess it depends?

We  really have no idea who is showing up to the BOF or what they want to talk  about.  Do we know if Szymon can make that  time?


Chris Recoskie
Team Lead,  IBM CDT Team
IBM Toronto

             "Schaefer, Doug"
    <> >                                                  To
            Sent by:                   "CDT General developers  list."
            cdt-dev-bounces@e          <cdt-dev@xxxxxxxxxxx>
    <>                                                    cc

             03/13/2008 12:50           RE: [platform-core-dev] RE:
             PM                         [cdt-dev] First gotcha with
                                        add/excludechildren of FFS

             Please respond to
               "CDT General
             developers list."

Did we just want to do this  at the CDT BOF? I don't see much room in the schedule. The CDT BOF is 8:45 on  Wed. Thoughts?


-----Original Message-----
From: platform-core-dev-bounces@xxxxxxxxxxx
[mailto:platform-core-dev-bounces@xxxxxxxxxxx]  On Behalf Of Chris Recoskie
Sent: Wednesday, March 12, 2008 9:41 AM
To:  CDT General developers list.
Cc: platform-core-dev@xxxxxxxxxxx
Subject:  [platform-core-dev] RE: [cdt-dev] First gotcha with add/excludechildren of  FFS

Please count me in for such a  meeting.


Chris Recoskie
Team  Lead, IBM CDT Team
IBM Toronto

             "Schaefer, Doug"
    <> >                                                  To
            Sent by:                   "CDT General developers  list."
            cdt-dev-bounces@e          <cdt-dev@xxxxxxxxxxx>,
    <>                  <platform-core-dev@xxxxxxxxxxx>

            03/10/2008  10:27                                        Subject
            AM                         RE:  [cdt-dev] First gotcha with
                                        add/exclude children of FFS

             Please respond to
               "CDT General
            developers  list."

Copying the platform-core-dev  folks too. Is there someone from the Platform who could attend a meeting at  EclipseCon about flexible file systems. John A, will you be there? John is Mr.  EFS and we can use his guidance.


From: cdt-dev-bounces@xxxxxxxxxxx  [mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of Ken Ryall
Sent: Saturday, March 08, 2008 5:08 PM
To: CDT  General developers list.
Subject: Re: [cdt-dev] First gotcha with  add/exclude children of FFS


Can we get together at  EclipseCon to discuss this issue specifically? Do you know the right platform  people to invite to the meeting? We really need to piece together a  plan.

I?m sure you have enough to do so if you can tell me who to  recruit I can help organize the meeting.

Thanks - Ken

From: "ext  Schaefer, Doug" <Doug.Schaefer@xxxxxxxxxxxxx>
Reply-To:  "CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Date: Wed, 20  Feb 2008 18:44:57 -0800
To: "CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Subject: RE:  [cdt-dev] First gotcha with add/exclude children of FFS

So, you know.  The more I think about what you guys are saying, I'm realizing that the EFS  solution probably is the right one. The objective should be to turn the  IResource tree into a logical project view and to remove all notions that it  represents physical file layout. That unfortunately starts with the .project  and .cproject files, but I think there are tricks we can do there. The  .settings may be harder but let me sleep on that.

At any rate, this has  piqued my interest again and I'll work on reviving it and see where it goes.  I'll try to get a prototype working by EclipseCon so we can talk about it more  concretely.


From: cdt-dev-bounces@xxxxxxxxxxx  [mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of Brunauer, Walter
Sent: Monday, February 18, 2008 8:53 AM
To:  CDT General developers list.
Subject: RE: [cdt-dev] First gotcha with  add/exclude children of FFS

Hi Warren,

well, the confusion my  origin from the different meanings of what project setup is: for me, project  setup is not equal to build setup. I.e., on our projects, the build setup is  an independent step from the project setup. We intentionally separated this to  overcome all kind of issues you obviously experienced as well. And this is how  we see it:

1. Create a project at the desired location (everything  beneath this root location is part of the project, but it can be an empty  project just as well with linked resources added to it later). By default, the  build setup is identical to the project content (there is one build target,  linking/archiving everything together).

2. If (a) specific build  setup(s) are needed, it is possible to specify as many build targets with  arbitrary contents as desired. This approach separates the physical file  system layout from logical build layouts, and it even works beyond project  boundaries. IOW, no matter from where source files are pulled in (the same  projects, nested projects, outer projects, sibling projects), one is able to  specify any build setup exactly are needed, as long as all source files are  known to Eclipse (as  resources).



     From:  cdt-dev-bounces@xxxxxxxxxxx   [
     mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of
      Sent: Freitag, 15. Februar 2008  14:51
     To:  cdt-dev@xxxxxxxxxxx
      Subject: RE: [cdt-dev] First  gotcha with add/exclude children of  FFS

     Hi Walter,

      I forgot all about the absolute paths issue with linked   resources.
     I'll update the  wiki.

     I'm a bit confused about your comment  about this not  being a project
     setup issue.   We have our own builder as well, and it  will happily
      build whatever the build description says, whether those files   are
     under the project root or not.  We even  have our own project explorer
     view which shows a  logical representation of the project rather than
     the   physical file system layout.  But we still run into a lot  of
     issues when  files are not under the project  root - that is, when you
     can't get an IFile  for  them.

     We have a wide range of user types  from small application  developers
     to large system  developers.  In many cases, a users code base
      consists of hundreds of directories with thousands of source  files.
     In  such a source base, there are many  hundreds of build artifacts
     and almost as  many  "logical projects".  It is a huge problem for
      these users to be able  to create projects currently.  They  will
     typically have a few projects  going at a  time, but many times the
     natural project root for all  of them will  be the same.  We've found
     ways  to work around some of the other issues,  but not this one.   It
     sounds like perhaps you guys have found a   way.  Could you elaborate
     on how you setup  your  projects?


     From: cdt-dev-bounces@xxxxxxxxxxx   [
     mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of ext Brunauer,
      Sent: Friday, February 15, 2008 1:56 AM
     To: CDT   General developers list.
     Subject: RE: [cdt-dev]  First gotcha with  add/exclude children of FFS

      Hi Warren,

     FWIW, you did not  mention anything about linked  resources and
      absolute paths these persist in the .project file by default.   Again
     a big issue around linked resources in  combination with sharing
     project  within a team  (even without team support), and one more
     reason why  they appear  to be so cumbersome to handle. To me it seems
      many times one has  to unsell linked resources to users:  Whereas
     linked resources  are (kind of!) nice for  evaluation purposes
     (because, yes, in this case   you might not want to pollute your
     sources), as  soon as you start serious  development, you run into all
      kind of troubles. The hurdle to get everything  right from  the
     beginning is overwhelming for novices (e.g. its not  possible to
     change a linked resource to use a variable  later). Sorry, I don't
     know how to  add this to  the Wiki page...

     Having said that, the  scenario you describe is  really about having
     the  flexibility around build and indexer setup, not around  project
      setup, IMO.

     It's rather  classic: users have common  code they want to reuse in
      multiple applications - so they create one or  a set of libraries  out
     of it, within one or a set of projects. Of course,   indexing should
     be able to handle only code going  into these libraries, and
     optionally ignore the rest.  Then, they create their application
     projects, which   use the binary artifacts of the library project(s).
      Now it would be great if  they would have automatic support  for
     application linkage specification, i.e.  some  nice wizard or UI
     allowing to select the library  binaries of other  projects to be
     linked in,  without the need to specify it manually in the  linker
      options. And probably also desired: during application code
      development, the public API's of all used library projects should  be
     the only  thing they see WRT code completion,  etc. I guess, some UI
     would be needed for  this as  well.

     And now think of all developers in  the world.  Wouldn't it be great
     to give as many  of them the freedom to choose how to  achieve this?
      Either everything in one project, or one project per build   artifact,
     or one project per  module/application/product, or with nested
     projects...  its possible. Our commercial IDE based on CDT supports
      all this,  and we did not have to provide some EFS or work  with
     linked resources.  Well, we had to override  the build system, and
     this is IMO the place to solve   this in CDT as well.

     Again, I  don't see anything specific to project  setup. The issue
      around having the source tree polluted with project files - I   don't
     think this is the big thing. I would not  leave the Eclipse path in
     this  area at all and  allow to separate the project file from the
     project  location.  Its a very general paradigm of Eclipse, and I am
      pretty sure doing everything  differently will generate lots  and lots
     of problems in all kind of areas   (probably much more than you
     already identified),  unless you make it a new  Eclipse way
     (add/change  this in the platform, not in CDT, that  is).

      Just my 2 or 3 cents again,


           From:  cdt-dev-bounces@xxxxxxxxxxx   [
           mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of
            Sent: Donnerstag, 14. Februar 2008   23:35
           To: cdt-dev@xxxxxxxxxxx
            Subject: RE: [cdt-dev]  First gotcha with  add/exclude children
           of  FFS

           I've updated the  Wiki page
           some more thoughts on the  issue.  It would be great to get
            feedback  from other CDT users - both those shipping  C/C++
           IDE's and end users.    You'll see that I'm not convinced that
            the linked resources route is a  viable option.  Maybe we  can
           get the platform team involved  in the  discussion to help find
            the best route forward.


            From: cdt-dev-bounces@xxxxxxxxxxx   [
           mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of ext Schaefer,
           Sent: Friday, January  25, 2008 11:15 AM
           To: CDT   General developers list.
            Subject: RE: [cdt-dev] First gotcha with  add/exclude  children
           of  FFS

           I guess what my  investigation has shown me that the EFS
            solution and linked resources are pretty much identical. I
            really  noticed this when trying to  figure out how to persist
           the adds  and found  myself wishing I could add that to the
            .project file just like linked  resource are. And  they are....

           I think  all the issues that we have with linked resources would
            be equally as bad with the EFS solution,   possibly worse
           because the  EFS adds are hidden. The CVS one is a great
            example. I really doubt CVS would work with the EFS  solution
           either. And I  don't  want us to think EFS would be better since
            it's not in the platform  where we'll have a battle  getting
           changes. Any platform  changes required to  make linked resource
            work correctly would also need to be done for   EFS.

           So my hope  is to save the effort at implementing  the
            add/remove functionality since I believe that's already  there
           with  linked/hidden  resources. We can then focus on making
            linked resources  work where we need them and improving  the
           workflows. But this   really needs to start now.

            So, Warren, you've somewhat started a list of  workflows  that
           we'd like to support with  this solution. This is a great  place
            to start. I've created a Wiki page to start capturing  these.
           Please  feel free to  add more information, especially to the
            workflow section. When  we have that we may get a better  idea
           of which of the two solutions  will  work best.




            From: cdt-dev-bounces@xxxxxxxxxxx   [
           mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of
            Sent: Friday, January 25, 2008 11:22   AM
           To: cdt-dev@xxxxxxxxxxx
            Subject: RE: [cdt-dev] First  gotcha with  add/exclude children
           of  FFS

           We've been working  on Eclipse/CDT based products for  about
            three years now.  I'm sorry to say that the project model  is
           still not satisfactory for our  purposes.  We've tried many
            angles, but  are still stuck with some pretty serious
            limitations.  I've volunteered  to  investigate the EFS route to
           see  if it will help at all.  Based on  this thread I'm  assuming
           it  won't.

           Let me give you  a brief overview of how our users work,  and
            then discuss the problem we've run into.  I don't  think any of
           this  is  specific to our users BTW.

            Most of our users have existing code bases.  They  simply  want
           to "import" it into the IDE.   Others will create new  projects
            from our templates.  The new projects are created in  the
           workspace.  Imported  projects could be anywhere in the file
            system.  Often times they will import several projects from  the
           same  source tree.   This is where our biggest problem is.
            Let's say the  source base looks like  this:




            Because both projects  share code in the Common directory,  the
           logical root project directory  for  both Project1 and Project 2
            is C:\MyProjects\.  But in Eclipse you  can't have two  projects
           with the same root.   This is where the .project  and .cproject
            files are created.  So currently our users would  import
           Project1 with the natural  root (C:\MyProjects\), but Project2
            has to be  rooted at C:\MyProjects\Project2\.  This means  that
           any source/headers  from  the common directory are not under
            Project2.  This means those  files are not in the  project
           explorer for that project,  are not indexed,  etc..  We logged
            this against the platform -
            Basically if you put the .project file  anywhere, but have a
           project root   attribute, this would cease to be a  problem.

            Our first product actually always created  the  .project in the
            workspace, and for imported projects, would create links to
            files and folders.  We ran into so many  issues with this that
           we had to   change the model.  I don't recall all of the issues,
            but here's a list  of  some:

           - Version  control simply didn't work at  all

            - You can't make file system changes with  links.   For example,
           if you want to  rename a file or folder, or move a  file around,
            you can't do this with linked resources.  It only  changes  the
           link itself, not  the underlying  resource.

            - Creating new resources in a project with links is   confusing
           at best.   Let's say you have a project with a linked folder
            and file at the root.  If you create a new  file or folder at
           the root,   it is created in the workspace, not where the other
            folder/file are in the  file system.  But  if you create a new
           file under the  linked  folder, it gets created where you'd

            - The location of the .project/.cproject files are
            problematic.  Some users will want to keep  these in version
           control,   while others won't.  Those that do want them created
            in the source  tree, but those that  don't want them elsewhere,
           like  the workspace.  I  forget now why this was a problem with
            linked resources, but there was   something weird going on

           I suppose  it's worth noting that the last time we  really
            looked at this was in Eclipse 3.2, so some of this may  have
           been  fixed by now.   But I doubt it.  In general linked
            resources are  second class citizens.  Some IResource  API's
           just don't work for linked   resources.  Just search for
            references to IResource#isLinked for  "special handling".   I
           suspect that we'll run  into similar issues with  EFS though -
            see getLocation vs  getLocationURI.

            We also have the same issue that Doug is  trying to  address
           (hiding  some branches in a source tree).  This is much less of
            an issue for us though.  You can already  reduce the scope of
           the   indexer and the build.  The only real issue for us is for
            a very large  source tree, you're going  to get IResource's for
           everything,  which slows  things down quite a bit.  There is
            actually somewhat of a problem in  reducing  the indexer scope -

            The hidden attribute addition sounds  promising for  hiding
            resources under the project root, but doesn't really do
            anything to  add flexibility to the contents  of a project.  EFS
           sounds  like it would  though.  What I mean by that is, having
            resources under a project that  are  real resources, not linked,
           but  that don't live under the project root  in the file system.
            I've just started looking into EFS, so maybe  it's  a bit of
           wishful  thinking at this point, but I'm hoping we could create
            a  project anywhere, and when we create it we pass  the URI
           location from our   own EFS.  Then when asked for the children,
            we could return URI's for  files from anywhere in the  file
           system, or on other machines  even.   This would seem to hold
            the potential for working around the issues listed   above.
           We'd basically have  an EFS map from what we want under a
            project to the actual file  system.

            So hopefully some of the experts can chime in   here.  Is my
           hope for  EFS unrealistic?  Is there a different  approach we
            should look at?


            From: cdt-dev-bounces@xxxxxxxxxxx   [
           mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of ext Brunauer,
           Sent: Friday, January  25, 2008 1:47 AM
           To: CDT   General developers list.
            Subject: RE: [cdt-dev] First gotcha with  add/exclude  children
           of  FFS


           after  reading this rather long thread, I'll  decided to throw
            in my personal opinion.

            I consider this approach to work against one  of  the most
           general Eclipse  platform paradigms, where a project is defined
            to be a root directory and everything in it. IMO, the  more
           workarounds are   introduced against this paradigm, the more
            problems will be faced, and the  more  incompatibilities (or at
           least  unawarenesses)  created.

            Isn't the whole problem you try to solve here  rather  about
           what files should go into  the build (and probably into the
            indexer) than what files are part of a project? I understand
            that CDT has no  separation of what a  project and what the
           build input  is (well, IIRC one can  exclude specific files from
            the build, but in general, the project content   defines the
           build input,  right?).

           In our  commercial IDE, we separated this. This not  only
            introduced much more powerful build setup capabilities  in
           general, but  especially  enabled users to setup build artifacts
            with arbitrary  contents (think of sources being  compilable
           with different compiler  flags  for different build artifacts,
            build input exclusion patterns, build input  from all over  the
           workspace, multiple build  artifacts within the same  project,
            reusable build artifacts accross project boundaries, etc.,
            etc.,  etc.). BTW, we call this build  system flexible managed
           build -  because  that's what it is:-)

            Of course, one can setup CDT projects as of today  to  exactly
           contain what is desired  (with the help of linked resources).
            However, I find linked resources to be cumbersome and error
            prone, though  many of our customers  start out with them during
            evaluation as well, mostly  because they are looking for a  way
           to achieve what they did in  the past with  other non-Eclipse
            based IDEs, but sooner or later I know of lots of  them
            realizing its much easier to use the  features of our flexible
           build   system instead, especially if projects need to be shared
            in a team.  And now think of virtual  file systems, the
           potential  complexity of  these, hidden assumptions,
            restrictions, etc. Sounds worse than linked  resources to  me.

           I guess, the point  I am trying to make is:  whatever you decide
            to do, make it understandable  and transparent (and  of course
           as simple as possible to  use) for  the user.

            As said, just my 2 cents,


                  From: cdt-dev-bounces@xxxxxxxxxxx   [
                  mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of
                  Schaefer,  Doug
                  Sent: Donnerstag, 24. Jänner 2008 23:17
                  To: CDT  General developers  list.
                  Subject: RE: [cdt-dev] First gotcha  with add/exclude
                  children of  FFS

                  Jogging through the code, it really looks like the
                  HIDDEN feature is what I was  looking for. What I haven't
                  found yet is UI  to make a resource hidden or a  navigator
                  filter to show hidden resources  (in case you want to
                  bring them back). Is  this planned?

                  Assuming we have the core features available to link   in
                 and  hide resource, I think we still have workflow issues
                  that need to  be addressed. I  like Ken's idea of a file
                  that controls the  linking/hiding. We could have  an
                  import/export mechanism for setting up  projects based on
                  this file. A nice  wizard for creating the file would
                  also be good, similar to the way the way the export  file
                 system  wizard  works.

                  Given this, we may be further along than we  thought
                 (BTW,  the hidden stuff seems to have been added in 3.4



                  From: cdt-dev-bounces@xxxxxxxxxxx   [
                  mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of
                  Schaefer,  Doug
                  Sent: Thursday, January 24, 2008 2:51 PM
                  To: CDT  General developers  list.
                  Subject: RE: [cdt-dev] First gotcha  with add/exclude
                  children of  FFS

                  Thanks Michael/Szymon,

                  Is there a bug describing the isHidden   feature?


                  From: cdt-dev-bounces@xxxxxxxxxxx   [
                  mailto:cdt-dev-bounces@xxxxxxxxxxx] On  Behalf Of Michael
                  Sent: Thursday, January 24, 2008 11:37 AM
                  To:  cdt-dev@xxxxxxxxxxx
                  Subject: RE: [cdt-dev] First  gotcha with  add/exclude
                  children of FFS

                  Doug et al,

                  Szymon is really the person you want  to bug  on this but
                  I'll throw in my 2 cents ;-)  First, I have to say that  a
                 solution at  the IResource level (e.g. using linked
                  resources and the new  hidden folder support)  is
                 infinitely  better from a repository provider  perspective
                  than an EFS based solution.  You  may not get all the Team
                  support you want at the IResource level but a solution  at
                 the EFS  level  would certainly break the existing CVS
                  client since the CVS client isn't   EFS aware to any great
                  extent. For instance, if you tried to hide a  folder
                 using  EFS, the CVS client would probably try and recreate
                  it the next time  you performed  a Team>Update. It is also
                  important to note that the  Platform does not provide  all
                 the hooks  required by repository providers  and I know of
                  at least one provider that has  resorted to using it's own
                  EFS implementation under projects that are mapped to  that
                 provider  to get  the capabilities it requires. I think it
                  is important that tooling in   Eclipse stick to using the
                  IResource layer as the layer they operate on  and let  the
                  repository provider (or any other tooling whose
                  responsibility  it is to manage  the available files)
                  control the underlying file system.  If there are
                  shortcomings or  enhancements required then you should
                  push to  get them in at the IResource  level.

                 As  for the current state of  Team support for linked
                  resources, I think the best approach  is to  enumerate
                  some specific scenarios of how you see linked resources
                  and  exclusions  working with descriptions of what you
                  need to do today to get  Team support and what  you would
                  like to see. It is also important to know  if you expect
                  all the links to come  from the same repository (or at
                  least  repository type) or whether a project  could
                 contain  content from different  repository types
                  (obviously the later would be more difficult  to
                  accommodate than the former).

                  Hope this  helps,

cdt-dev  mailing list
cdt-dev  mailing list

platform-core-dev  mailing list
cdt-dev  mailing list

cdt-dev  mailing list
cdt-dev  mailing list

cdt-dev mailing list

Back to the top