Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [eclipse-incubator-e4-dev] [resources] Lazy Resources

Hi Doug,

>> So I'm opposed to suggestions like pushing meta info and markers and
delta notifications to this layer.

If you are referring to the mentioned Monitoring API, this is not about
moving delta notifications to EFS.
But John A. explained it some mails earlier.

I've been investigating the branched FS issue. iFS (integrated file system
on i5/OS) is indeed very different from what I know already about
However it seems like a small change in EFS API will be enough to support
such filesystems.

Szymon Brandys
Staff Software Engineer
(+48 12) 628 9857

IBM SWG Lab, Cracow, Poland
IBM Polska Sp. z o.o. oddział w Krakowie
ul. Armii Krajowej 18
30 -150 Kraków

NIP: 526-030-07-24
Sąd Rejonowy dla Warszawy, XIII Wydział Gospodarczy KRS
KRS 0000012941, Kapitał zakładowy: 3.073.600 PLN

             "Schaefer, Doug"                                          
   >                                               To
             Sent by:                  "E4 developer list"             
             eclipse-incubator         <eclipse-incubator-e4-dev@eclipse.o
             -e4-dev-bounces@e         rg>                             
             2008-10-17 04:08          RE: [eclipse-incubator-e4-dev]  
                                       [resources] Lazy Resources      
             Please respond to                                         
             E4 developer list                                         

I'm also worried about complexity. I think we need to be careful about what
we push down to the file system layer. And I think we do need to keep them
stateless, or we're losing all the advantages of making this a separate

So I'm opposed to suggestions like pushing meta info and markers and delta
notifications to this layer. That's what resources are for. Resources for
the most part have served us well. Our initial goals were to make them
flexible. All we were really trying to do is loosen the bonds between
resources and the underlying file systems. And that really meant not
assuming that the file system parent/child relationship matched the
resource parent/child relationship.

I do think we need to add features to EFS to allow it to more accurately
model file systems and to allow operations that clients of the API need to
perform there in a generic manner. For more file system specific
operations, adapters could be deployed to allow clients easy access to the
real objects underneath. But that should be all, to keep it simple.

Prototyping the APIs and file system implementations will help guide us on
the cleanest solution that meets our needs. And once we get the e4 project
provisioned we can get started on that.


 From: eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx [
 mailto:eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx] On Behalf Of
 Oberhuber, Martin
 Sent: Thursday, October 16, 2008 2:32 PM
 To: E4 developer list
 Subject: [eclipse-incubator-e4-dev] [resources] Lazy Resources

 Hi all,

 As mentioned on the E4 call today, I'd like to bring up the
 idea of "lazy resources" once again.

 I had mentioned this at the end of my E-Mail on Oct 7 already,
 but it was somehow hidden beneath the other stuff.

 The background is, I noticed that we were talking about pushing
 down a variety of stuff from the Resource layer into the File,
 System Layer. Which might work for some stuff (like meta info,
 and markers though I'd not be sure about the life cycle of markers
 when a file gets renamed) and certainly won't work for others such
 as delta  notifications (which just won't work without state).

 So I was wondering why we don't do it the other way round, and
 allow a kind of IResource that is more loosely connected to the
 Workspace (by means of having been visited before, like with an
 external editor), and that's not eagerly refreshed like the resources
 we know in the workspace.

 Today, Resources are problematic with EFS-shared slow, remote,
 huge file systems sice the eager deep refresh would generate
 masses of data that's not necessary. We should think about a
 kind (flag) of IResource that's more loosely connected to the

 Does that make sense? - McQ argued that he's concerned about
 making the (currently easily understood) resource model overly
 complex, and about unclear user experience with such afeature.

 I could imagine using Lazy Resources for
   * Object files when no incremental build is desired
     (we don't care about update notifications in this case)
   * Static, frozen, read-only reference file systems where we
     *know* nothing will change

 In some sense, such a lazy resource is nothing other than
 a Linked Resource to a file in a hidden project. Could we
 solve this in a more elegant manner?

 Discussion is opened, any thoughts??

 Martin Oberhuber, Senior Member of Technical Staff, Wind River
 Target Management Project Lead, DSDP PMC Member.

 From: eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx [.
 mailto:eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx] On Behalf Of
 Oberhuber, Martin
 Sent: Tuesday, October 07, 2008 7:16 PM
 To: E4 developer list
 Subject: [eclipse-incubator-e4-dev] [resources] File system

 Hi all,

 I had some thoughts about the Strawman proposal, and the file system
 layer in particular.
       We have a requirement to extend usability of Eclipse tools beyond
       the Workspace. Bugs are open which request, for instance,
       capabilities to Search files and folders outside the workspace, open
       editors, add markers, ... apparently, we'll want to do all that on
       the Filesystem layer.
       Given that, the Filesystem layer must be stateless (we cannot
       maintain state in memory for a tree that can become arbitrarily
       large, since that wouldn't scale). The Filesystem layer must take
       its information from the filesystem alone, and nowhere else. Which
       seems to tie in nicely with ideas of having the FS layer RESTful.
       If the Filesystem layer is stateless, we cannot push down any
       resource deltas,, since these require state ("before" vs "after" the
       change). The Resources (Project) layer would remain the one which
       holds state just as it does today.
       I like the idea of pushing down metadata such that (a) markers can
       live outside the workspace on FS objects, and (b) file system
       capabilities for storing metadata such as Encoding or content type
       can be leveraged. Perhaps that metadata layer could even be totally
       separate from both FS Layer and Resource layer, linked with them
       through URI as the identifier, and some resource delta callbacks for
       lifecycle management. The other option is to leave it with the
       Resource layer, but make it lazy (see below).
       This brings up the question, where we really need to beef up the FS
       layer? I actually don't see much need for this, except for
       (a) adding asynchronous support if needed ... though that brings up
       other questions (see my other E-Mail), and
       (b) adding an IFileStore#getCanonicalPath() API which we clearly
       need for Alias resolution.
       I think that we can not have full Alias Management on the FS Layer,
       1.)  one requirement of Alias management is that given some file X,
       you need to know "who else links to X?".
       2.)  Now that kind of "reverse lookup" of symbolic links is not
       supported by file systems, so it must be solved in code.
       3.)  That, again, requires that clients have "expressed interest" in
       X before, which is adding state to the file system, which we cannot
       have on the FS layer.
       I think that we need to keep Alias Management on the
       Resource/Project layer, supported by the getCanonicalPath() API on
       the FS layer. In order to still support Alias Management for items
       outside the workspace (that have been looked at before), we'll
       probably want some "lazy addition to Workspace" paradigm which adds
       files and folders to the workspace as they are being visited (and
       probably removes them again after some time with an LRU paradigm).
 Now that being said, it looks for me as if the necessary enhancements on
 the FS layer could even be done in the Eclipse 3.5 Stream (adding

 Or am I missing any requirements on the FS Layer?

 Martin Oberhuber, Senior Member of Technical Staff, Wind River
 Target Management Project Lead, DSDP PMC Member
 eclipse-incubator-e4-dev mailing list

Back to the top