[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [eclipse-incubator-e4-dev] [resources] EFS IFileTree
|
Here is a different idea that might also
work:
1. Initiate breadth-first-search retrieval of IFileTree
data
2. While retrieving data in a stream, allow to send
a
blacklist of folders to the remote as
evaluate by a
local programmatic
filter
3. As soon as the remote receives the blacklist,
it
prunes the BFS tree that it's currently
working on
by the root nodes
received.
Sounds complex, but would allow for the most
efficient
evaluation of complex filters like "return folders
under
Team support only" even on a remote file
system.
Cheers,
--
Martin Oberhuber, Senior Member of Technical
Staff, Wind River
Target Management Project
Lead, DSDP PMC Member
The advantage of the Directory Stream approach is that
clients can
see / work on intermediate results before the traversal
is complete,
and they can cancel the Stream retrieval if the
intermediate
results don't match their
expectations.
I
agree that programmatic approaches like visitors or
arbitrary
coded filters are problematic when the receiver of the
query
is
remote (Java does support things like remote object
execution,
but
not all server-sides support Java...and local evaluation
of
the
filter might not always be a good fallback). LDAP filters
might be an interesting thing to explore, I haven't heard
about
these before but anything Standard seems like a good
thing
to
look at.
Here
is a different idea that might also work:
*
Initiate
Cheers,
--
Martin Oberhuber, Senior Member of Technical
Staff, Wind River
Target Management
Project Lead, DSDP PMC Member
IFileTree doesn't have a depth
because it was really intended to optimize the DEPTH_INFINITE case. For
DEPTH_ZERO or DEPTH_ONE, it doesn't provide much advantage. I agree though
that it could be interesting to support different scoping to pick up smaller
subtrees, or to have some termination condition. The purpose was really to
cut down on round-trips, so I'm not sure you'd get the same effect with a
visitor or directory stream approach. Perhaps filters could be passed in a
serialized form (like an LDAP filter), so that it can be interpreted on the
machine where the file system lives, to avoid passing unnecessary data over
the wire.
John
"Oberhuber, Martin"
<Martin.Oberhuber@xxxxxxxxxxxxx>
Sent by:
eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx
10/20/2008 12:49 PM
|
Please respond
to
E4 developer list
<eclipse-incubator-e4-dev@xxxxxxxxxxx> |
|
|
To
| "E4 developer list"
<eclipse-incubator-e4-dev@xxxxxxxxxxx>
|
|
cc
|
|
|
Subject
| RE: [eclipse-incubator-e4-dev]
[resources] EFS IFileTree |
|
Hi John,
One disadvantage of the IFileTree today is that the file
tree to be
returned must terminate at
some point, and not the slightest
amount
of information is returned if it does not terminate.
This makes it unusable for
file systems that can be endless
(like
the Internet), or even very very large with the user only
being interested in parts of the file tree. Even
refreshLocal
performs a
breadth-first-search and can be limited by the
depth to visit. fetchFileTree() does not support that.
One option for
improving this situation could be if the
fetchFileTree() method would not return a snapshot like
the IFileTree, but rather return a Stream of
IFileStore / IFileInfo
objects that the
client can cancel at any time. A
breadth-first-search algorithm for returning these nodes might
be preferred, though I'd also see some potential
for allowing
arbitrary ordering of
returned nodes with some concept of
allowing a node to be "incomplete" e.g. a folder node with
not all children returned yet.
In JSR 203, the
Path#newDirectoryStream()
[1] and
Files#walkFileTree() [2] methods go in this direction.
[1] http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Path.html#newDirectoryStream(java.nio.file.DirectoryStream.Filter)
[2] http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Files.html#walkFileTree(java.nio.file.Path,%20java.nio.file.FileVisitor)
Cheers,
--
Martin Oberhuber, Senior Member of Technical Staff,
Wind River
Target Management Project Lead,
DSDP PMC Member
http://www.eclipse.org/dsdp/tm
From:
eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx
[mailto:eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx] On Behalf Of
John Arthorne
Sent: Friday, October 17, 2008 7:40
PM
To: E4 developer list
Subject:
[eclipse-incubator-e4-dev] [resources] EFS IFileTree
I forgot to mention
this during the meeting, so I just wanted to mention a little know EFS
interface called IFileTree. The idea of this interface was to allow for
batched interaction with an entire file sub-tree. This prevents the large
number of round-trips needed when a client needs to walk over an entire
subtree of a slow/remote file system. It allows you to get a snapshot of an
entire remote tree state in a single round-trip. In some experiments we did
back in the day, this dramatically sped up certain operations like
refreshLocal over a high latency remote tree. I mention it only because it's
probably an under-exploited concept that could perhaps be expanded upon to
improve performance in remote resource scenarios. I could imagine expanding
the idea to allow a client to queue up a whole batch of file changes, that
could be fired off in a single round-trip to the remote system for
processing.
John_______________________________________________
eclipse-incubator-e4-dev
mailing
list
eclipse-incubator-e4-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/eclipse-incubator-e4-dev
