Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[eclipse-incubator-e4-dev] [resources] EFS, ECF and asynchronous 
Hi Scott,

> 2) Asynchronous access to files/resources is desirable and in 
> some cases necessary (for some use cases)

Could you cite a use case where async access is necessary?

I think that (assuming all synchronous methods have progress 
monitors for cancellation, which is the case in EFS), the 
only difference between sync and async access is 
  (1) the number of Threads in "wait" state,
  (2) locking of resources while Threads synchronously wait,
  (3) potential for coalescing multiple requests to the
      same item in the case of asynchronous queries.

In the asynchronous case, no Threads are waiting and resources
*may* be unlocked until the callback returns, but this unlocking
of resources needs to be carefully considered in each case. 
Does the system always remain in a consistent state? RESTful
systems ensure this by placing all state info right into the 
request, which is a great idea but likely not always possible.
It's not only a matter of the API being complex or not. The fact 
is that the concept of being asynchronous as such is more flexible,
but also requires adopters to be more careful, or at least think
along different lines.

I also think that we should look into the need for being 
asynchronous or not separately for the kinds of requests:
  (A) Directory retrieval (aka childNames())
  (B) Full file tree retrieval
  (C) Status/Attribute retrieval for an individual file
  (D) File contents retrieval

For (D) we already use Streams in EFS, which can be 
implemented in an asynchronous manner. What's currently 
missing in EFS is the ability to perform random access, 
like the JSR 203 SeekableByteChannel [1]. Interestingly, nio2 
has both a synchronous FileChannel [2] and 
AsynchronousFileChannel [3].

For (A), (B), (C) I'm not sure how much we would win from
an asynchronous variant, since I'd assume that not much
work could be done (and not much resources freed) while
asynchronously waiting for their result anyways. But perhaps
I'm wrong?

> 3) Using (e.g.) adapters it's not necessary to force such an API on 
> anyone (rather it can be available when needed)

Hm... so, let's assume that client X wants to do something 
asynchronous. So it does
   myFileStore.getAdapter(IAsyncFileStore.class);
some file systems would provide that adapter, others not.
What's the client's fallback strategy in case the async 
adapter is not available?

I'm afraid that if we use such adapters, we end up with the
same code in clients again and again, because they need some
fallbacks strategy. It seems wiser to place the fallback 
strategy right into the EFS provider, since it is always 
possible to write a bridge between a synchronous and an
asynchronous API in a single, generic way.

Therefore, I'm more in favor of determining what APIs we want
to be asynchronous, and just adding them to EFS. The adapter
idea could be used for adding provisional API, but the final
API should not need that.

> > To that extent, let's start assuming that files are quick 
> and local. And
> > let's investigate how we could leverage ECF to support remote file
> > systems. If that doesn't meet our needs, we can always add 
> async later.

I'm not sure if this is a good strategy. It seems to lead
towards more and more separation of local vs. remote -- 
which, I think, leads to either duplication of code in the 
end, or non-uniform workflows for end users.

Let me draw some sceanrio of what the world could look like 
in 10 years: with the Internet getting more and more into
our lives, you'd want to use an Eclipse based product to 
dive into some code base that you just found on the net.
Without downloading everything in advance. Or you browse into
some mp3 music store. Add some remotely hosted Open Source 
Library to your UML drawing just by drag and drop.

I think that users will more and more want to operate on
remote networked resources just the same as on local 
resources. E4 gives us the chance to try and come up with
models that support such workflows in a uniform way. Let's 
not throw away that chance prematurely.

I agree that we need to start on concrete work items
rather than endlessly discussing concepts. But as we
start on these work items, let's keep the concept that
things may be remote in our minds.

> Sounds reasonable.  Just as an aside: I think there's a lot 
> of potential to use asynchronous file transfer + replication
> to do caching of remote resources.

That's a great approach, especially if it works on the 
file block level (such that random access to huge remote 
files can be cached). Again, one thing that's missing from EFS
today is random access to files. Does ECF have it?

[1]
http://openjdk.java.net/projects/nio/javadoc/java/nio/channels/SeekableB
yteChannel.html
[2]
http://openjdk.java.net/projects/nio/javadoc/java/nio/channels/FileChann
el.html
[3]
http://openjdk.java.net/projects/nio/javadoc/java/nio/channels/Asynchron
ousFileChannel.html

Cheers,
--
Martin Oberhuber, Senior Member of Technical Staff, Wind River
Target Management Project Lead, DSDP PMC Member
http://www.eclipse.org/dsdp/tm

Back to the top