On 10/19/2014 11:18 AM, Wim Jongman wrote:
I guess I forgot what the question was ;)
On Sun, Oct 19, 2014 at 3:42 PM, Wim Jongman
<wim.jongman@xxxxxxxxx <mailto:wim.jongman@xxxxxxxxx>> wrote:
Hi,
Suppose I want to stream data from a remote service to
myself. I am now thinking that this is polling only. I call
the remote service to get any available data. However, if the
remote service does not have data this leads to either
locking (the remote service waits until data comes available)
or unwanted network traffic (the remote service says "I have
nothing"). I can use Async calls but this still requires me
to poll the Future to find out if there is data available.
To continuously push data from the remote server when it
becomes available I need to "reverse" e.g. I have to make a
service available that the data provider can consume and it
can then push whenever new data is ready without me having to
poll (either through a Future or any other primitive way).
I wish I could consume a remote service and then set a
callback that it could call whenever data is available.
Yes. A few thoughts/ideas here below. I wish I could say to
you that your use case is trivially supported with OSGi Remote
Services (any implementation...or specifically ECF's), but in
truth I think that at the root is a slight mismatch between the
'remote procedure call' (RPC) model underneath OSGi Remote
Services and what you are looking to do.
First, with ECF's support for CompletableFuture, it is possible to
easy to define an asynchronous/non-blocking remote method
invocation, e.g. see [1]. But this isn't exactly what you are
looking for, at least partially because it only completes *once*.
But it does provide what I think of as a very nice way to express
asynchronous/non-blocking remote services.
Let's say we have declared a service API that looks like this:
void myMethodCall(String s, SomeCallback callback)
with the behavior being that 'callback can/is' called potentially
many times. An example is all the Eclipse methods that have
IProgressMonitor as the final argument in the method call.
The question is: How to implement this remotely/in a distributed
app? As you know, it's possible to pass method arguments and
return values either by reference or by value. For most remote
method calls passing by value is the norm (e.g. 's' argument in
the above), but it is possible to implement remote references,
allowing 'callback' instances/object references to be referred
to/called from out of process (e.g. a remote service host).
For example, Java's RMI is an implementation that has call by
reference. And it would be possible using RMI and/or other
call-by-reference distribution to implement a provider that
supports the above callback structure. For example, an RMI-based
provider could (pretty) easily be created for OSGi remote services.
But call-by-reference can/does end up being very complex in the
real (distributed) world...for a number of reasons. One reason is
that unlike local/memory object references, remote references
frequently fail and when they do it's often unclear what should
happen. Another reason is that remote references make garbage
collection particularly complex (since not all object references
are in local memory). A third reason is that the network can
introduce blocking that is problematic (for example, with Eclipse
EFS if a remote file system is used the calls to the
IProgressMonitor can block/be very slow...meaning that Eclipse
can/does just 'freeze up' for long periods). So in any event,
the distributed systems world has kind of moved away from fully
pass-by-reference for RPC systems, because of these and other
complexities.
Another common way to implement this sort of desired behavior is
with event-driven approaches/asynchronous messaging. One example
is the ECF filetransfer API...another is the ECF datashare and
shared object APIs, as well as JMS, vert.x, akka, zeromq, mqtt,
javagroups, etc...all of the asynchronous/messaging-based middleware.
More than an implementation difference, I think what you are
asking for implies a different sort of API than the 'normal' RPC,
which I summarize as: A thread calls a method synchronously and
method returns a single value to same thread. What I think you
are really asking for is a simple way to express a behavior that
isn't clearly expressed with a single method call (RPC), e.g. send
a message and some data, return from this
immediately/non-blocking, and receive one or more (remote)
callbacks (with arbitrary data). IMHO this is awkward for the
remote procedure call/OSGi remote service model in general, and it
gets really complicated when you factor in the distributed systems
issues of (e.g.) partial failure (what should happen if one of n
callbacks fails?). And then things get *really* complex when we
are talking about > 2 group members.
There are of course other ways to deal with this sort of behavior
e.g. callbacks can be 'client-exported' remote services., the
'remote whiteboard pattern' like the chat example that you and
Markus created, etc. IMHO these approaches have their own
complexities (e.g. how to dynamically associate the callback
remote service instance with the service interface, how to do
reliable discovery of the 'whole/associated thing', what happens
when things fail because of network unreliability, etc.). But
even so, there could probably be more/better standardized API
and/or meta-data specifically for doing the 'remote whiteboard
pattern'.
It's my hope that we and others can/could eventually create
standardized API (e.g. MultiCompletableFuture?, remote streams,
standardized event/messaging models, replicated data structures,
or collectors, etc :) that would help deal with this and similar
use cases. ECF has much of the underlying support for different
approaches (e.g. asynchronous messaging, replicated data models,
etc), meaning that we can easily try out/implement a variety of
approaches.
My $0.03.
Scott
_______________________________________________
ecf-dev mailing list
ecf-dev@xxxxxxxxxxx <mailto:ecf-dev@xxxxxxxxxxx>
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
