[ecf-dev] Re: More interfaces on remote OSGI services page

[Scott Lewis <slewis@xxxxxxxxxxxxx>]

Hi Ken,

Ken Gilmer wrote:
> These two documents shine brightly for me in regards to distributed OSGi:
>
> flowSGi - Collaborative middleware for mobile devices
> http://www.iks.inf.ethz.ch/publications/files/flowSGI.pdf  (Jan would 
> make an ideal ECFer...)

Thanks...I'll check it out and attempt to contact Jan.  If anyone 
already knows Jan

> A Note on Distributed Computing
> http://research.sun.com/techrep/1994/smli_tr-94-29.pdf

Yes...this is one of my favorite articles on distributed computing.  
They go into a very clear and concise explanation of why network 
transparency...although an appealing idea...is not necessarily a good 
idea...and why it usually makes sense to put as much control of 
networking...including partial failure handling, etc...into the hands of 
the application programmer (i.e. not try to hide the fact that there is 
an unreliable, relatively slow network...but rather expose to the 
application failure detection and ability to recover from partial 
failure in an application specific manner).  For another seminal paper 
along these lines see Six Misconceptions about Reliable Distributed 
Computing:  http://www.cs.cornell.edu/vogels/papers/ew98.pdf

My take-away lessons for me of this research and own experience with 
building frameworks for app development is that 1) Network communication 
abstractions (e.g. sockets, pub-sub messaging, rpc, IContainer, failure 
detection, etc) should be exposed to rather than hidden from 
applications; 2) attempts to do 'transparent' distribution of object 
graphs (i.e. hide network distribution and remote method invocation from 
the app) don't really work very well, because networks fail, are slow, 
etc., and devising a generic response to network failures is 
hard/impossible...that is, it seems better to allow the application to 
*know* (be notified) of such failures and then allow the application to 
decide how it should recover.  Because distribution of state *without* 
knowledge of appropriate application behavior (i.e. at OS level) is very 
hard/impossible to do 'appropriately' for all applications.

But exposing things like partial failure and asynchrony at the API level 
are difficult...because this adds unfamiliar concepts to any API.  
Therein lies the challenge, I think...create APIs that are as simple as 
possible, without hiding too much from programmers/applications.

> Regarding this topic as a whole, an implicit transition has been made 
> from "Eclipse" to "OSGi".  They are different worlds with different 
> problem domains, and I have been pondering the pluses and minuses of 
> ECF in a pure OSGi context.  The value of ECF in the Eclipse context 
> is clear, but it's value (in relation to other solutions) at the OSGi 
> level is ambiguous to me.  

Yes.  While Eclipse is an application, OSGI is a runtime...and in some 
senses like an OS.  Inter-process communication (whether RPC or asynch 
messaging or whatever) is by definition a cross-runtime concern...and 
ECF as a framework has been focused on supporting IPC by applications 
rather than as part of the OS.  Partially for the reasons expressed in 
the papers above.  Note also that ECF has a model for object replication 
in a distributed group (shared objects), but the replication is *not* 
automatic (i.e. it's explicitly controlled by the application).  ECF's 
shared object container also include the notion of a failure 
detector...which is used to detect and inform containers and shared 
objects within those containers of remote process or network failures 
and subsequent group membership changes (represented as events like 
IContainerDisconnectedEvent, etc).  It doesn't do much automatically in 
terms of replication (except 'clean up' non-primary replicas when 
primary becomes disconnected)...but rather notifies the shared 
object/application and allows it to deal with the given failure.

> It's tricky because OSGi has some very compelling native services that 
> seem ripe for utilizing in a distributed manner (Event Admin, Device 
> Service).  Again from yesterday's phone conversation it really depends 
> on your application.  But given "A Note on Distributed Computing", 
> does it make sense to represent a remote service transparently in an 
> OSGi runtime?  

I think of the OSGI services layer as an application-level API...to 
allow bundles to expose services/APIs to one another in a 
structured/consistent way (with the appropriate clean up, dealing with 
classloading/dependencies, etc).  So with the musings here 
http://wiki.eclipse.org/index.php/Remote_OSGI_Services I've simply been 
thinking about how to expose these app-level services across process 
with ECF.  I'm not sure that this (OSGI services) is the right level for 
inter-process communication, but at first blush it seems reasonable to 
me.  At least all of the OSGI services structure for registration, 
lookup, and clean up generalizes well...i.e. going cross 
bundle/classloader is not *too* far from going cross-process :).

> Is there merit in distinguishing Eclipse-ECF and OSGi-ECF?

Well, I think there is merit to distinguishing parts of ECF...ECF core 
(useful in both domains), ECF datashare (useful in app), ECF presence 
(useful in app), ECF 'remote services' (useful to app that uses the OSGI 
services layer), etc. 

Thanks for note.

Scott


> -Ken
>
>
> On Jul 10, 2006, at 5:46 PM, Scott Lewis wrote:
>
> > Hi Bob, Ken, and all,
> >
> > This afternoon I put some more of the ideas for OSGI remote service 
> > API that I've been playing around with here:
> >
> > http://wiki.eclipse.org/index.php/Remote_OSGI_Services
> >
> > Note as usual for ECF :-) this says exactly nothing about how these 
> > APIs would be implemented...e.g. they could be implemented with some 
> > existing RPC mechanism, or via RPC built on asych messaging (e.g. ecf 
> > generic).  Clearly I think of missing functionality as a feature :).  
> > Also, as discussed during the conf call today the actual 
> > implementation bundles (e.g. whether they are dynamically 
> > loaded/installed or rather must be there to begin with) is left 
> > unspecified...meaning that (for example) a call to 
> > getRemoteServiceReferences could actually retrieve the bundles 
> > holding the proxies associated with the given remote service.  I 
> > think that's a pretty exciting idea, actually.
> >
> > Also note that the intention here was to present to remote clients an 
> > OSGI service interface that 'looks' a fair amount like the OSGI 
> > 'local' service interface (e.g. BundleContext.registerService, 
> > BundleContext.getServiceReferences, BundleContext.getService, etc).   
> > This may/may not be a good idea...as Ken hinted at during the 
> > call...that is, it's not clear whether a remote service interface is 
> > best...or some way to access remote bundle code, or certain packages, 
> > within bundle, etc.  The process-local OSGI service concept, though, 
> > is at least one that is standardized by OSGI itself...and used in 
> > Eclipse and other RCP apps.
> >
> > Any thoughts/comments/criticisms/'you are crazy's appreciated.
> >
> > Scott