|Re: [ecf-dev] RPC for Remote P2 repositories|
Hi Thomas, Thomas Hallgren wrote:
Hi,I'm implementing a remote service for p2 repositories. Initially, I chose JSON as the RPC mechanism since its simplistic, very lightweight, and fairly mature. We've also had some good experiences with it when we did the AOL XDrive implementation in the Spaces project.In this case I have an Equinox based OSGi framework on both sides and perhaps it would be far more efficient to use ECF remote services? I have the following requirements:1. I must be able to pass some objects by value that are neither java.io.Serializable nor standard java beans. The only way to serialize/deserialize them is to register custom serializers. Typical examples are the p2 ArtifactDescriptor or ArtifactKey.
ECF would support this in a similar manner...i.e. with custom serializers within a provider that supports the remote services API...and this would depend upon the flexibility exposed by whatever the provider(s) uses to do serialization.
2. It must be possible to register custom serializers based on interfaces (i.e. IArtifactDescriptor).
As with above, some providers would support this, others would not (depending upon the serializers being used by that provider). The remote services API doesn't specify/restrict this at the API level.
3. I need to pass some objects by reference. Either the RPC mechanism must manage garbage collection internally or I must have access to some API that allows me to do an efficient implementation.
Again, if a provider supports passing by references (e.g. an RMI-based provider) then this is transparently doable (i.e. without API changes).
But I admit I'm not at all a big fan of pass by reference in RPC interfaces for a straightforward reason that you give: distributed garbage collection with strong (remote) references is *extremely* hard to do below the application level IMHO...and even when done correctly it doesn't recognize the needs that many apps have for things like (e.g.) notification upon network/remote failure. But again...we have JMS-providers (e.g. activemq) that I believe support both customizable serialization, pass-by-reference, and distributed garbage collection (I would have to verify to be sure)....these are at http://ecf1.osuosl.org.
My current JSON based implementation uses java.lang.reflect.Proxy instances in the client, registered in a WeakHashMap. I then keep track of object-id's that are no longer used and periodically tell the server to drop them.
It would also be possible to do something similar with ECF's remote services API...with ECF IDs. ECF IDs provide a serializable, unique-within-a-given-namespace way to reference things...so this can/could/is used to refer to out-of-process references. With the ECF remote services API, this is reified via the IRemoteServiceReference
http://www.eclipse.org/ecf/org.eclipse.ecf.docs/api/org/eclipse/ecf/remoteservice/IRemoteServiceReference.htmlIRemoteServiceReference is very much like a weak reference, in that it does not imply anything about actually memory/object allocation remotely. It does, however, give clients a way to refer to a remote service (note the getContainerID() method uniquely identifies the remote process and within-process ecf IContainer, while the properties from the IRemoteServiceReference identifies the service exposed by that identified process). And, it can also be turned into a real reference in the same way as an OSGi ServiceReference can be turned into an Object...i.e. via IRemoteServiceContainerAdapter.getRemoteService(remote reference). Obviously, the IRemoteServiceReference notion is almost identical to the OSGi *in-process* lookup/management API...but the IRemoteServiceReference has a container ID associated with it (which, of course, the ServiceReference does not). The idea was/is to use/generalize the mechanisms that OSGi uses (ServiceReference) with great success to provider for bundle dynamicity and make them similar enough to the OSGi mechanisms to transfer knowledge (and expectations). (Incidently, I've been contemplating writing a RemoteServiceTracker utility class...to make things much easier in terms of reference mgmt).
So I believe a similar strategy to what you are doing could be used to do pass by reference, but using IRemoteServiceReferences rather than objectids, weakhashmaps, etc. I would think that IRemoteServiceReferences would make this easier.
Another approach would be to define explicitly/non-transparent remote interfaces/facades for p2 services (e.g. IRemoteArtifactRepositoryManager) that would eliminate call-by-reference...or use IDs in their place (which are serializable). Note that the lookup for IRemoteServiceReferences can be filtered by both IDs and properties...i.e.
4. It must be possible to register callable references based on interfaces.
I'm not sure I understand...so you mean to be able to execute an arbitrary block of code (closure) on a remote system?
5. I must be able to limit the access to some methods/interfaces based on session data (isUserInRole() or something similar).
The question of how much/how to specify the limiting of access is a crucial question in how difficult this will be. One important mechanism that ECF has in place to support this is that ECF IDs extend Principal...so they can/could be used within the JAAS framework for authentication and authorization (i.e. specific access policy enforcement). With the Equinox JAAS work recently done, such integration is a natural thing. If you are already using JAAS for users/roles/permissions, etc then all that comes for free...and sits well with where Equinox seems to be headed WRT authentication and authorization. I expect that we will begin implementing providers that use the JAAS work in Equinox to do authorization checks of given IDs/Principals...and would be happy to work with you on this and/or make necessary/appropriate remote services API additions.
How will ECF remote services stand up to these requirements?
Pretty well, I think. Pluggable serialization is a relatively common thing these days...and available in several ECF providers. Pass-by-reference gets at the whole network transparently thing pretty deeply, as in-memory references don't look/behave exactly like remote references...and this shows up in garbage collection difficulty, failure handling, etc...but ECF's/OSGi's notion of IRemoteServiceReferences/ServiceReference does help here, I think.
And finally WRT security we're very much in line with all the JAAS and Equinox work here...we just haven't had it available to us in Equinox until 3.4 and so haven't built in a lot of dependencies to JAAS and the JAAS/Equinox integration...yet.
Thanks for the questions. These three areas: serialization, reference mgmt/garbage collection, and security are three very tough areas for RPC. Some might even say that 2 and 3 are at least partially responsible for the previous 'failures' in successful usage/adoption of transparent network RPC/RMC (which, as I'm sure you know, has been around for a long time).
Back to the top