[ecf-dev] ECF Publish/Subscribe mechanism

[Peter Nehrer <pnehrer@xxxxxxxxxxxxx>]

Hi all,

I'd like to solicit everyone's feedback on something I've been working 
on recently. I think ECF is a good fit for group scenarios where one 
member has a service (e.g., a live model) that others may want to 
"subscribe" to; in other words, receive some domain-specific events that 
originate at the remote source. As an example, consider the scenario in 
which one group member publishes a blog (a series of entries) and that 
some other members want to view it and be notified as soon as any 
changes are made. (There are numerous other examples one can imagine -- 
monitoring someone's task list as it's being completed, error log as new 
entries are logged, etc.)

Continuing with the blog example, one way to implement this would be for 
the author to create a shared object that would replicate itself to all 
containers (and any new ones as they join), and whenever changes are 
made to the primary instance, events would be forwarded to remote 
replicas. Other members would monitor these events and react appropriately.

While this might work in the simple case, it wouldn't scale well -- even 
containers that are not explicitly interested in these events would have 
to bear the traffic. A more efficient approach might be to only create 
replicas in containers that request it -- the client would request that 
a certain service (published in a specific container) be replicated 
locally. When that occurs, the client would be notified and could 
consume the service in a domain-specific way.

To make this approach even more generic, a utility service (local to 
each container) would track publication of services in all remote 
containers, so that interested clients may be notified whenever a 
service is published at a remote site (so that they may then subscribe 
to it if they want to).

The more concrete class of replicated service that I'm interested in is 
the "shared model" (but no editing this time :-) ) -- the source 
container publishes some model with an "updater" that knows how to apply 
changes to it. Remote containers then subscribe to it and receive the 
model's current state, followed by any incremental updates to it (as 
they happen).

I created an example plugin (org.eclipse.ecf.example.pubsub) that 
contains the experimental API as well as the implementation, along with 
an example of a simple model that can be published and subscribed to (an 
appendable list of strings). The example consists of a view that shows 
remote published services and allows the user to publish shared lists. 
Remote services (if they're also the sample appendable lists)  may be 
subscribed to and their contents are presented in a separate view. I'll 
put the details on how to run it on the wiki.

Some details about the protocol:
1. The client may use the IPublishedServiceDirectory to obtain/listen to 
remote services being published/unpublished. This instance is obtained 
as the local container's adapter. It is local to the container (i.e., 
does not replicate itself) and has a fixed ID.
2. IPublishedServiceDirectory uses a helper shared object (the Discovery 
Agent) to learn about what's published remotely. Upon activation, it 
creates the agent (with a GUID), which then replicates itself to all 
other containers (and any new ones, as they join). The remote agent 
replicas then collect information about published services (i.e., their 
activation/deactivation) and forward it to their 
IPublishedServiceDirectory (in their home container). With this 
strategy, no "master directory" of published services needs to be 
maintained, thus greatly simplifying the solution.
3. The client may also obtain an IPublishedServiceRequestor (again as 
the container's adapter), which allows it to subscribe to a specific 
published service on a specific container (and receive an ISubscription 
upon completion, which is disposed when done).
4. Upon obtaining the subscription, the client checks if it is a shared 
model (of the right kind -- and AppendableList, in this example), 
obtains its initial state and listens to any model changes.
5. Whenever the master instance is updated (i.e., an item is added), the 
change is sent out to all remote replicas and applied to their local 
states. The clients receive the appropriate model change notification.

There's a lot more to talk about, but this should be enough to start the 
conversation. I'll continue to harden the implementation and revise the 
API. I'll move the API to a dedicated plugin once it's matured a bit. 
Please let me know what you think. Thanks!

--Peter