Re: [ecf-dev] Large number of clients

[Scott Lewis <slewis@xxxxxxxxxxxxx>]

Hi Leen,

Leen Toelen wrote:
> Hi,
>
> I am investigating the feasibility of a project, and ecf is one of the 
> technologies which looks like it might be very useful for it. We're 
> going to deploy a large number of nodes (+-20.000) with an equinox 
> stack on them. From time to time they need to call home and download 
> updated software if available.
> Most of the time they will be watching their environmental sensors and 
> have some predefined triggers. When a trigger goes off they need to 
> send a message to the server, which takes action if needed.
> There are administrators as well, which should be given the option to 
> remotely use the node's OSGI services, set its parameters, stream 
> audio and video, get its GPS location etc.
> I was thinking of setting up an xmpp server (wrapped in an OSGI 
> bundle) and use ecf as a communication platform. The xmpp server 
> should be able to handle 20000 clients (which are not doing a lot most 
> of the time) and so the administrator gets a good idea on when each 
> node is online. When the administrator is interested in a specifc 
> node, he can use its remote services as well.
>
> My questions are:
> - is ecf/xmpp the best library/protocol for something like this?

Although the ECF team would like to build and XMPP server, we have not 
yet done so....so the difficulty for you would be in creating a scalable 
XMPP server implementation (using OSGi, Equinox and ECF).  It's possible 
that you could use one of the commercial xmpp servers, such as Jive 
Software's, but I'm not sure whether their server scales to that many 
simultaneous clients.

> - is ecg capable of streaming video and audio?

ECF has a call signalling API, and thanks to some work by Moritz Post, 
there is an implementation of Jingle (an XMPP extension protocol) that 
does streaming audio.  There is more information about this work here:

http://wiki.eclipse.org/VOIP
http://wiki.eclipse.org/VoIP_via_the_ECF_Call_API_and_the_Jingle_Protocol

> - I've got a basic setup running with little text messages, but will 
> it scale to that many nodes?

For XMPP I don't know.  It would likely depend heavily on the quality 
and scalability of the server (less so on ECF APIs actually). 

Sorry I can't be more definite, but in truth the main scalability point 
for that many clients and doing things like streaming audio is the 
server/servers, and like I said we would very much like to create an 
XMPP server from Equinox/OSGi, and ECF, we haven't yet done so.

If you are interested in creating an XMPP server, however, this would 
likely be a very good sub-project for ECF going forward, as we are 
moving into the new Eclipse Runtime project.  Also, we have received 
several inquiries about an ECF-based XMPP server previously, and so 
there seems to be very good community interest.

Thanks,

Scott