Re: [iot-sp] IoT Server Event Hub Scalability

[Henryk Konsek <hekonsek@xxxxxxxxx>]

Hi Regis,

Kai mentioned AMQP transport as a candidate for the underlying technology for IoT connector. I really like this idea as there are many existing AMQP server implementations that can resolve scalability issue and optimize bandwidth.

Network of Apache Qpid Dispatch routers can be used for multiplexing client connections so even with millions of devices connected to the AMQP IoT connector we can scale the load easily. Connection multiplexing can be combined with the network of routers to end up with only several network connections  to the core broker, even if millions of devices are connected to the IoT connector routers network.

Apache Qpid is just one of the AMQP implementations. I'm sure that RabbitMQ, Azure event hub and many others AMQP implementors have solutions with similar scalability. This is also why I believe that AMQP is so great choice for our purposes - we abstract communication method and avoid locking to any particular implementation of the IoT connector.

Cheers!

śr., 2.12.2015 o 20:59 użytkownik Scott Lewis <slewis@xxxxxxxxxxxxx> napisał:

On 12/2/2015 11:05 AM, Piccand, Regis wrote:

Hello!

 

I believe one of the key aspect of the IoT Server will be the “blackbox event hub” mentioned in the https://wiki.eclipse.org/IoT/IoTServerPlatform – that is, the ability for consumers to subscribe to events/feeds. From a scalability and bandwidth perspective, this could however become tricky with lots of subscribers. A single event could end up generating tons of messages distributed across to all subscribers.

 

One way to work around the bandwidth issue could be that all consumers need to be ‘local’ to an IoT Server. The subscription could then happen between IoT servers (on behalf of the final subscriber), thus minimizing the need for bandwidth, as an  event would be propagated just once between all (subscribed) IoT servers, and then to the ‘local’ devices.

 

On the other hand, having multiple IoT Servers also implies multiple feed “repositories”. By making the servers participate in a peer-to-peer network, we could then share the other server’s feed registry and thus facilitate subscription to ‘remote’ servers. Another way could be to manage a centralized feed repository where IoT servers would publish their feed registry.

 

Hope this makes sense - looking forward to discuss more.

 

Best,

-Regis

For everyone's info:   the OSGi EventAdmin service [1] provides a within-process event bus.   Of course there are multiple spec-compliant implementations...e.g. one with Felix, one with Equinox (org.eclipse.equinox.event), probably others.

There's been a fair amount of work on a distributed eventadmin...e.g. [2].   There are ECF consumers that are using it in production.  It's in the recent ECF 3.12 release  [3].

Combining 'local' and 'distributed' eventadmins can result in some sophisticated topologies and use cases, as Regis hints at above.

Finally: FWIW, the OSGi Enterprise Experts group is engaged in standardization efforts around distributed eventing. [4].

Scott

[1] http://eclipsesource.com/blogs/2009/09/15/osgi-eventadmin/
[2] https://wiki.eclipse.org/Distributed_EventAdmin_Service
[3] http://eclipseecf.blogspot.com/2015/11/ecf-3120-released.html
[4] https://github.com/osgi/design/blob/master/rfps/rfp-0158-DistributedEventing.pdf

 

Régis Piccand Manager – Software Development rpiccand@xxxxxxxxxxxx Office +41 26 408 77 12 Mobile +41 79 898 13 11 Route de Chandolan 8, 1752 Villars sur Glâne Switzerland http://www.verisign.com

 

 

 

 

_______________________________________________
iot-sp mailing list
iot-sp@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/iot-sp

_______________________________________________
iot-sp mailing list
iot-sp@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/iot-sp

--

Henryk Konsek

http://about.me/hekonsek

Attachment: gifJbegZnHz5Y.gif
Description: GIF image

Attachment: pngMbqGmvtdxm.png
Description: PNG image

Attachment: pngTCOnWaLV0E.png
Description: PNG image

Attachment: gifAp3r2A4tE4.gif
Description: GIF image