Re: [iot-sp] LWM2M protocol adapter prototype

[Scott Lewis <slewis@xxxxxxxxxxxxx>]

On 12/9/2015 3:43 AM, Dejan Bosanac wrote:

Just a couple of thoughts from me.

Thanks for this.

The job of the IoT Connector is to provide the nice programming model for the IoT use cases (telemetry, provisioning, etc.) for a heterogenous (and large) devices environment. So, in my view this is something that needs to be defined on the northbound of the connector. People tend to prefer reactive/asynchronous APIs these days, but if there’s a need, the synchronous style API can be created as well (at least for certain cases). Then connector needs to translate that to whatever protocol/semantics devices use. And that’s the job of Protocol Adapters. So the adapter is just a one piece in the whole picture and it’s not where we should be solving “application developer” view of the system.

My $0.03:   I agree with you that the protocol adapters isn't where the entire 'application developer' view of 'the system' should be addressed, but I would argue that some choices around intermediate models of communication (e.g.) RPC, events, queues, topics, futures/promises, etc *will* bubble up through the IoT use cases.    I happen to believe that one of the hard problems here is defining the mappings of IoT use cases to intermediate models (and possibly creating new ones and/or mixing existing communication models).

For example, Eclipse Kura already uses MQTT topics for all use cases (http://eclipse.github.io/kura/ref/mqtt-namespace.html). So if some devices are connected through Kura, we need to provide that conversion from MQTT to RPC world on the northbound.

Yes.  I would argue that it would reduce everyone's workload to (when possible) adopt standards for these intermediate layers...in particular OSGi Remote Services/RSA...for RPC  [1].   That way consumers won't be bound to your (or anyone's) particular RPC implementation.

In my opinion, the messaging is a good fit for this intermediate layer, as it provides many benefits, like decoupling, flexibility in terms of scalability, routing, security, etc.

I don't disagree with your assertion that messaging is a good fit for this intermediate layer, but my assertion is that many models (sync RPC, async RPC, 1-1 channels/queues, high-level events, multipoint delivery, etc) will likely be needed cover a variety of use cases.   I also agree with you that async messaging is a good place to start, because it can accommodate/implement other models (e.g. RPC).  

TLDR version; I agree that we need to provide an easy and natural model for developers using IoT Connector. We should do that with a proper design of API(s) on the northbound. The job of the connector and adapters then is to translate that API to the real world of various devices and gateways.

Agreed.

One thing to point out about ECF that may be useful for this IoT adapter effort:   From the beginning (10+ years) ECF has been designed to accommodate two notions: intermediate models of communication and protocol independence.    In short, we accomplish this by having the notion of an abstract 'container' [2] and using the now-well-known IAdaptable for run-time API extensibility [3].  

We (and others) have used this to create a number of intermediate communication models...e.g. remote services/rpc, network service discovery, async channels (pt-to-pt and multipoint), filetransfer, presence and IM, and others.   Each model is represented by an abstract API that is a container adapter, and so a given protocol can be used implement zero or more of these adapters.   This allows both API extensibility (new models) and transport/wire protocol independence.

An example is the relatively new MQTT provider [4].   This uses Paho to implement the remote service adapter and so automatically converts it into a spec-compliant implementation of the OSGI Remote Services/RSA specification.  

I'll explain more and provide more doc pointers, code examples, tutorials, etc, if desired.   But since it seems that there will/might be work this mapping between IoT use cases and these 'intermediate' communication models, and we've been doing that for some time now, I wanted to make this aspect of ECF more directly known.   And of course anyone is able to create their own intermediate forms (APIs) and implement them with appropriate protocols (AMQP) or reuse/extend what we have done.

Scott

[1] https://wiki.eclipse.org/ECF#OSGi_Remote_Services
[2] http://download.eclipse.org/rt/ecf/latest/javadoc/org/eclipse/ecf/core/IContainer.html
[3] http://www.eclipsezone.com/articles/what-is-iadaptable/
[4] https://github.com/ECF/Mqtt-Provider

On Tue, Dec 8, 2015 at 10:17 PM, David Ingham <dingham@xxxxxxxxxx> wrote:

I think the Spring documentation you referenced is based on AMQP 0.9.1 as it talks about “exchanges” and “queues” neither of which are part of the AMQP 1.0 spec.

 

Anyway, all that said, I think the interesting technical question here is what form the higher-level protocol messages take between the protocol adaptors and the IoT Connector which is I think the point you were getting at too.

 

From: iot-sp-bounces@xxxxxxxxxxx [mailto:iot-sp-bounces@xxxxxxxxxxx] On Behalf Of Scott Lewis
Sent: 08 December 2015 21:08
To: iot-sp@xxxxxxxxxxx
Subject: Re: [iot-sp] LWM2M protocol adapter prototype

 

On 12/8/2015 12:48 PM, David Ingham wrote:

<stuff deleted>

Thanks I understand that Dave.  But my point was that AMQP basically implements queues and topics for the model of inter-process interaction. 

[DBI] No, AMQP 1.0 has no notion of queues and topics, it has a notion of 1-way Links.

All I meant to say was that the typical communication model exposed to the programmer is queues and topics.   I understand that 1 way-links may lie underneath in AMQP, but for this discussion that doesn't much matter to me at the moment.

However, if you feel their explanation is incorrect, feel free to correct the folks behind this article:  https://spring.io/understanding/AMQP

<stuff deleted>

As one whose implemented RPC over AMQP and other messaging impls (e.g. [1]), it hasn't been my experience that layering non-trivial RPC on async messaging is as simple as you implying.  Particularly when one considers service-and-distribution-level issues in addition to transport-level protocol differences [2].

 

[DBI] I agree that implementing RPC over AMQP 0.x, JMS and other queuing systems isn’t trivial. Doing request/response over a pair of AMQP 1.0 Links is different.

Indeed it is.  I would argue that generally what people are 'looking for' is closer to RPC than it is req/resp over a link.  Granted not true for all use cases, but generally true I would say.   That's why I'm asking the following...

>My follow-up question really is:   what thinking is currently happening WRT this mapping between IoT >communication needs (e.g. provisioning) and these abstractions (e.g. async/sync, point vs. multipoint, reliable vs. >unreliable, rpc, events, etc).

Thanks,

Scott

_______________________________________________
iot-sp mailing list
iot-sp@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/iot-sp

--

Regards
--
Dejan Bosanac
about.me/dejanb

_______________________________________________
iot-sp mailing list
iot-sp@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/iot-sp