Re: [paho-dev] MQTT <-> REST+websocket mapping

["Nicholas O'Leary" <nick.oleary@xxxxxxxxx>]

Hi Matteo,

A couple thoughts and links for your first two questions:

> 1) The websocket tunneling issue is longstanding in this community. Some
> previous attempt has been done, and there was even a proposal for routing
> binary MQTT inside a websocket tunnel.
> I believe that the best approach to make this work is to map MQTT to JSON in
> a standard way, and then route it inside websocket & polyfills (es.
> socket.io or sock.js and the like)
> This will simplify implementations and diffusion.

There is already a proposal for using MQTT with Websockets on the Paho
wiki (which I would link to if the entire eclipse wiki wasn't down
right now). It takes the approach of tunnelling the MQTT binary
packets through the websocket.
This is the approach taken by the Mosquitto websocket implementation afaik.

In my opinion, mapping it to a JSON payload across the websocket
(which is what we did with earlier versions when browser support for
ArrayBuffer etc was not as wide spread as it is today) means you lose
the benefit of the light-weight packet size over the network - one of
the core principles of the protocol.

> 2) While creating an interface for publishing messages is easy, the core
> issue is on reading them.
> For example, ActiveMQ uses a DELETE to read a message, which can timeout if
> no message is published in a given period of time. This approach is not
> really web-compatible,

can you explain what you mean by "not really web-compatible". DELETE
is absolutely the right verb to use if you request a resource that as
a side-effect removes it from the server. From javascript you can
issue a DELETE XHR request just as easily as any of the other verbs.

> so QEST adopt a standard GET approach, exposing to
> the web every published message (here you can find is my home temperature:
> http://qest.me/topics/temp). Every message is stored on a Redis database.
> However, exposing the last message on the web is extremely similar to the
> retained message behavior. In QEST, I did assimilate the two concepts
> stating that every message is retained, even on MQTT.
> At the moment, I am revising this decision to be MQTT-compliant. Still, the
> web behavior needs to be clear. One possible solution is to expose on the
> web only retained messages, or ignore the 'retained' flag at the web level
> and expose everything.

You cannot only expose retained messages and call that MQTT-compliant.

A possible suitable approach would be: When an http-client issues its
first 'subscribe' to a topic, return any retained message on that
topic. Then, as you would with an MQTT non-clean session client,
subscribe to the topic in the MQTT-sense for them and start queuing up
messages for that client. Subsequent requests from that client will
then consume messages off that client-specific delivery queue. A set
of custom HTTP headers could be defined to say how long that
subscription should be maintained for before discarding the queued up
messages.

It is worth noting this is well trodden ground. A few years ago, we
released a support pack for WebSphere MQ that provided an HTTP
interface. This isn't explicitly MQTT as it covers queues as well, but
it did examine a lot of these issues. This has since been rolled into
the base product, but you can see the original support pack
documentation here to see how they defined the interface
http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24016142.
Again - this was for a more complex messaging system than MQTT in
terms of options and properties, but the basic concepts are there. I
should add, I don't like how they chose to do subscriptions in the
support pack - they went for the 'do a request with a timeout and only
deliver a message that happens to arrive in that period' approach. But
interesting background material.


Regards,
Nick



On 15 March 2013 14:57, Matteo Collina <hello@xxxxxxxxxxxxxxxxx> wrote:
>
> Hi Everyone,
>
> I am working on a web adapter for MQTT, and in general to the whole Internet
> of Things and M2M worlds. My preliminary work is available at:
> http://qest.me/.
> It seems everyone is working on this problem, however there is not a shared
> effort on it, and it is time to reignite the discussion.
>
> I am identifying the requirements for a general solution to this problem:
> 1) Web-tunnelling of MQTT to achieve compatibility with web apps..
> 2) REST interface that must enable to read and write the last message from a
> topic.
> 3) REST interface to read the history of the messages on a topic/query.
> 4) No modification to MQTT and pluggable to any MQTT server.
> 5) Security & Access Control Systems.
> 6) Support for webhooks, to ease the integration in other server-side
> systems.
>
> In order to clarify more, I will expand some points.
>
> 1) The websocket tunneling issue is longstanding in this community. Some
> previous attempt has been done, and there was even a proposal for routing
> binary MQTT inside a websocket tunnel.
> I believe that the best approach to make this work is to map MQTT to JSON in
> a standard way, and then route it inside websocket & polyfills (es.
> socket.io or sock.js and the like)
> This will simplify implementations and diffusion.
>
> 2) While creating an interface for publishing messages is easy, the core
> issue is on reading them.
> For example, ActiveMQ uses a DELETE to read a message, which can timeout if
> no message is published in a given period of time. This approach is not
> really web-compatible, so QEST adopt a standard GET approach, exposing to
> the web every published message (here you can find is my home temperature:
> http://qest.me/topics/temp). Every message is stored on a Redis database.
> However, exposing the last message on the web is extremely similar to the
> retained message behavior. In QEST, I did assimilate the two concepts
> stating that every message is retained, even on MQTT.
> At the moment, I am revising this decision to be MQTT-compliant. Still, the
> web behavior needs to be clear. One possible solution is to expose on the
> web only retained messages, or ignore the 'retained' flag at the web level
> and expose everything.
>
> 3) An History interface has to be provided. The best work done in that area
> is Cosm.
> However, a standard representation of message meta-data has to be defined.
>
> 5) There is much discussion on how to expose M2M and IoT to end users.
> Exposing it to the web is a key point, and doing it a safe way for both
> users and devices is extremely important.
> In particular, both the provisioning of devices and the pairing between
> users and devices must be addressed.
> This will necessarily lead to a paring protocol on top of both MQTT and
> HTTP, and might include an OAuth step.
>
> Have you got some suggestions on these topics?
>
> Cheers,
>
> Matteo
>
>
> _______________________________________________
> paho-dev mailing list
> paho-dev@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/paho-dev
>