Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [paho-dev] MQTT <-> REST+websocket mapping

Interesting stuff. We're very much in favour of keeping MQTT with a binary mapping. It makes it trivial for implementations to add a WebSockets tunnel, using a wide range of different technologies that aren't tidied to a broker, etc.

The challenge is doing a little binary handling in _javascript_, but done once, it's done.

The only weakness with WebSockets is the blasted framing (XOR? Text frames? What where they smoking? Oh well, that's a committee for you). Many of the more generic WebSockets <=> TCP implementations (correctly, in my view), scoop up as many bytes as possible and send them on as WS binary frames. Thus there's no delineations in the frames so that they align with MQTT data structures.

And the W3C's WS API is naive and assumes one should present complete frames at each callback event. This sort of thing seems to make a js' programmers job harder, until he realises that it's just life, and a WebSocket is nothing more than a TCP stream. All he has to do is keep a buffer, and pre-pend it to the next WS event...

I'd agree with Nick on the challenges of using HTTP for messaging. It's quarts into pint pots - messaging is async, HTTP is sync and typically short-lived. It's hard to scale a broker that does HTTP mq.

We do need to register an IANA subprotocol for MQTT at some point

Raphael Cohn
Chief Architect, stormmq
Secretary, OASIS AMQP Standard
raphael.cohn@xxxxxxxxxxx
+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com


On 15 March 2013 15:24, Nicholas O'Leary <nick.oleary@xxxxxxxxx> wrote:
Clearly someone was busy at eclipse, and they got the wiki working in
the time it took me to write my reply.

Here's the link for the MQTT/Websockets stuff -
http://wiki.eclipse.org/Paho/Paho_Websockets

On 15 March 2013 15:22, Nicholas O'Leary <nick.oleary@xxxxxxxxx> wrote:
> Hi Matteo,
>
> A couple thoughts and links for your first two questions:
>
>> 1) The websocket tunneling issue is longstanding in this community. Some
>> previous attempt has been done, and there was even a proposal for routing
>> binary MQTT inside a websocket tunnel.
>> I believe that the best approach to make this work is to map MQTT to JSON in
>> a standard way, and then route it inside websocket & polyfills (es.
>> socket.io or sock.js and the like)
>> This will simplify implementations and diffusion.
>
> There is already a proposal for using MQTT with Websockets on the Paho
> wiki (which I would link to if the entire eclipse wiki wasn't down
> right now). It takes the approach of tunnelling the MQTT binary
> packets through the websocket.
> This is the approach taken by the Mosquitto websocket implementation afaik.
>
> In my opinion, mapping it to a JSON payload across the websocket
> (which is what we did with earlier versions when browser support for
> ArrayBuffer etc was not as wide spread as it is today) means you lose
> the benefit of the light-weight packet size over the network - one of
> the core principles of the protocol.
>
>> 2) While creating an interface for publishing messages is easy, the core
>> issue is on reading them.
>> For example, ActiveMQ uses a DELETE to read a message, which can timeout if
>> no message is published in a given period of time. This approach is not
>> really web-compatible,
>
> can you explain what you mean by "not really web-compatible". DELETE
> is absolutely the right verb to use if you request a resource that as
> a side-effect removes it from the server. From _javascript_ you can
> issue a DELETE XHR request just as easily as any of the other verbs.
>
>> so QEST adopt a standard GET approach, exposing to
>> the web every published message (here you can find is my home temperature:
>> http://qest.me/topics/temp). Every message is stored on a Redis database.
>> However, exposing the last message on the web is extremely similar to the
>> retained message behavior. In QEST, I did assimilate the two concepts
>> stating that every message is retained, even on MQTT.
>> At the moment, I am revising this decision to be MQTT-compliant. Still, the
>> web behavior needs to be clear. One possible solution is to expose on the
>> web only retained messages, or ignore the 'retained' flag at the web level
>> and expose everything.
>
> You cannot only expose retained messages and call that MQTT-compliant.
>
> A possible suitable approach would be: When an http-client issues its
> first 'subscribe' to a topic, return any retained message on that
> topic. Then, as you would with an MQTT non-clean session client,
> subscribe to the topic in the MQTT-sense for them and start queuing up
> messages for that client. Subsequent requests from that client will
> then consume messages off that client-specific delivery queue. A set
> of custom HTTP headers could be defined to say how long that
> subscription should be maintained for before discarding the queued up
> messages.
>
> It is worth noting this is well trodden ground. A few years ago, we
> released a support pack for WebSphere MQ that provided an HTTP
> interface. This isn't explicitly MQTT as it covers queues as well, but
> it did examine a lot of these issues. This has since been rolled into
> the base product, but you can see the original support pack
> documentation here to see how they defined the interface
> http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24016142.
> Again - this was for a more complex messaging system than MQTT in
> terms of options and properties, but the basic concepts are there. I
> should add, I don't like how they chose to do subscriptions in the
> support pack - they went for the 'do a request with a timeout and only
> deliver a message that happens to arrive in that period' approach. But
> interesting background material.
>
>
> Regards,
> Nick
>
>
>
> On 15 March 2013 14:57, Matteo Collina <hello@xxxxxxxxxxxxxxxxx> wrote:
>>
>> Hi Everyone,
>>
>> I am working on a web adapter for MQTT, and in general to the whole Internet
>> of Things and M2M worlds. My preliminary work is available at:
>> http://qest.me/.
>> It seems everyone is working on this problem, however there is not a shared
>> effort on it, and it is time to reignite the discussion.
>>
>> I am identifying the requirements for a general solution to this problem:
>> 1) Web-tunnelling of MQTT to achieve compatibility with web apps..
>> 2) REST interface that must enable to read and write the last message from a
>> topic.
>> 3) REST interface to read the history of the messages on a topic/query.
>> 4) No modification to MQTT and pluggable to any MQTT server.
>> 5) Security & Access Control Systems.
>> 6) Support for webhooks, to ease the integration in other server-side
>> systems.
>>
>> In order to clarify more, I will expand some points.
>>
>> 1) The websocket tunneling issue is longstanding in this community. Some
>> previous attempt has been done, and there was even a proposal for routing
>> binary MQTT inside a websocket tunnel.
>> I believe that the best approach to make this work is to map MQTT to JSON in
>> a standard way, and then route it inside websocket & polyfills (es.
>> socket.io or sock.js and the like)
>> This will simplify implementations and diffusion.
>>
>> 2) While creating an interface for publishing messages is easy, the core
>> issue is on reading them.
>> For example, ActiveMQ uses a DELETE to read a message, which can timeout if
>> no message is published in a given period of time. This approach is not
>> really web-compatible, so QEST adopt a standard GET approach, exposing to
>> the web every published message (here you can find is my home temperature:
>> http://qest.me/topics/temp). Every message is stored on a Redis database.
>> However, exposing the last message on the web is extremely similar to the
>> retained message behavior. In QEST, I did assimilate the two concepts
>> stating that every message is retained, even on MQTT.
>> At the moment, I am revising this decision to be MQTT-compliant. Still, the
>> web behavior needs to be clear. One possible solution is to expose on the
>> web only retained messages, or ignore the 'retained' flag at the web level
>> and expose everything.
>>
>> 3) An History interface has to be provided. The best work done in that area
>> is Cosm.
>> However, a standard representation of message meta-data has to be defined.
>>
>> 5) There is much discussion on how to expose M2M and IoT to end users.
>> Exposing it to the web is a key point, and doing it a safe way for both
>> users and devices is extremely important.
>> In particular, both the provisioning of devices and the pairing between
>> users and devices must be addressed.
>> This will necessarily lead to a paring protocol on top of both MQTT and
>> HTTP, and might include an OAuth step.
>>
>> Have you got some suggestions on these topics?
>>
>> Cheers,
>>
>> Matteo
>>
>>
>> _______________________________________________
>> paho-dev mailing list
>> paho-dev@xxxxxxxxxxx
>> http://dev.eclipse.org/mailman/listinfo/paho-dev
>>
_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/paho-dev


Back to the top