Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [paho-dev] Question about MessageHandler (called when a new message is received) GO Paho mqtt

Hi Danny,

Unfortunately the answer is "it depends..." - if you receive one message every 10 minutes then a handler that takes a minute is unlikely to be an issue but at 100 messages a second... The comment is in the readme because there have been quite a few issues raised (for example 1, 2, 3)  where a blocking handler is the cause; in many cases this is due to a deadlock in the users code but, in other cases, a slow handler causes message loss. The main issues caused by blocking handlers include:

Dropped Messages (QOS1+) - Because the client will wait for the handler to complete before sending an ACK to the broker you need to ensure that your handler can keep up. Some brokers limit the number of messages queued for a client (e.g. Mosquitto has max_queued_messages which defaults to 1000) so if you don't keep up then the broker may drop messages (and this may not be logged anywhere). This is probably not an issue when you are receiving one message every 10 minutes but, when processing 100+ messages a second, the queue can grow very rapidly!. Generally the delay within a handler will be due to I/O (e.g. storing the packet in a database) so allowing concurrency (ClientOptions.SetOrderMatters(false)) can significantly increase the volume of messages you can handle (the impact of this will depend upon broker settings e.g. max_inflight_messages in Mosquitto).

Publishing - If your message handler calls Publish (using the same instance of client and waiting on the token) then you will end up with a deadlock (at QOS1+). This is because, in order for the publish to complete, the client needs to process the ACK and it cannot do that until the message handler for the original message returns. Calling Publish from a go routine is fine (it will also work with ClientOptions.SetOrderMatters(false) but the original message will not be acknowledged until your handler returns) .

Ping - If the handler ends up in a deadlock (or extended delay) then the client will not respond to PINGREQ packets (or send them) which can lead to the connection dropping (and the client cannot recover from this until the handler exits).

In my experience for most use-cases the order in which messages are processed is not critical; hence the advice to use ClientOptions.SetOrderMatters(false)which avoids many of these issues (processing messages concurrently is generally more efficient than serial processing). It is still important to note that a message will only be acknowledged after the handler returns so there is still a potential for dropped messages if your handler can't keep up.

Regards,
Matt

On Thu, 30 Jun 2022 at 05:04, Danny Joshua <dannyjoshua9999@xxxxxxxxx> wrote:
 HI,

I have a question regarding the below from Go Paho mqtt README.
How long (in Seconds) the MessageHandler can do processing of message without causing the issue? Can you please guide me on this ?

=====
A MessageHandler (called when a new message is received) must not block (unless ClientOptions.SetOrderMatters(false) set). If you wish to perform a long-running task, or publish a message, then please use a go routine (blocking in the handler is a common cause of unexpected pingresp not received, disconnecting errors).
====


Thanks
Danny
_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/paho-dev

Back to the top