Re: [paho-dev] Offline buffering in Python3 Paho MQTT client

[Lars Noodén <lars.nooden@xxxxxxxxx>]

On 10/7/20 4:02 PM, Greg Troxel wrote:
> Lars Noodén writes:
>> I am trying to have a Python3-based Paho MQTT client do offline
>> buffering during longer intervals when there is no network connectivity.
>> ...
> You say "buffering", but once you talk about offline then this veers
> into database sync.  I think it's an important use case.
> ...

Thanks.  I am unsure of the right terminology but buffering seems to be
closest concept.  Some systems are without networking for minutes or
hours at a time yet still able to measure and have plenty of RAM in that
context.  This data is still useful to have even when it comes in very
late.

For this project, several systems feed timestamped temperature
measurements into a broker.  A database saves those incoming messages
whenever they happen to arrive.  Separately, another script does a
database query for GNUplot for visualizations.  Yet another script
subscribes to MQTT and updates an e-paper display with the latest
measurements.  On the e-paper display different colors mark the
staleness of the data, if it is not obvious from a discrepancy between
the current time and the timestamp it displays.  The e-paper does not
benefit from the held MQTT messages, but the GNUplot rendering and other
after-the-fact analysis can still use the data.

> In the case of reporting temperatures once a minute, persistence makes
> sense, but to me at least, only if you include timestamps.

For this particular project, that is the first of the three fields.

> Do you think this sort of offline/retransmission schemes need to
> preserve ordering?

Ordering would be useful but not necessary, at least for me, at least at
the moment.  However, I don't count on it.  Due to device and network
latency, which can be several seconds, even when all the devices are
online at the same time, I have to use timestamps in the MQTT messages.

Anyway, I tried adding QoS when publishing,

   mqttclient.publish("temperatures", qos=2, payload=msg)

but that does not seem to send a backlog of data after the longer
outages finish.

Regards,
Lars