[mosquitto-dev] Debugging Mosquitto CPU use

[Wiebe Cazemier <wiebe@xxxxxxxxxxxx>]

Hi,

I have a Mosquitto server that increases to over 100% CPU use over the course of a few days. After a SIGHUP when all clients reconnect (should they..?), $SYS/broker/clients/total and $SYS/broker/load/messages/received/1min are the same after a few minutes is allowed for all the clients to come back, 1400 and 80000 respectively.

However, $SYS/broker/load/connections/1min is very different: about 10 after the SIGHUP, 500 before the SIGHUP. That is about 8 per second.

The thing is, if I look at mosquitto.log and look for 'New client connected', I never see multiple of those per second.

How do TCP SYN and MQTT CONNECT relate? You can re-initiate MQTT connections on an existing TCP connection? 

Is this perhaps a client that is doing something funky? If so, how do I determine which one?

For reference, this is $SYS:

$SYS/broker/uptime 354611 seconds
$SYS/broker/clients/total 1476
$SYS/broker/clients/active 1436
$SYS/broker/clients/connected 1436
$SYS/broker/load/messages/received/1min 76365.76
$SYS/broker/load/messages/received/5min 69054.25
$SYS/broker/load/messages/received/15min 72974.01
$SYS/broker/load/messages/sent/1min 10094.54
$SYS/broker/load/messages/sent/5min 8644.84
$SYS/broker/load/messages/sent/15min 8680.41
$SYS/broker/load/publish/received/1min 46.59
$SYS/broker/load/publish/received/5min 37.74
$SYS/broker/load/publish/received/15min 38.28
$SYS/broker/load/publish/sent/1min 8266.35
$SYS/broker/load/publish/sent/5min 6533.69
$SYS/broker/load/publish/sent/15min 6850.82
$SYS/broker/load/bytes/received/1min 5315437.46
$SYS/broker/load/bytes/received/5min 4819534.85
$SYS/broker/load/bytes/received/15min 5105105.81
$SYS/broker/load/bytes/sent/1min 576765.34
$SYS/broker/load/bytes/sent/5min 482419.39
$SYS/broker/load/bytes/sent/15min 516706.54
$SYS/broker/load/sockets/1min 35.48
$SYS/broker/load/sockets/5min 137.20
$SYS/broker/load/sockets/15min 87.15
$SYS/broker/load/connections/1min 33.32
$SYS/broker/load/connections/5min 133.81
$SYS/broker/load/connections/15min 83.33
$SYS/broker/messages/stored 309993
$SYS/broker/messages/received 413176615
$SYS/broker/messages/sent 48710179
$SYS/broker/store/messages/count 309993
$SYS/broker/store/messages/bytes 5802187
$SYS/broker/subscriptions/count 4431
$SYS/broker/retained messages/count 268341
$SYS/broker/heap/current 167254856
$SYS/broker/publish/messages/received 239597
$SYS/broker/publish/messages/sent 39748977
$SYS/broker/publish/bytes/received 10248128246
$SYS/broker/publish/bytes/sent 772533676
$SYS/broker/bytes/received 28895693155
$SYS/broker/bytes/sent 2796658721

Regards,

Wiebe