Re: [mosquitto-dev] queue sizes by client?

[Robert Linden <r.linden@xxxxxxxxx>]

Hello Roger,

On 28.06.19 00:24, Roger Light wrote:
> That still remains the case. I am reluctant to publish per client
> information because it either means periodically publishing messages
> about every single client, of which there may be many, or of
> publishing once only based on a magic subscription, which feels
> wrong.

ok, thanks for the answer. Yes, not an easy decision...

> Something that may be useful right now - there is a log message given
> when a client starts dropping messages. You may be able to use that.
> I cant check exactly what/where the message is right now.
Right, it looks like this:

1561719759: Outgoing messages are being dropped for client queueTest.

But it comes only the first time and doesn't give an indication about the current status.
We added a new log-message that shows all the info whenever a new message needs to be queued, maybe
that is useful to somebody (patch should fit version 1.6.3):

*** original-database.c	2019-06-28 14:11:36.241789958 +0200
--- debug-database.c	2019-06-28 14:11:30.221750719 +0200
***************
*** 97,102 ****
--- 97,105 ----
  	bool valid_bytes = source_bytes - adjust_bytes < max_queued_bytes;
  	bool valid_count = source_count - adjust_count < max_queued;
  
+         log__printf(NULL, MOSQ_LOG_NOTICE, "db__ready_for_queue(id=%s): source_bytes=%lu, adjust_bytes=%lu, max_bytes=%lu, valid_bytes=%d, source_count=%d, adjust_count=%d, max_count=%d, valid_count=%d\n",
+         	context->id, source_bytes, adjust_bytes, max_queued_bytes, valid_bytes, source_count, adjust_count, max_queued, valid_count);
+ 
  	if(max_queued_bytes == 0){
  		return valid_count;
  	}

> You can also use the persistent client expiry option to remove
> persistent clients that have not connected for a given amount of
> time.

We cleaned up some full queues by connecting as those obsolete clients with clean-session-flag and QoS=0.
We don't normally have that problem, we think, so this manual process was ok.

What also helped us now (and I don't know how I missed that before) was the mosquitto_db_dump-tool.
With that we inspected the database and saw all the DB_CHUNK_CLIENT_MSG which where queued. Really nice! 

I have a new question regarding the mosquitto.db though. It seemed to me that there are DB_CHUNK_MSG_STORE,
which hold one message each, and then there's the DB_CHUNK_CLIENT_MSG, which link to these messages,
for each client that is supposed to receive that message. Is that correct? Because the weird thing is
the numbers of these that we see:

446 DB_CHUNK_CLIENT_MSG:
242826 DB_CHUNK_MSG_STORE:

That looks like there are messages, which no client actually needs to receive. Like they should have
been "garbage collected" but were not. Or am I misunderstanding how it is supposed to work?
Oh, but if it is a bug then maybe it is obsolete, because we don't actually have the current
version of mosquitto in this installation (it's 1.5.6).

Thanks & all the best,
rob