[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [paho-dev] sharing subscriptions among connected group members
|
It's worth asking this question on the Mosquitto mailing list
where there will be more experience in using this feature on
Mosquitto and other brokers. Which client receives which message
is not determined by the spec, so the behavior you're asking for
is allowed. It's one way it could work.
The spec says:
A Session can explicitly detach itself from a Shared
Subscription by sending an UNSUBSCRIBE Packet that contains the
full Shared
Subscription Topic Filter. Sessions are also detached from the
Shared Subscription
when they terminate.
I think that brokers will probably start by taking the approach,
as I did when I implemented the broker in paho.mqtt.testing to
ignore the connected status of a shared subscriber. This is
because that's required by MQTT for non-shared subscriptions.
When the session expires (the session expiry value on connect),
the subscription will be removed, so if you connected with session
expiry=0, messages wouldn't be queued once the client has
disconnected. The downside of that approach is not being able to
fully process QoS 1 or 2 messages if the processing is
interrupted. So, the messages won't be queued up forever (or
shouldn't be), but until the session expires.
On rerouting of unacked messages to other subscribers. This is
allowed for QoS 1 in the spec, but not for QoS 2, because it could
result in duplicated messages which is forbidden for QoS 2.
On 08/04/2021 13:54, Peter Silva wrote:
If you're telling me that is, according to the standard,
how it is supposed to work... that is good and helpful.
yes. I can do interop tests... I figure mosquitto is the lowest
bar, in terms of being the lightest one likely the most
conservative in implementation. If different brokers do
different things... That is sub-optimal, as I want it to work
with at least a good subset of them. Is there a list
somewhere that compares brokers for these sorts of things? I
have experimented with: emqx, verneMQ, and mosquitto, so far.
I can add Hive, I guess. In earlier work they seemed to all
work fine, but I wasn't getting as detailed as now. The
wikimedia MQTT implementation page is long. MQTT's origins
are in SCADA, and with mqtt v5 it has much better support for
message queueing apps that are a little different... hard to
know which brokers to try for an app from a message queueing
background. We want reliable messaging (no loss, yes
queueing) good load balancing (dynamic shared subscriptions?)
and error recovery (which seems to mean different things to
different people.) It's very close to amqp stuff... just can't
quite tell if it is really there yet.
more concretely: It occurred to me that brokers might want
to load balancing based on ReceiveMaximum... which mosquitto
appears to do... in that it doesn't queue data for any client
that has reached it's receive Maximum... but that seems to
mean that if there are no clients connected who have <
receiveMaximum messages pending... it looks like it drops the
message. Sigh... that's not what I'd prefer either... It
should queue as long as it sees the client is connected, and
receiveMaximum should control the number of messages in
flight.
I'm searching for a way to provide backpressure... My only
guess for now is to disconnect temporarily...
Hi Peter,
the behavior you described that you want
only online clients to receive messages is the
behavior that should be expected with Shared
Subscriptions. Also that unacked messages are
redistributed among subscribers that are online is
something that some brokers implement. The MQTT 5 spec has some
room for implementation-specific behavior.
If you want to verify that what you are
seeing is not an application bug but the behavior you
see is a mosquitto implementation detail, you could
also test against other (open source) MQTT brokers
like HiveMQ Community Edition
or you could quickly test against a free online MQTT
broker like HiveMQ Cloud.
Best,
Dominik
On 8 Apr 2021, at 2:52, Peter Silva wrote:
Hi there,
Example: client A, B, and C establish sessions
subscribed to:
$share/G/test/#
All three clients are part of group G and get 1/3
messages for topic:
test/#
So far, so good.. but let's say B dies... ... so
sad... Once we have
determined that B is dead, I would want the broker
to send half the
messages to A and the other half to C.
My tests are really short (a few minutes) so perhaps
I didn't give it
enough of a chance, but as far as I can tell, it
continues to split 1/3,
1/3, 1/3 forever queueing up for the B that is gone.
If D comes in... then
it gets 1/4, as do A and C, and 1/4 piles up with B.
I imagine I could
connect with B again, and do an unsubscribe, to tell
the broker that B has
left the group... but that relies on b being healthy
enough to do so.
It would be nice if there were a way to tell the
broker to round-robin
among the clients that are currently connected? so
the collective work
continues at 1/3 slower aggregate rate, but without
queueing 1/3 of the
work forlornly for B to return. When you decide B is
dead, it should also
re-distribute the queue of unacknowledged messages
in it's queue to the
remaining group members.
Leaving a queue for B will mean the load will be
asymmetrical for some time
after it returns, as it will be processing its
backlog.
Does this seem reasonable, and is there a way to
convince a broker to
behave this way (e.g. mosquitto?)
_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/paho-dev
_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/paho-dev
_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/paho-dev
--
Ian Craggs
Eclipse IoT PMC; Eclipse Paho Project Lead; OASIS MQTT TC Member