Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [mosquitto-dev] Advice needed (long, sorry)

Thanks guys for your advice,

Then,

- we will start working on AWS from our 2nd phase instead of the 3rd one.
- we will return to a multi-instance mosquitto infrastructure behind a load balancer and try to make our best to force the client to connet to the desired broker using an initial http request.
- we hope that the load balancer itself does not become the bottleneck now.
- we continue tuning linux kernel to scale better.

Thank you for all. Be sure that we will document everything better to be able to give you our results.

Nice to meet you "virtually" :-)

El 12/02/2015 19:36, "Manuel Domínguez Dorado" <manolodd@xxxxxxxxx> escribió:
Hi, attached you have sysctl.conf and mosquitto.conf files that we used for that.

Also:

Changes to /etc/rc.d/rc.local

defrt=`ip route | grep "^default" | head -1`
ip route change $defrt initcwnd 10

And changes to /etc/security/limits.conf

* soft nofile 100000
* hard nofile 100000

And changes to /etc/pam.d/sshd

session required pam_limits.so


And changes to /etc/ssh/sshd_config

UsePAM yes




Sources we read to tune the aforementioned paramenters:


And... that's all. We configured dozens VM (20-30... I don't remember, but more or less), each one running a java software that creates hundreds of threads each one of them creating a paho client and connecting to the broker over TLS. No subscriptions and no publishing. When we started publishing everything went wrong.  Everything on our servers using gigabit LAN.

Sorry (and sorry also Stefano), we did the test without any other planification as we did lots of tests of different types at that moment. As we realized that we were far of 100K connections we tried other strategies. So we have no a very formalized and structured documentation, nor charts, neither a dataset... :-( I remember that we achieved 23.2xx connections.

Hope it helps.



2015-02-12 12:00 GMT+01:00 Karl Palsson <karlp@xxxxxxxxxxxx>:

Manuel Domínguez Dorado <manolodd@xxxxxxxxx> wrote:

> But Mosquitto...
>
> We have read some experienced os people using Mosquitto sustaining 100K
> concurrent connection and more. But in our test we have reached ~25K
> concurrent connections at maximum, which is not enough (and operating
> not
> very well, to be realistic).
>
> Could you give us some advice on how to improve these numbers at least
> to
> be able to reach our 100K users? We don't know, but maybe 20K concurrent
> connections are enough to support 100K users that are not connected at
> the
> same time.
>

I'm actually curious to hear about how you configured mosquitto and your
servers to get 20k connections at all.  I couldn't get anywhere near
that, and the people who'd claimed to reach 100k on mosquitto here in
the past have never replied to any requests for further information.

Sincerely,
Karl Palsson



--
---
Manuel Domínguez Dorado
ingeniero ARROBA ManoloDominguez PUNTO com
http://www.ManoloDominguez.com

Back to the top