Re: [mosquitto-dev] Advice needed (long, sorry)

[Manuel Domínguez Dorado <manolodd@xxxxxxxxx>]

Hi, many thanks.

if everybody agrees on going to AWS from day 1, sure it is the more reasonable option. We'll think on it. However all linux kernel tuning has been already done (a very hard work).

My question is... Going on AWS or not, we will still have the same problems to scale Mosquitto as AWS adds nothing that solve that, isn't it? Perhaps I will be able to see it clearer after analysing what Michael has commented.

Thanks again.

2015-02-12 0:14 GMT+01:00 Oegma2 <oegma2@xxxxxxxxx>:

Hi Manuel

I agree with David on going for AWS from day 1 - I've been using AWS the last 8 months designing a very large scale system that can auto scale using ELB, Route53, EC2 instance, doing load testing, using metrics from AWS and so much more options, ranging from Redis, to Posgtres, Mongo, to Docker, S3 ...te list of options you get using AWS on a technology stack is scary! Not to mention options like DNS routing for lowest latency, DR-sites, power outages, replication, backups, ext - and you can start small using a t1.micro EC2 linux ubuntu instance for running mosquitto on it and also run HAProxy ...one key thing if you want to use AWS's ELB in future or just try out - remeber that your mqtt ping time must be 60sec or below, or the AWS elb will drop your packets....

If you go to AWS, you get Redis and memcache aslo @price, but for basic learning you can run Redis on EC2 servers

So well worth checking out what the ROI will be using an AWS server 

You can also scale up from smaller EC2 instances - so start small and move up to bigger servers - only "learning" part here will be to find the best match instance for you solution - things like advance networking vs memory, vs cpu all play a role later in finding the correct server

Then you have to think about tweaking kernels levels - your stock std linux kernel will not handle much concurrent connections without some lower level changes...things like file handles, tcp-frame buffer ..

^_^ IN SHORT - try AWS as well

On Thu, Feb 12, 2015 at 12:50 AM, Manuel Domínguez Dorado <manolodd@xxxxxxxxx> wrote:

Hi, thank you.

we have not tried redis. We could try it. But we tried multiple Mosquittos behind haproxy loadbalancing connections to the less occuped broker (instead of roundrobin). There is a problem: as we need persistence (clear session = false), after a disconnection several messages were retained in the broker and the user not allways reconnect to the same broker and therefore the broker cumulates lots of messages that are not longer valid  and the client does not receive them after reconnecting. We tried using a brokers frontend line (5 brokers) bridging to a central one. Also, having only the frontend line. But persisted messages are a problem because of sessions, that are exclusive of a single mosquitto.

Maybe it would be good a hook to allow session migration between Mosquittos.

I've also do some proofs having only a Mosquitto with several listener and haproxy roundrobin on their ports. But we did not notice that things went better.

We'll read more about redis. Thank you so much.

Hi Manuel, I agree with Stefano - done a lot of detailed work 

Just though of an option - never tried it but maybe worth while looking into using Redis with more than one mosquitto instance - using HAPraox as load balancing you can run say 3-5 mosquttos with a redis layer behind mosquitto ....so in short HA --> Mosquitto --> Redis <-- other sub/pub on server side.....

On Wed, Feb 11, 2015 at 11:47 PM, Stefano Costa <stefano.costa@xxxxxxxxxxx> wrote:

> Il giorno 11/feb/2015, alle ore 22:15, Manuel Domínguez Dorado <manolodd@xxxxxxxxx> ha scritto:
>
>
> Could you give us some advice on how to improve these numbers at least to be able to reach our 100K users? We don't know, but maybe 20K concurrent connections are enough to support 100K users that are not connected at the same time.

Hi Manuel, it seems to me that you studied all details very well, good architecture as a starting point. But in order for people here to give useful advices I suspect that an important detail is missing: apart from "connected" and "existing" users, what's the messages rate you expect to have per user / per second (or minute, hour) under normal conditions and maximum?
_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev

_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev

_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev

--

---
Manuel Domínguez Dorado
ingeniero ARROBA ManoloDominguez PUNTO com
http://www.ManoloDominguez.com