Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [mosquitto-dev] Advice needed (long, sorry)
A little more (@Oegma2):

I was originally suggesting a pure PaaS like Heroku / Azure / Dotcloud. I don't know which one of those can run mosquitto (can any of them? business opportunity?)

But if you choose to build on top of raw AWS or another infrastructure provider ("IaaS") such as Digital Ocean/Azure/etc., there are even layers there.
  • At the lowest level, you're using the IaaS basically as a substitute for bare-metal machines. You'd do this because you want maximum flexibility to migrate away from the provider to another one, your own colo, some mix of the two (check out Direct Connect on AWS), or even to sell an on-premise version of the software (huge hassle, but some people might want it - whether you want to sell that is your own business decision)
  • At the higher levels, most IaaS providers also have their own proprietary infrastructure components you can use e.g. SQS, DynamoDB, SES, RDS, etc. for AWS. The more you use this stuff, the closer you get to PaaS-like economics (low engineering spend, fast time to market, minimal headache but ongoing higher $$) but you get more locked-in to your provider. In practice I don't think people switch infrastructure providers much (just like how they don't switch databases that much) but it's something people fuss over anyway.
In any case, I'd be really curious to know whether any company is offering a hosted MQTT broker 2-4ms RTT from the major AWS regions, e.g. us-east-1/us-west-2, sort of like MongoEngine/MongoHQ/MongoLab do for MongoDB. If not, someone should build this.

On Wed, Feb 11, 2015 at 3:14 PM, Oegma2 <oegma2@xxxxxxxxx> wrote:
Hi Manuel

I agree with David on going for AWS from day 1 - I've been using AWS the last 8 months designing a very large scale system that can auto scale using ELB, Route53, EC2 instance, doing load testing, using metrics from AWS and so much more options, ranging from Redis, to Posgtres, Mongo, to Docker, S3 ...te list of options you get using AWS on a technology stack is scary! Not to mention options like DNS routing for lowest latency, DR-sites, power outages, replication, backups, ext - and you can start small using a t1.micro EC2 linux ubuntu instance for running mosquitto on it and also run HAProxy ...one key thing if you want to use AWS's ELB in future or just try out - remeber that your mqtt ping time must be 60sec or below, or the AWS elb will drop your packets....

If you go to AWS, you get Redis and memcache aslo @price, but for basic learning you can run Redis on EC2 servers

So well worth checking out what the ROI will be using an AWS server 
You can also scale up from smaller EC2 instances - so start small and move up to bigger servers - only "learning" part here will be to find the best match instance for you solution - things like advance networking vs memory, vs cpu all play a role later in finding the correct server

Then you have to think about tweaking kernels levels - your stock std linux kernel will not handle much concurrent connections without some lower level changes...things like file handles, tcp-frame buffer ..

^_^ IN SHORT - try AWS as well

On Thu, Feb 12, 2015 at 12:50 AM, Manuel Domínguez Dorado <manolodd@xxxxxxxxx> wrote:

Hi, thank you.

we have not tried redis. We could try it. But we tried multiple Mosquittos behind haproxy loadbalancing connections to the less occuped broker (instead of roundrobin). There is a problem: as we need persistence (clear session = false), after a disconnection several messages were retained in the broker and the user not allways reconnect to the same broker and therefore the broker cumulates lots of messages that are not longer valid  and the client does not receive them after reconnecting. We tried using a brokers frontend line (5 brokers) bridging to a central one. Also, having only the frontend line. But persisted messages are a problem because of sessions, that are exclusive of a single mosquitto.

Maybe it would be good a hook to allow session migration between Mosquittos.

I've also do some proofs having only a Mosquitto with several listener and haproxy roundrobin on their ports. But we did not notice that things went better.

We'll read more about redis. Thank you so much.

Hi Manuel, I agree with Stefano - done a lot of detailed work 

Just though of an option - never tried it but maybe worth while looking into using Redis with more than one mosquitto instance - using HAPraox as load balancing you can run say 3-5 mosquttos with a redis layer behind mosquitto ....so in short HA --> Mosquitto --> Redis <-- other sub/pub on server side.....



On Wed, Feb 11, 2015 at 11:47 PM, Stefano Costa <stefano.costa@xxxxxxxxxxx> wrote:

> Il giorno 11/feb/2015, alle ore 22:15, Manuel Domínguez Dorado <manolodd@xxxxxxxxx> ha scritto:
>
>
> Could you give us some advice on how to improve these numbers at least to be able to reach our 100K users? We don't know, but maybe 20K concurrent connections are enough to support 100K users that are not connected at the same time.

Hi Manuel, it seems to me that you studied all details very well, good architecture as a starting point. But in order for people here to give useful advices I suspect that an important detail is missing: apart from "connected" and "existing" users, what's the messages rate you expect to have per user / per second (or minute, hour) under normal conditions and maximum?
_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev


_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev

_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev


_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev



--
david r. albrecht
prefiat llc | cloud powered hardware

Back to the top