Hi Roger
Thanks for feedback on my ideas - my input in blue below
I see you're being more ambitious with your requests than everybody else :) Being serious, I would be very keen on improving the scalability of course.
Think a first step with scaling is - what is the current limitation of mosquitto that you know is holding you back from scaling on a single server - I know Michael is looking at ZMQ which can help with nodes keeping in sync, but you know mosquitto better and may be able to shed some light on area(s) you feel is holding mosquitto back and sure you have some plans of your own on how to make this go to the next level. With AWS spot EC2 instance we can maybe look at throwing some load onto mosquitto to see/monitor and revisit scaling next mosquitto for load tests, but the key is to understand the current limitation and maybe some idea(s)
2. Integration with other brokers like Kafka, RabbitMQ for passing MQTT to say AMQP - bridge but more on code level - allowing routing/transformation
Could you give more of an example of what you mean here please?
:) Sure - was thinking that if mosquitto can provide some plugins for transforming MQTT message into other protocols, like STOMP, AMQP, WS, various other players can join in using Mosquitto as a front-end broker for handling the MQTT connection to devices but provide topic to other protocols - classic example will be to use mosquitto as MQTT and hook this up with say RabbitMQ to do complex routing/exchange and even topics to queue, allowing workers to handle the load - so lets take an example - you have 10 000 000 devices around the globe taking temperture readings - the all publish to mosquitto on topic TEMP/device_id - mosquitto will be the MQTT broker keeping the socket open but with an integration gateway can almost like PUBLISH to RabbitMQ to a queue called TEMP - this allows anybody on the RabbitMQ side access to that queue, in a different manner - distrusted processing of taking the TEMP of the queue and dumping in a mongodb, updating redis,ext ...the benefit with this approach is - with topics, you can not use workers - everybody gets the same message - but if we use mosquitto as a gateway between devices and queues, you will really see mosquitto being used even more ^_^ - it allows other systems, access to MQTT IoT, without worrying to much about the protocol stack, keeping TCP/IP connections open and everybody wins - queues stays queues, and mosqutto stays mosquitto with MQTT power, and not mixing open sockets/MQTT with queues really
4. BIG options is router - allowing device to be route to dedicated mosquitto broker - diff to clustering :)...clustering is complex, routing is easy
5. Rules engine for routing in-comming message
6. MONITORING options - keeping an eye on load and troughput for tests and support
They all sound like interesting ideas as well.
4. Routing in my mind is something that can help with scaling - let say you have a routing service, sort of load balancing the connections between various mosquitto instances, - if each mosquitto instance can keep the rest in sync (zmq...or like erlang's way of nodes keeping in sync), you can end up with a massive single broker ....my TCP/IP knowledge is to little to say if the following is possible - but worth thinking about - why not change the tcp/ip frame to route a device to a dedicated mosquitto on dedicated port...? Meaning, the routing agent change the frame+tcp info so that when it's sent back the device will automatically talk to a mosquitto boker for that time....not sure fi this is possible.... but then you can route devices around the globe to dedicated instance - even as far as an one of many instance on single server ^_^
5. An very handy option with rules engine can be - "sort of lambda on data" - "if avg_temp > 65 and hit_count = 3 push msg to SQS on AWS for all support engineer, device is running like the devil. - even better will be to "route" message to say a topic that can take avg over time with rules - ...applied in real time - With this workflow can also play a very handy role
6. On the monitoring side - just a basic option to push mosquitto metrics to say something like Graphite (open source) will help already - and will be even more handy when more nodes is added - any metrics will also go a long way with point 1 - scaling things - you need to measure and revisit your changes - only way will be to store the metrics of the server in some sort o "graph" database that can shed some light on why on earth your now struggling with 5000 user load...Graphite is but one example -- sure many other metric logging system exist today
Hope this helps Roger - keen on trying V1.4 when it release ;)
Warm regards
Izak
