| Paho java client - cannot create more than 5000 connections [message #1723882] |
Thu, 18 February 2016 12:11  |
Eclipse User |
|
|
|
Hi,
I am trying to create multiple connections to mqtt broker (mosquitto) using Paho's java client. But After ~5000 connections, I get error:
"java.lang.OutOfMemoryError: unable to create new native thread".
This is a big VM with 16 CPU/ 16g ram.
- I have already increased the ulimit parameters on this ubuntu box.
- tried creating multiple connections within one java thread
- one connect per java thread.
None of the above has helped. Based on some of the other forums, people have been able to make close to 100k connections at a time from one box. What am I missing ?
Here is my code:
for (int i = 0; i < connectionCountPerThread ; i++) {
MqttClient newClient = new MqttClient("tcp://" + url, credsList.get(i).getClientId());
sampleClient.add(newClient);
MqttConnectOptions connOpts = new MqttConnectOptions();
connOpts.setCleanSession(true);
connOpts.setPassword(credsList.get(i).getPassword().toCharArray());
connOpts.setUserName(credsList.get(i).getUsername());
newClient.connect(connOpts);
newClient.setTimeToWait(1000000);
}
MqttMessage message1;
int qos = 0;
MockObservations mo = new MockObservations();
for (int i = 0 ; i < msgCount ; i++) {
for (int conn = 0; conn < connectionCountPerThread ; conn++) {
message1 = new MqttMessage(mo.getSensorReading(credsList.get(conn).getPayloadType()).getBytes());
message1.setQos(qos);
sampleClient.get(conn).publish(credsList.get(conn).getTopicId(), message1);
try {Thread.sleep(5000); } catch (Exception e) {}
}
727b8ad00873:~$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 128311
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 3000000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 200000
virtual memory (kbytes, -v) unlimited
|
|
|
|
|
| Re: Paho java client - cannot create more than 5000 connections [message #1724845 is a reply to message #1724270] |
Fri, 26 February 2016 09:09  |
| Eclipse User |
|
|
|
|
You might find that the C client is more lightweight and so you could fit more onto one box. An alternative would be to set up a number of either Mosquitto or RSMB brokers in bridge mode with bridges connected to bridges. You should be able to get enough connected that would create enough connections for your needs. IIRC, RSMB only takes up around 200K of memory per instance and should be able to create a couple of hundred connections!
|
|
|
Powered by
FUDForum. Page generated in 0.03534 seconds
Home