Hi,
I'm suspecting that my paho.mqtt.cpp client is not freeing
memory of subscribed messages. I modified slightly the sample
code paho.mqtt.cpp/src/samples/sync_subscribe.cpp;
First modification: Using qos = 0, should not matter here,
perhaps:
< const int QOS = 0;
---
> const int QOS = 1;
The second modification: Subscribe in "void connected()"
did not work for me, therefore added "cli.subscribe(TOPIC,
QOS);" into the main loop after making sure that connection is
on.
I'm sending large 1MB messages with 25 Hz. Memory increase
used by the process can be seen very easily with this rate.
Everything works just fine until all the system memory
including swap is used :-)
For reference, paho.mqtt,.c client has been working for
days without process memory grow with similar test setup.
System: Ubuntu 16.04 4.15.0, broker tested with RabbitMQ
and Emqx, same result. Client paho.mqtt.cpp is 1.0.1 (
9d53436).
Any tips for debugging? Do I miss some free() in my client?
See below some traces from valgrind commands as a starter.
BR, Petri
$ valgrind --tool=massif ./sync_subscribe # Here is the
command for data collection, another client is publishing 1MB
messages with 25 Hz
...
$ grep mem_heap_B massif.out.<pid of the previous> #
Grepping some suspicious memory usage
mem_heap_B=0
mem_heap_B=9311263
mem_heap_B=15312373
mem_heap_B=22313668
mem_heap_B=32315518
mem_heap_B=38317140
mem_heap_B=45318435
mem_heap_B=51319497
mem_heap_B=62321532
mem_heap_B=73324079
mem_heap_B=81325559
mem_heap_B=91327409
mem_heap_B=98329216
mem_heap_B=108331066
mem_heap_B=114332176 <<----------------- *)
mem_heap_B=120333286
Snapshot from massif.out:
#-----------
snapshot=14
#-----------
time=249949132
mem_heap_B=114332176 <<----------------- *)
mem_heap_extra_B=59496
mem_stacks_B=0
heap_tree=detailed
n2: 114332176 (heap allocation functions) malloc/new/new[],
--alloc-fns, etc.
n2: 112000501 0x523A8DB: void
std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char>
>::_M_construct<char const*>(char const*, char
const*, std::forward_iterator_tag) (in
/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.26)
n1: 112000112 0x4ED1221: void
__gnu_cxx::new_allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>::construct<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
char const*&, unsigned
long&>(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char>
>*, char const*&, unsigned long&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED1196: void
std::allocator_traits<std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
> >::construct<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
char const*&, unsigned
long&>(std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>&, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char>
>*, char const*&, unsigned long&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED1103:
std::_Sp_counted_ptr_inplace<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>,
(__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<char
const*&, unsigned
long&>(std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>, char const*&, unsigned long&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED0F9A:
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>, char const*&, unsigned
long&>(std::_Sp_make_shared_tag,
std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char>
>*, std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
> const&, char const*&, unsigned long&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED0E70:
std::__shared_ptr<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
(__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>, char const*&, unsigned
long&>(std::_Sp_make_shared_tag,
std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
> const&, char const*&, unsigned long&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED0DFF:
std::shared_ptr<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>::shared_ptr<std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>, char const*&, unsigned
long&>(std::_Sp_make_shared_tag,
std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
> const&, char const*&, unsigned long&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED0D0E:
std::shared_ptr<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>
std::allocate_shared<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
>, char const*&, unsigned
long&>(std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
> const&, char const*&, unsigned long&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED0BB2:
std::shared_ptr<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >
> std::make_shared<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
char const*&, unsigned long&>(char const*&,
unsigned long&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED09B3:
mqtt::buffer_ref<char>::buffer_ref(char const*, unsigned
long) (in /usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED064D:
mqtt::message::set_payload(void const*, unsigned long) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4ED0053:
mqtt::message::message(mqtt::buffer_ref<char>,
MQTTAsync_message const&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EC15FB: void
__gnu_cxx::new_allocator<mqtt::message>::construct<mqtt::message,
mqtt::buffer_ref<char>, MQTTAsync_message
const&>(mqtt::message*,
mqtt::buffer_ref<char>&&, MQTTAsync_message
const&) (in /usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EC0663: void
std::allocator_traits<std::allocator<mqtt::message>
>::construct<mqtt::message,
mqtt::buffer_ref<char>, MQTTAsync_message
const&>(std::allocator<mqtt::message>&,
mqtt::message*, mqtt::buffer_ref<char>&&,
MQTTAsync_message const&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EBF497:
std::_Sp_counted_ptr_inplace<mqtt::message,
std::allocator<mqtt::message>,
(__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<mqtt::buffer_ref<char>,
MQTTAsync_message
const&>(std::allocator<mqtt::message>,
mqtt::buffer_ref<char>&&, MQTTAsync_message
const&) (in /usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EBDB3E:
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<mqtt::message,
std::allocator<mqtt::message>,
mqtt::buffer_ref<char>, MQTTAsync_message
const&>(std::_Sp_make_shared_tag, mqtt::message*,
std::allocator<mqtt::message> const&,
mqtt::buffer_ref<char>&&, MQTTAsync_message
const&) (in /usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EBBD2E:
std::__shared_ptr<mqtt::message,
(__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<mqtt::message>,
mqtt::buffer_ref<char>, MQTTAsync_message
const&>(std::_Sp_make_shared_tag,
std::allocator<mqtt::message> const&,
mqtt::buffer_ref<char>&&, MQTTAsync_message
const&) (in /usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EBA749:
std::shared_ptr<mqtt::message>::shared_ptr<std::allocator<mqtt::message>,
mqtt::buffer_ref<char>, MQTTAsync_message
const&>(std::_Sp_make_shared_tag,
std::allocator<mqtt::message> const&,
mqtt::buffer_ref<char>&&, MQTTAsync_message
const&) (in /usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EB8DCC:
std::shared_ptr<mqtt::message>
std::allocate_shared<mqtt::message,
std::allocator<mqtt::message>,
mqtt::buffer_ref<char>, MQTTAsync_message
const&>(std::allocator<mqtt::message> const&,
mqtt::buffer_ref<char>&&, MQTTAsync_message
const&) (in /usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EB71E0:
std::shared_ptr<mqtt::message>
std::make_shared<mqtt::message,
mqtt::buffer_ref<char>, MQTTAsync_message
const&>(mqtt::buffer_ref<char>&&,
MQTTAsync_message const&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EB5838:
mqtt::message::create(mqtt::buffer_ref<char>,
MQTTAsync_message const&) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x4EB0F24:
mqtt::async_client::on_message_arrived(void*, char*, int,
MQTTAsync_message*) (in
/usr/local/lib/libpaho-mqttpp3.so.1.0.1)
n1: 112000112 0x5AD1D27:
MQTTAsync_deliverMessage (in
/usr/local/lib/libpaho-mqtt3as.so.1.3.0)
n1: 112000112 0x5AD2028:
Protocol_processPublication (in
/usr/local/lib/libpaho-mqtt3as.so.1.3.0)
n1: 112000112 0x5AD8D23:
MQTTProtocol_handlePublishes (in
/usr/local/lib/libpaho-mqtt3as.so.1.3.0)
n1: 112000112 0x5AD5452:
MQTTAsync_cycle (in /usr/local/lib/libpaho-mqtt3as.so.1.3.0)
n1: 112000112 0x5AD0064:
MQTTAsync_receiveThread (in
/usr/local/lib/libpaho-mqtt3as.so.1.3.0)
n1: 112000112 0x674D6B8:
start_thread (pthread_create.c:333)
n0: 112000112 0x57FC41B: clone
(clone.S:109)
n0: 389 in 13 places, all below massif's threshold (1.00%)
n0: 2331675 in 42 places, all below massif's threshold
(1.00%)
