Home » Eclipse Projects » 4DIAC - Framework for Distributed Industrial Automation and Control » mqtt(Content of mqtt message is not correct)
| | |
Re: mqtt [message #1754400 is a reply to message #1754289] |
Fri, 17 February 2017 09:39 |
|
Great.
Just for clarification. The main reason for this additional parameter is that mqtt does not define a payload format. With the additional param you specify in 4diac an so called communication layer to be used on top of MQTT. This communication layer is now in charge of taking the data from IEC 61499 and generate an according payload format or take the payload and transform it for the FB's outputs. the raw-layer is just accepting strings and sends the raw string content. If you need other payload formats you can also write your own layer. Please have a look on the communication architecture documentation which can be found in the FORTE development section of 4diac's documentation.
Cheers,
Alois
|
|
|
Re: mqtt [message #1756319 is a reply to message #1754400] |
Wed, 15 March 2017 11:06 |
Adrian Orive Messages: 38 Registered: March 2017 |
Member |
|
|
I will ask here to keep the forum tidy, but if you think this should be a separate post, split it or let me know so I can do it myself.
I was trying to communicate over MQTT (for that, I compiled Paho MQTT C library and added the following command line arguments to cmake in the setup_posix.sh: -DFORTE_MODULE_SysFs=ON -DFORTE_COM_PAHOMQTT=ON -DFORTE_COM_PAHOMQTT_DIR="../../../paho.mqtt.c") and added a publisher and subscriber FBs as shown in the images:
EDIT: The http:// should not be there and the port should be 1883. These are just typos when restoring to the first version after the tests I carried out and they are not the source of the problem.
I have also tries swapping raw[] for fbdk[] as suggested in MQTTComLayer.h (line 21):
And the only difference is that I'm getting a INVALID_ID message instead of TERMINATED in both cases.
MQTTComLayer.cpp only has 3 posible spots for TERMINATED, messages as shown in the next image: the first one is overwritten before returning, and the second and third ones are for MQTT clients and servers which are not implemented. INVALID_ID is just present once and can also be seen in the image.
My guess is that the error comes from a lower layer, so I will inspect raw and fbdk layers now but if you guys have any idea about what could be the cause please let me know.
UPDATE: I only found TERMINATED in rawdatacomlayer.cpp as seen in the following image.
Correct me if I'm mistaken, but I think raw only works for publishers with at least one data input, so this means that fbdk should be my option. This also tells me that the ID I showed in the first 2 images must have some kind of error, are domain names allowed or do I need to give the IP address?
UPDATE 2: I've tried to use the IP address and it stays yielding the same.
[Updated on: Wed, 15 March 2017 11:46] Report message to a moderator
|
|
|
Re: mqtt [message #1756364 is a reply to message #1756319] |
Wed, 15 March 2017 20:52 |
|
Thank for the elaborate description and analysis. You are right the raw layer is currently only working with a single in or output and this in or output has to be of type STRING. The problem for PUB and sUb 0 is that the raw layer wouldn't know what to generate as payload for the MQTT messsage. With FBDK a 0x5 will be send for pure event PUBs. So this could be better on your case.
Regarding the invalid ID I see two issues. First of all I think you mixed up ClientID and Topic name. The ClientID should be the second mqtt parameter and currently it needs to be the same for all MQTT pubs and subs within one device. We know this is a little bit inconvenient and we should improve this. The second thing can really be that the domain names is also an issue. However this depends on the paho lib and if your device supports name resolving. For example on the Bosch Rexroth PLC name resolving was not working and therefore I had to replace the the domain names with ips.
I hope this helps.
|
|
| | |
Re: mqtt [message #1756435 is a reply to message #1756381] |
Thu, 16 March 2017 20:09 |
|
Adrian Orive wrote on Thu, 16 March 2017 08:29
FBDK then. Is it possible to use a single MQTT layer? Just including as ID:
mqtt[tcp://ip-address:port, client-id, topic]
No. The problem is that MQTT does not define a payload format and therfore you always need in 4diac a layer on top of MQTT layer.
Adrian Orive wrote on Thu, 16 March 2017 08:29
The PUBLISH_0 and SUBSCRIBE_0 FBs are in different HW devices, each with its own client id (4diac-pub-test and 4diac-sub-test respectively) and they are both using the same topic (4diac-events). The idea is that one device triggers an event on the other device, as a first test for MQTT communication with 4DIAC.
Not sure if i got your idea. This will only work if the pub and sub are mapped on two different devices. In your screenshot both where mapped onto the same device and then you need to use the same clientid for both FBs otherwise the second FB will not work.
Adrian Orive wrote on Thu, 16 March 2017 08:29
I think that the problem may be inside the MQTTHandler::registerLayer() method. Is there anyhow that I can check that MQTT paho was properly linked to FORTE?
I have a hard time to believe this as the MQTT is used by many users which din't had such problems.
|
|
|
Re: mqtt [message #1756458 is a reply to message #1756435] |
Fri, 17 March 2017 08:41 |
Adrian Orive Messages: 38 Registered: March 2017 |
Member |
|
|
Alois Zoitl wrote on Thu, 16 March 2017 20:09Adrian Orive wrote on Thu, 16 March 2017 08:29
FBDK then. Is it possible to use a single MQTT layer? Just including as ID:
mqtt[tcp://ip-address:port, client-id, topic]
No. The problem is that MQTT does not define a payload format and therfore you always need in 4diac a layer on top of MQTT layer.
I understand that. I meant for PUBLISH_0 and SUBSCRIBE_0 FBs, as they do not need any payload. I tried it and worked just fine, so I can confirm that PUBLISH_0 and SUBSCRIBE_0 MQTT FBs do not require an additional communication layer.
Alois Zoitl wrote on Thu, 16 March 2017 20:09Adrian Orive wrote on Thu, 16 March 2017 08:29
The PUBLISH_0 and SUBSCRIBE_0 FBs are in different HW devices, each with its own client id (4diac-pub-test and 4diac-sub-test respectively) and they are both using the same topic (4diac-events). The idea is that one device triggers an event on the other device, as a first test for MQTT communication with 4DIAC.
Not sure if i got your idea. This will only work if the pub and sub are mapped on two different devices. In your screenshot both where mapped onto the same device and then you need to use the same clientid for both FBs otherwise the second FB will not work.
Those FBs are in two different HW devices, sorry if I didn't make that clear from the start. They are both with a white background as they are not defined in the app view, but inside each HW resource. I already show the part of the code that raises an INVALID ID error if either the host or the client id of every mqtt FB in a HW device are different.
Alois Zoitl wrote on Thu, 16 March 2017 20:09Adrian Orive wrote on Thu, 16 March 2017 08:29
I think that the problem may be inside the MQTTHandler::registerLayer() method. Is there anyhow that I can check that MQTT paho was properly linked to FORTE?
I have a hard time to believe this as the MQTT is used by many users which din't had such problems.
Sorry, I didn't mean that there was a bug there, I was just saying that I suspect that the error was being raised from that function. I have it working properly both for empty payloads with just a single communication layer (mqtt[...]) and for strings as payloads with two communication layers (raw[].mqtt[...]).
[Updated on: Fri, 17 March 2017 09:34] Report message to a moderator
|
|
|
Re: mqtt [message #1756501 is a reply to message #1756458] |
Fri, 17 March 2017 22:13 |
|
Adrian Orive wrote on Fri, 17 March 2017 08:41
I understand that. I meant for PUBLISH_0 and SUBSCRIBE_0 FBs, as they do not need any payload. I tried it and worked just fine, so I can confirm that PUBLISH_0 and SUBSCRIBE_0 MQTT FBs do not require an additional communication layer.
I don't know that MQTT allows also empty messages. Interesting and good to know. But it is a bit of a hack and can be fragile. Maybe it would be better to expand the raw layer also to PUB/SUB 0.
Adrian Orive wrote on Fri, 17 March 2017 08:41
Those FBs are in two different HW devices, sorry if I didn't make that clear from the start. They are both with a white background as they are not defined in the app view, but inside each HW resource. I already show the part of the code that raises an INVALID ID error if either the host or the client id of every mqtt FB in a HW device are different.
Sorry I mixed up the screenshots. Yes with your explanation it your intention and setup is clearer.
Adrian Orive wrote on Fri, 17 March 2017 08:41
Sorry, I didn't mean that there was a bug there, I was just saying that I suspect that the error was being raised from that function. I have it working properly both for empty payloads with just a single communication layer (mqtt[...]) and for strings as payloads with two communication layers (raw[].mqtt[...]).
That's good to here. But still I'm a bit curios why it didn't work in the first place.
[Updated on: Fri, 17 March 2017 22:15] Report message to a moderator
|
|
|
Re: mqtt [message #1757806 is a reply to message #1756501] |
Mon, 20 March 2017 07:34 |
Adrian Orive Messages: 38 Registered: March 2017 |
Member |
|
|
Alois Zoitl wrote on Fri, 17 March 2017 22:13Adrian Orive wrote on Fri, 17 March 2017 08:41
I understand that. I meant for PUBLISH_0 and SUBSCRIBE_0 FBs, as they do not need any payload. I tried it and worked just fine, so I can confirm that PUBLISH_0 and SUBSCRIBE_0 MQTT FBs do not require an additional communication layer.
I don't know that MQTT allows also empty messages. Interesting and good to know. But it is a bit of a hack and can be fragile. Maybe it would be better to expand the raw layer also to PUB/SUB 0.
Empty payloads are allowed in MQTT's specification, so I would not consider it to be fragile. Using a single layer sounds fine to me, but if you want to have 2, fbdk[].mqtt[...] also works. From a network efficiency point of view this 2 layer messages are sending non-needed bytes that in a constrained application may be relevant.
Alois Zoitl wrote on Fri, 17 March 2017 22:13That's good to here. But still I'm a bit curios why it didn't work in the first place.
I was using a WSTRING data output for one of the SUBSCRIBE_1 FBs using the raw data layer which requires STRING outputs.
So my conclussions are:
- A single layer setup (mqtt[...]) can only be used for PUBLISH_0 and SUBSCRIBE_0 FBs as MQTT does not enforce any payload.
- The raw data layer can be used in a two layer setup (raw[].mqtt[...]) only for PUBLISH_1 and SUBSCRIBE_1 FBs as this layer enforces a single input/output. Only STRING inputs or outputs are allowed, not even WSTRING ones will work.
- The fbdk layer can be used as two layer alternative (fbdk[].mqtt[...]) for PUBLISH_0 and SUBSCRIBE_0 FBs, including a non-meaningful payload. This increases both the message size and the processing time so I would not recommend it as it is not needed.
- Using the fbdk layer for higher ammount of inputs or outputs has not been tested.
|
|
|
Re: mqtt [message #1757864 is a reply to message #1757806] |
Mon, 20 March 2017 21:15 |
|
Adrian Orive wrote on Mon, 20 March 2017 07:34Alois Zoitl wrote on Fri, 17 March 2017 22:13Adrian Orive wrote on Fri, 17 March 2017 08:41
I understand that. I meant for PUBLISH_0 and SUBSCRIBE_0 FBs, as they do not need any payload. I tried it and worked just fine, so I can confirm that PUBLISH_0 and SUBSCRIBE_0 MQTT FBs do not require an additional communication layer.
I don't know that MQTT allows also empty messages. Interesting and good to know. But it is a bit of a hack and can be fragile. Maybe it would be better to expand the raw layer also to PUB/SUB 0.
Empty payloads are allowed in MQTT's specification, so I would not consider it to be fragile. Using a single layer sounds fine to me, but if you want to have 2, fbdk[].mqtt[...] also works. From a network efficiency point of view this 2 layer messages are sending non-needed bytes that in a constrained application may be relevant.
With fragile I meant that it is fragile from 4diacs' mqtt layer implementation. Which some further investigation may be needed.
Compared to the overhead of a TCP message sending one or two additional bytes seams to be negligible. One could for example send the event count that ocured so far. This would allow to detect if messages where lost on the way.
|
|
|
Re: mqtt [message #1757907 is a reply to message #1757864] |
Tue, 21 March 2017 15:53 |
Adrian Orive Messages: 38 Registered: March 2017 |
Member |
|
|
Alois Zoitl wrote on Mon, 20 March 2017 21:15Adrian Orive wrote on Mon, 20 March 2017 07:34Alois Zoitl wrote on Fri, 17 March 2017 22:13I don't know that MQTT allows also empty messages. Interesting and good to know. But it is a bit of a hack and can be fragile. Maybe it would be better to expand the raw layer also to PUB/SUB 0.
Empty payloads are allowed in MQTT's specification, so I would not consider it to be fragile. Using a single layer sounds fine to me, but if you want to have 2, fbdk[].mqtt[...] also works. From a network efficiency point of view this 2 layer messages are sending non-needed bytes that in a constrained application may be relevant.
With fragile I meant that it is fragile from 4diacs' mqtt layer implementation. Which some further investigation may be needed.
If the MQTT layer is the top most one, the sendData() function will be called with no arguments a 0 in as the second argument if PUBLISH_0 is being used so an empty payload will be sent. I don't think it needs further checking but it could be added.
Alois Zoitl wrote on Mon, 20 March 2017 21:15Compared to the overhead of a TCP message sending one or two additional bytes seams to be negligible. One could for example send the event count that ocured so far. This would allow to detect if messages where lost on the way.
TCP already offers you losless guarranty. However the event number may be useful for other reasons. The two bytes may be negligible compared with TCP overhead (20 bytes, 10%) in most situations but for ultra-optimized applications even these two bytes may count. The communication layers allow both options to be implemented and be the user the one who selects which to use.
[Updated on: Wed, 22 March 2017 07:40] Report message to a moderator
|
|
|
Re: mqtt [message #1757972 is a reply to message #1757907] |
Wed, 22 March 2017 12:51 |
|
Adrian Orive wrote on Tue, 21 March 2017 15:53
If the MQTT layer is the top most one, the sendData() function will be called with no arguments a 0 in as the second argument if PUBLISH_0 is being used so an empty payload will be sent. I don't think it needs further checking but it could be added.
The only reason I have some doubts is the pointer which is pointing to SD_1 which is not valid in case of size 0. Maybe there is a general issue here. Have to think this through a bit.
Adrian Orive wrote on Tue, 21 March 2017 15:53
TCP already offers you losless guarranty. However the event number may be useful for other reasons. The two bytes may be negligible compared with TCP overhead (20 bytes, 10%) in most situations but for ultra-optimized applications even these two bytes may count. The communication layers allow both options to be implemented and be the user the one who selects which to use.
This depends to whome the TCP connection is. In MQTT only to the broker. I noticed but can not confirm that if the subscriber is gone or connects later it does not get any information how many messages have been. Sometimes dont even get the latest value but only updates. But I've to say I'm not very into MQTT.
|
|
|
Re: mqtt [message #1757998 is a reply to message #1757972] |
Wed, 22 March 2017 16:30 |
Adrian Orive Messages: 38 Registered: March 2017 |
Member |
|
|
Alois Zoitl wrote on Wed, 22 March 2017 12:51Adrian Orive wrote on Tue, 21 March 2017 15:53
TCP already offers you losless guarranty. However the event number may be useful for other reasons. The two bytes may be negligible compared with TCP overhead (20 bytes, 10%) in most situations but for ultra-optimized applications even these two bytes may count. The communication layers allow both options to be implemented and be the user the one who selects which to use.
This depends to whome the TCP connection is. In MQTT only to the broker. I noticed but can not confirm that if the subscriber is gone or connects later it does not get any information how many messages have been. Sometimes dont even get the latest value but only updates. But I've to say I'm not very into MQTT.
I agree. Just as a side note: MQTT offers a way of getting messages previous to the connection called retained messages, but they can only store one message per topic. This is probably not implemented in the MQTT layer for 4DIAC as other MQTT features but should be easy to implement them.
[Updated on: Wed, 22 March 2017 16:32] Report message to a moderator
|
|
|
Re: mqtt [message #1758061 is a reply to message #1757998] |
Thu, 23 March 2017 08:30 |
|
Sounds interesting. If you like to do experiments and enhancments we are happy to hear about it. Also improvements to the MQTT layer are warmly welcome. In the docs there is a section on how you can rather easily contribute to FORTE using git and gerrit.
|
|
|
Goto Forum:
Current Time: Thu Apr 18 11:17:21 GMT 2024
Powered by FUDForum. Page generated in 0.02928 seconds
|