Home » Eclipse Projects » Eclipse Titan » IPL4 port TCP connection problem
| IPL4 port TCP connection problem [message #1749800] |
Mon, 12 December 2016 09:02  |
Eclipse User |
|
|
|
Hello everyone,
I have already used the IPL4 port to run CoAP tests on UDP and everything works ok.
Today I wanted to make the same thing, but with MQTT or HTTP (TCP), so I downloaded the corresponding Protocol Modules and the conversion works fine (at least from what I have seen), but on connection the TCP port reports an error:
MTC@egm-VirtualBox: entering f__IPL4__PROVIDER__connect: localhost:0 -> localhost:35000 / TCP
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: enter
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: create new socket: localhost:0 -> localhost:35000
MTC@egm-VirtualBox: IPL4: SetLocalSockAddr: locName: localhost loc_port 0 def_loc_host localhost, add_family AF_INET
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::setOptions: enter, number of options: 1
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::setOptions: sock: 7
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::setOptions: Socket option REUSEADDR on socket 7 is set to: 1
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::setOptions: TCP option TCP_KEEPCNT on socket 7 is set to: 20
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::setOptions: TCP option TCP_KEEPIDLE on socket 7 is set to: 180
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::setOptions: TCP option TCP_KEEPINTVL on socket 7 is set to: 60
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::setOptions: socket option SO_KEEPALIVE on socket 7 is set to: 1
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::setOptions: leave
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: sock: 7
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: error: Operation now in progress
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER: ConnAdd enter: type: 2, ssl_tls_type: 0, sock: 7, parentIx: -1
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::ConnAdd: new sockListSize: 16
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::ConnAdd: connId: 1
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::ConnAdd: connId: set ssl options for connId : 1
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::set_ssl_supp_option: set SSL options
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::ConnAdd: getpeername failed: Transport endpoint is not connected
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::ConnAdd: leave: sockListCnt: 1
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: EAGAIN: waiting in poll: fd 7 connId: 1
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: EAGAIN: poll returned: 1
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: EAGAIN: revents: 0x001D
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: EAGAIN: writable
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: Probing connection: getsockopt returned: 0, connection result code: 111
MTC@egm-VirtualBox: IPL4: f__IPL4__PROVIDER__connect: connect error: Connection refused
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::ConnDel: enter: connId: 1
MTC@egm-VirtualBox: IPL4: IPL4asp__PT_PROVIDER::ConnDel: fd: 7
MTC@egm-VirtualBox: { errorCode := ERROR_SOCKET (4), connId := omit, os_error_code := 111, os_error_text := "Connection refused" }
MTC@egm-VirtualBox: Dynamic test case error: Using the value of an optional field containing omit. (Transport endpoint is not connected)
The function that makes this connection looks like this:
function f_cf01Up() runs on ARMOURComponent{
var Socket_API_Definitions.Result vl_result;
map(mtc:IPL4, system:IPL4);
vl_result := IPL4asp_PortType.f_IPL4_connect(IPL4,
finalConnectionParams.remHost,
finalConnectionParams.remPort,
finalConnectionParams.locHost,
finalConnectionParams.locPort,
connectionId, //pl_connectionParams.connId,
{tcp := {}},
);
log(vl_result)
connectionId := vl_result.connId;
}
Am I missing something?
Thanks in advance.
Best regards,
Naum
|
|
| | | | |
| Re: IPL4 port TCP connection problem [message #1749847 is a reply to message #1749812] |
Tue, 13 December 2016 03:34  |
| Eclipse User |
|
|
|
Hi Naum.
the two logs are not the same;
the first one shows that the connection was refused for some reason:
MTC@egm-VirtualBox: { errorCode := ERROR_SOCKET (4), connId := omit, os_error_code := 111, os_error_text := "Connection refused" }
in the second case, connection was established , a connId is returned, and the error happened somewhere in the TTCN-3 code:
17:08:58.879219 - { errorCode := omit, connId := 1, os_error_code := omit, os_error_text := omit }
17:08:58.879345 - Dynamic test case error: Using an unbound optional field. (Operation now in progress)
In your code snippet the initial value of connectionId is unclear; I suggest the following mod:
function f_cf01Up() runs on ARMOURComponent{
var Socket_API_Definitions.Result vl_result;
var ConnectionId connectionId :=-1;
map(mtc:IPL4, system:IPL4);
vl_result := IPL4asp_PortType.f_IPL4_connect(IPL4,
finalConnectionParams.remHost,
finalConnectionParams.remPort,
finalConnectionParams.locHost,
finalConnectionParams.locPort,
connectionId, //pl_connectionParams.connId,
{tcp := {}},
);
log(vl_result)
connectionId := vl_result.connId;
}
Please let me know if this changes anything.
BR
Elemer
|
|
|
| Re: IPL4 port TCP connection problem [message #1749875 is a reply to message #1749847] |
Tue, 13 December 2016 08:30 |
| Eclipse User |
|
|
|
Hello Elemer,
Sorry for the late response.
I rerun the tests and I am sending you the new logs. They show the same error.
I made a basic mistake to get the first error "Transport endpoint is not connected", because I put a wrong (inactive) port as remotePort.
The value of connectionId is taken from a variable defined in the component initialised with -1, so it should be ok, but thanks for pointing out.
I also checked if I have omitted something that must be in the port connection (connect or listen). To verify this, I added an Option reuseAddress in the optionList:
function f_cf01Up() runs on ARMOURComponent{
var Socket_API_Definitions.Result vl_result;
map(mtc:IPL4, system:IPL4);
vl_result := IPL4asp_PortType.f_IPL4_connect(IPL4,
finalConnectionParams.remHost,
finalConnectionParams.remPort,
finalConnectionParams.locHost,
finalConnectionParams.locPort,
connectionId,
{tcp := {}},
{{reuseAddress := {true}}}
);
log(vl_result)
connectionId := vl_result.connId;
}
but the result is the same (as in the logs).
Best regards,
Naum.
Attachment: log2.zip
(Size: 7.10KB, Downloaded 196 times)
|
|
| | | |
| Re: IPL4 port TCP connection problem [message #1749941 is a reply to message #1749901] |
Wed, 14 December 2016 03:14 |
| Eclipse User |
|
|
|
Hi Naum,
I don't know how well the compressed project you have sent reflects the structure of your project;
however, my recommendation is that you keep different products (test ports, protocol modules etc.) in their separate directories ,
for instance
~/src/Abstract_Socket
~/src/IPL4_testport
~/src/TCCUsefulFunctions
etc.
and these directories should contain all source files (TTCN-3, C++ etc.) that belong to the product.
The main reason for this , besides the structure in itself, is that when you change the version of a product , say you want to install a new IPL4 version,
it's straightforward and simple, and there are less chances of missing a file.
The TTCN-3 files and C++ files of a test port belong together, usually when one changes, this might trigger a change in the other so files belonging to different revisions should not be mixed.
And one more thing: in you Makefile the generated sources and headers reappear as user sources and headers , e.g.::
GENERATED_SOURCES = ARMOUR_Adapter_HTTP.cc General_Types.cc HTTP_MessageLen.cc HTTP_Types.cc IPL4asp_Functions.cc IPL4asp_PortType.cc IPL4asp_Types.cc Socket_API_Definitions.cc TCCAssertion_Functions.cc TCCDateTime_Functions.cc TCCEncoding_Functions.cc TCCEnv_Functions.cc TCCFileIO_Functions.cc TCCFileSystem_Functions.cc TCCIPsec_Definitions.cc TCCIPsec_Functions.cc TCCInterface_Functions.cc TCCMaths_Functions.cc TCCMaths_GenericTypes.cc TCCMessageHandling_Functions.cc TCCPermutatedParameters_Definitions.cc TCCPermutatedParameters_Functions.cc TCCSystem_Functions.cc TCCTemplate_Functions.cc TCCTitanMetadata_Functions.cc TestPurposeSuiteHTTP.cc
:
:
USER_SOURCES = ../ARMOUR_Adapter_HTTP.cc ../Abstract_Socket.cc ../General_Types.cc ../HTTP_EncDec.cc ../HTTP_MessageLen.cc ../HTTP_MessageLen_Function.cc ../HTTP_Types.cc ../HTTP_parse_.tab.c ../IPL4asp_Functions.cc ../IPL4asp_PT.cc ../IPL4asp_PortType.cc ../IPL4asp_Types.cc ../IPL4asp_discovery.cc ../Socket_API_Definitions.cc ../TCCAssertion.cc ../TCCAssertion_Functions.cc ../TCCDateTime.cc ../TCCDateTime_Functions.cc ../TCCEncoding.cc ../TCCEncoding_Functions.cc ../TCCEnv.cc ../TCCEnv_Functions.cc ../TCCFileIO.cc ../TCCFileIO_Functions.cc ../TCCFileSystem.cc ../TCCFileSystem_Functions.cc ../TCCIPsec.cc ../TCCIPsec_Definitions.cc ../TCCIPsec_Functions.cc ../TCCInterface.cc ../TCCInterface_Functions.cc ../TCCMaths.cc ../TCCMaths_Functions.cc ../TCCMaths_GenericTypes.cc ../TCCMessageHandling.cc ../TCCMessageHandling_Functions.cc ../TCCPermutatedParameters_Definitions.cc ../TCCPermutatedParameters_Functions.cc ../TCCSystem.cc ../TCCSystem_Functions.cc ../TCCTemplate_Functions.cc ../TCCTitanMetadata.cc ../TCCTitanMetadata_Functions.cc ../TestPurposeSuiteHTTP.cc ../lex.HTTP_parse_.c
This at a first glance is not a dramatic issue, the only possible adverse effect I can see is that it probably slows down compilation (as object files are generated twice) but it suggests that there's something wrong with the way you are generating Makefiles;
besides there might be hidden pitfalls I don't see right now.
User sources and headers should look like this:
USER_SOURCES = Abstract_Socket.cc HTTP_EncDec.cc HTTP_MessageLen_Function.cc HTTP_parse_.tab.c IPL4asp_PT.cc IPL4asp_discovery.cc TCCAssertion.cc TCCDateTime.cc TCCEncoding.cc TCCEnv.cc TCCFileIO.cc TCCFileSystem.cc TCCIPsec.cc TCCInterface.cc TCCMaths.cc TCCMessageHandling.cc TCCSystem.cc TCCTitanMetadata.cc lex.HTTP_parse_.c
USER_HEADERS = Abstract_Socket.hh HTTP_parse_.tab.h IPL4asp_PT.hh HTTP_parse.h IPL4asp_protocol_L234.hh TCCInterface_ip.h
please try to re-generate the Makefile with : makefilegen ...ttcn3_modules...user_sources....user_headers...
and check what it generates.
Best regards
Elemer
|
|
|
| Re: IPL4 port TCP connection problem [message #1749972 is a reply to message #1749941] |
Wed, 14 December 2016 09:55 |
| Eclipse User |
|
|
|
Hello Elemer,
It seems that I am doing something wrong, because I am going from one error to another.
I separated the files in different folders, as you suggested. In fact, I re-downloaded all modules and added them as folders. I made one little change in the TCCUsefulFunctions module, deleted the TCCRegexp_Functions.ttcn, TCCXPathSupport_Functions.ttcn and their .cc files, because I had to resolve lots of dependencies with tinyxml and tinyxpath. I could compile the code normally with this change (no errors in compilation). The archive is in the attachment.
Now I have another type of error on execution, "The executable is not actually executable. Please set an executable generated for parallel mode execution as the executable". If I run it on the command line, I get a segmentation error.
Also, I uploaded the generated makefile as you suggested so you can compare. Just to note that with this new makefile, one file seems to not be reachable, so it doesn't compile.
Best regards,
Naum
|
|
| | | | | | |
| Re: IPL4 port TCP connection problem [message #1750168 is a reply to message #1750117] |
Fri, 16 December 2016 11:22 |
| Eclipse User |
|
|
|
Hello Elemer,
Thank you for helping me debug those codecs and ports! You are right, the problem was in the codecs, MQTT and HTTP, respectively.
For MQTT, (initially) there was a compiling bug (two pairs of parentheses more than needed) and after that, one field that is not optional in the codec was treated as it is optional (to check if there is a value). I have corrected that and will make a commit soon.
In the HTTP codec the problem was that I didn't understand that I have to declare all headers with, at least, omit value.
I will make more tests with different servers next week, and I will try to use (D)TLS and report the results.
Regarding the makefile, it works when the include parameters are added, but isn't this supposed to be generated automatically with the tool?
Best regards,
Naum.
[Updated on: Fri, 16 December 2016 11:25] by Moderator |
|
| | | |
| Re: IPL4 port TCP connection problem [message #1751884 is a reply to message #1750763] |
Mon, 16 January 2017 08:33 |
| Eclipse User |
|
|
|
Hello Elemer,
Instead of opening a new thread, I will ask here, because it is related to the MQTT codec.
In the MQTT specs it says: "A single WebSocket data frame can contain multiple or partial MQTT Control Packets. The receiver MUST NOT assume that MQTT Control Packets are aligned on WebSocket frame boundaries [MQTT-6.0.0-2]." This is a situation that the MQTT Titan codec doesn't support yet.
The solution should be simple, though. In the .cc file of the codec (http://git.eclipse.org/c/titan/titan.ProtocolModules.MQTT.git/tree/src/Mqtt_v3.1.1_EncDec.cc), line 344 (first part of the condition, str.lengthof() > position), its where should be checked if there are multiple messages queued in the same TCP packet (first check the size of the packet, then search for the next mqtt request and its size). The only question that I have, is what is the best solution to organise the data here? Using sets, arrays or something else? In the attachments you can find a Wireshark capture file with sample of this type of message.
Thank you in advance.
Best regards,
Naum.
|
|
|
| Re: IPL4 port TCP connection problem [message #1751892 is a reply to message #1751884] |
Mon, 16 January 2017 09:57 |
| Eclipse User |
|
|
|
Hi Naum,
please look into the below post:
JSON in Titan part IV, usage of IPL4 test port and extracting messages from a TCP stream
https://www.eclipse.org/forums/index.php/t/1070600/
Quote:
"The TCP protocol is stream oriented, meaning that the API gives us the ability to send or receive a byte stream. There is no preservation of message boundaries. TCP can bundle up data from many send() calls into one segment, or it could break down data from one send() call into many segments - but that's transparent to applications sitting on top of TCP,
and recv() just gives you back data, with no relation to how many send() calls produced the data you get back. To be able to extract messages in general , and JSON messages in particular, a TTCN-3 or external C++ function has to be used to find the message boundary in streams; This is implemented as a function reference registration in the test port. The function is declared for the test port, but the user has to implement it depending on what session protocol is used.
In our case , the getMsg_Func plays this role; it refers the function f_GetMsgLengthJSON declared in JSON_common.ttcn , which calls
the external function f_calc_JSON_length implemented in JSON_slicer.cc."
If you register such a dissection function, you don't have to care about multiple messages; the test port will separate the messages using the dissector and send them one by one to the upper layer. This is the recommended usage.
For exact details , please see the code attached to the above post. Also , the IPL4 port documentation contains this info, search for "dissection" .
I hope this helps
Best regards
Elemer
[Updated on: Mon, 16 January 2017 09:59] by Moderator |
|
| | |
| Re: IPL4 port TCP connection problem [message #1751972 is a reply to message #1751954] |
Tue, 17 January 2017 07:28 |
| Eclipse User |
|
|
|
Hello Elemer,
Unfortunately, I have a problem.
I have tested this with mosquitto on local with mosquitto_sub to generate messages (as in the wireshark capture in the previous post). This works flawlessly, Titan decodes all messages.
When I tested it with FIT-IoT LAB and their equipment, there are lots of MQTT messages. Until one instant, TITAN works as it should. After that, on my local machine, it produces a repeated message multiple times and after that it doesn't decode the MQTT messages. On a server with 4GB RAM, it crashes. You can see that it decodes the previous messages even if they are sent in the same TCP packet (captured in Wireshark). Do you have any idea why it is doing this?
I have included the wireshark capture and the two logs (from each machine). The request that blocks is the one before the last message in wireshark.
Thanks in advance.
Best regards,
Naum.
|
|
| | | | | | |
| Re: IPL4 port TCP connection problem [message #1754645 is a reply to message #1752158] |
Tue, 21 February 2017 04:51 |
| Eclipse User |
|
|
|
Hello Elemer,
I don't know if I should ask it here or in another thread, but however: I am using the IPL4 port and the HTTP codec to send message to a server, but there is a strange thing: the server doesn't receive the body of the message. I tried to send the message to a local server on the same machine, I tried to www.requestb.in, but it doesn't get the body. When I try the same request from Postman, everything is OK. I even tried to copy the exact same request from Postman, to ignore the HTTP codec and to insert it directly as a TCP message, but the result is the same, the body is empty.
Can you help me resolve this problem?
Thanks in advance.
Best regards,
Naum
|
|
| | | | | |
Goto Forum:
Current Time: Wed Jul 23 12:39:01 EDT 2025
Powered by FUDForum. Page generated in 0.23187 seconds |
Home
