Generating invalid UTF-8 [message #1797533] |
Thu, 01 November 2018 10:23 |
|
Hello,
while implementing conformance test for MQTT, I discovered the need of generating invalid UTF-8 characters/strings.
This need originates from the MQTT specification. There, the conformance statements [MQTT-1.5.3-1] and [MQTT-1.5.3-2] state that invalid UTF-8 characters must be handled by the SUT by closing the network connection.
For this purpose, we have already a few defined and implemented test cases (e.g. f_TC_MQTT_BROKER_CONNECT_020 in MQTT_Testcase_Functions.ttcn). However, these TCs started to fail with a Dynamic test case error after updating Titan.
After an investigation on this issue, I figured out that the error is caused by the @update feature. The attached UTF8.ttcn file illustrates the error with a minimal example.
However, with regard to Titan's release notes and TTCN-3 Core Language specification the @update feature seems to be unofficial anyway.
Unfortunately, I don't see any "standard compliant" solution to this issue. Anyone out there with a proposal or handy tips?
Best regards,
Alexander
-
Attachment: UTF8.ttcn
(Size: 2.44KB, Downloaded 66 times)
|
|
|
Re: Generating invalid UTF-8 [message #1797546 is a reply to message #1797533] |
Thu, 01 November 2018 16:00 |
|
Hello again,
after diving into the TTCN-3 Core Language specification I stumbled across oct2unichar which converts octetstrings to universal charstrings in UTF-8 (I'm still astonished why I didn't use this one from the early beginning).
I replaced the @update instructions with oct2unichar and Titan seems now to digest the encoding of the strings without any complaint. The following snippet from the log demonstrates a user_name string with an appended oct2unichar('EDA080'O) (which sould represent U+D800)
user_name := { stringLength := 36, stringItem := "4bb415c8-2c5b-48af-9a0e-46c34f7672e4" & char(0, 0, 216, 0) }
However, I'm still struggling with the verification whether the produced string is really invalid according to the MQTT specification. At least Mosquitto and most of the other Brokers I checked so far seem to accept those strings.
BR,
Alexander
|
|
|
|
Re: Generating invalid UTF-8 [message #1797698 is a reply to message #1797695] |
Mon, 05 November 2018 11:37 |
|
Hi Alex,
nothwithstanding this concrete example, let me tell you that the idea of @erronous, @update has been accepted by ETSI and it will be part of the standard
probably from the next release; the syntax might be slightly changed though.
So whatever you do it will be future proof to an extent.
Best regards
Elemer
|
|
|
Re: Generating invalid UTF-8 [message #1797718 is a reply to message #1797533] |
Mon, 05 November 2018 15:31 |
|
Hi Botond and Elemer,
thank you for your responses.
@Botond
I attached the 'MQTT_v3_1_1_Types' file I used. However, this should be identical to the latest one from Titan's Git repository.
I'm using the following version of ttcn3_compiler
ttcn3_compiler -v
TTCN-3 and ASN.1 Compiler for the TTCN-3 Test Executor
Product number: CRL 113 200/6 R4A
Build date: Jun 12 2018 08:36:32
Compiled with: GCC 7.3.0
Using OpenSSL 1.1.0g 2 Nov 2017
Copyright (c) 2000-2018 Ericsson Telecom AB
@Elemer
These are great news :)
While I was able to send some invalid (according to the MQTT specification) UTF-8 characters like U+D800 and U+DFFF, completely invalid UTF-8 characters (e.g. 0xC0AE as stated in RFC3629 Section 10) are not possible with oct2unichar. For now our Test Cases (TC_MQTT_BROKER_CONNECT_[020..025]) should work again as intentended.
But once we figured out the issue with @update I will try to define some additional test purposes with some "evil UTF-8 characters" specifically for security considerations.
Best regards,
Alexander
|
|
|
|
Re: Generating invalid UTF-8 [message #1797842 is a reply to message #1797808] |
Wed, 07 November 2018 10:42 |
|
Hi Botond,
thank you for your investigation and explanation.
Disabling the error solved the issue. As I was setting the stringLength also via the '@update' feature as described in your example, ignoring this error should be an appropriate solution.
Now the encoded octetstrings look good, thanks again.
Best regards,
Alex
|
|
|
Powered by
FUDForum. Page generated in 0.03447 seconds