Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: (Test of JSON -based REST web services)
Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1702445] Wed, 22 July 2015 11:49 Go to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Generalities
The ability to move back and forth between JSON and TTCN-3 constructs is a new, and yet unstandardised feature in Titan. We are working on the appropriate standard but due to the nature of standardization it will probably take some time, measurable in months, until the standard will hit the public libraries.
JSON is becoming more and more popular as a data interchange format, especially in the realm of the web, due to its' low overhead, simplicity and capabilities. Similar to XML, it even has a schema (http://json-schema.org/ ), which does not seems to be very popular among users, as it is considered unnecessary, main reason for that being the dynamically typed nature of languages in which JSON has its' roots: a JSON instance can be converted into a dynamically created class without any formal description being needed.
However, as TTCN-3 is a statically typed and strict language used in verification, the concept of schema is important, as schemas constitute the reference in decisions to be made by the code. As in XML, a JSON schema is the counterpart of a TTCN-3 type, while a JSON instance or document is the equivalent of a TTCN-3 variable/constant/template. TTCN-3 instances can be encoded into JSON instances and vice-versa.
TTCN-3 types can be used to generate the equivalent JSON schemes using the built-in Titan converter (--ttcn2json option). Conversion from JSON schemas to TTCN-3 types is not (yet) available in Titan, although it's quite straightforward; using the relevant chapter in the reference guide, this can be done manually.
Let's examine a number of use cases and associated work flows.

Test of JSON -based REST web services

The general layout of testing a JSON-based web service is the following:
we create some TTCN-3 templates which need to be encoded in JSON and sent to the web server to the endpoint and with the method indicated in the REST API. The reply, in JSON format has to be decoded and confronted to a reception template in TTCN-3.
To be able to encode/decode, we need to create TTCN-3 types equivalent to the JSON schema which could generate the JSON instances involved. Most of the time, there is no such JSON schema; TTCN-3 types can be written manually from JSON samples just by looking and them and converting them mentally. It might help to generate a schema from the JSON instance; if so, the JSON schema generator at http://jsonschema.net/ could be used.
Let's take the example of the geoplugin web service , one that cannot be simplified any further:
a GET request sent to http://www.geoplugin.net/json.gp?ip=89.147.71.240 will return some geo etc. data in JSON format:
curl -v http://www.geoplugin.net/json.gp?ip=89.147.71.240
* About to connect() to proxy www-proxy.ericsson.se port 8080 (#0)
* Trying 153.88.253.150... connected
* Connected to www-proxy.ericsson.se (153.88.253.150) port 8080 (#0)
> GET http://www.geoplugin.net/json.gp?ip=89.147.71.240 HTTP/1.1
> User-Agent: curl/7.19.0 (x86_64-suse-linux-gnu) libcurl/7.19.0 OpenSSL/0.9.8h zlib/1.2.3 libidn/1.10
> Host: www.geoplugin.net
> Pragma: no-cache
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Server: geoPlugin/3.2.6
< Date: Tue, 14 Jul 2015 10:28:53 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 753
< Proxy-Connection: Keep-Alive
< Connection: Keep-Alive
<
{
"geoplugin_request":"89.147.71.240",
"geoplugin_status":200,
"geoplugin_credit":"Some of the returned data includes GeoLite data created by MaxMind, available from <a href=\\'http:\/\/www.maxmind.com\\'>http:\/\/www.maxmind.com<\/a>.",
"geoplugin_city":"Part",
"geoplugin_region":"Zala",
"geoplugin_areaCode":"0",
"geoplugin_dmaCode":"0",
"geoplugin_countryCode":"HU",
"geoplugin_countryName":"Hungary",
"geoplugin_continentCode":"EU",
"geoplugin_latitude":"46.549999",
"geoplugin_longitude":"16.9167",
"geoplugin_regionCode":"24",
"geoplugin_regionName":"Zala",
"geoplugin_currencyCode":"HUF",
"geoplugin_currencySymbol":"Ft",
"geoplugin_currencySymbol_UTF8":"Ft",
"geoplugin_currencyConverter":282.3375
* Connection #0 to host www-proxy.ericsson.se left intact
* Closing connection #0
}

Let's try to create the equivalent TTCN-3 type; obviously it's a record where the fields are mainly strings, , there's one integer (geoplugin_status) and one float (number in JSON-geoplugin_currencyConverter).
Let's name this type Geoplugin_response:
type record Geoplugin_response
{
charstring geoplugin_request,
integer geoplugin_status ,
charstring geoplugin_credit ,
charstring geoplugin_city,
charstring geoplugin_region ,
charstring geoplugin_areaCode ,
charstring geoplugin_dmaCode ,
charstring geoplugin_countryCode ,
charstring geoplugin_countryName ,
charstring geoplugin_continentCode ,
charstring geoplugin_latitude ,
charstring geoplugin_longitude ,
charstring geoplugin_regionCode ,
charstring geoplugin_regionName ,
charstring geoplugin_currencyCode ,
charstring geoplugin_currencySymbol ,
charstring geoplugin_currencySymbol_UTF8 ,
float geoplugin_currencyConverter
}

Now we can declare the encoding-decoding functions for this type; Titan will do the rest automatically.
What if we have several different JSON structures? It would be awkward to be in need of separate encoding-decoding structures for each; so let's collect them all into a union which we can name appropriately JSON_PDU (PDU for Protocol Data Unit , as these JSON messages are in fact protocol messages in the client -server communication):
In our case the union will have a single alternative:
type union JSON_PDU
{
Geoplugin_response geoplugin_response
}

There's a problem though: if we encode a geoplugin_response type template , the encoded JSON will reflect the nested union/record structure:
{"geoplugin_response" : 
{ 
"geoplugin_request" : "89.147.71.240",
"geoplugin_status" : 200, 
"geoplugin_credit" : "AAAAAAAAAAAAAAA",
"geoplugin_city" : "Part", 
"geoplugin_region" : "Zala", 
"geoplugin_areaCode" : "0", 
"geoplugin_dmaCode" : "0", 
"geoplugin_countryCode" : "HU", 
"geoplugin_countryName" : "Hungary", 
"geoplugin_continentCode" : "EU", 
"geoplugin_latitude" : "46.549999", 
"geoplugin_longitude" : "16.9167", 
"geoplugin_regionCode" : "24", 
"geoplugin_regionName" : "Zala", 
"geoplugin_currencyCode" : "HUF", 
"geoplugin_currencySymbol" : "Ft", 
"geoplugin_currencySymbol_UTF8" : "Ft", 
"geoplugin_currencyConverter" : 300.000000 }
}

To circumvent this, we have to use the
with {
variant "JSON : as value"
}

encoding instruction to tell the codec that special treatment is needed: at encoding,
the nesting union has to be ignored, while at decoding, it has to be reconstituted.
To trigger the generation of JSON codecs by Titan, we need to decorate the geoplugin.ttcn module
with the
with {
encode "JSON";
}

instruction.
The rest of the code does not require much explanation: an HTTP GET with the appropriate headers is sent to the server using the HTTP test port; the received JSON is extracted from the body and decoded/matched etc.

How to use the attached code:

Unpack to a SimpleREST directory
cd SimpleREST
cd bin
../src/install.script
make


The Makefile was generated with:
makefilegen -s -e REST *.ttcn *.cc *.hh


./REST REST.cfg



Best regards

Elemer
Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1814169 is a reply to message #1702445] Thu, 05 September 2019 08:37 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

In my perspective, geo do have json encoding and decoding function,
but it is specified for the response of http://www.geoplugin.net/json.gp,
and this response is a specified JSON format, is that correct?
I can see a decoded message of response in log, that's why I think so.

Another question is, Ts29518NAMF_MT.ttcn provided on titan.core page looks so similar to geoplugin,
does that means the mechanism and usage are the same?

Best regards

Sean
Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1814181 is a reply to message #1814169] Thu, 05 September 2019 10:40 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Sean,

Once the TTCN-3 structures describing the JSON messages are in place, and decorated with encoding variants,
Titan will automatically generate the encoding functions.

For message sending the flow is: TTCN-3 template/variable --->encoded into JSON message
For receiving: JSON message ---->decoded into TTCN-3 variable <----> compared with template

The mechanism and usage of 5G REST APIs is similar, except they use HTTP/2 as transport.


I hope this helps

BR

Elemer
Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1814407 is a reply to message #1814181] Wed, 11 September 2019 05:48 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

I have deploy a simplified namf JSON protocol module,
and successfully sent it through data frame,
however, this dataframe doesn't recognized as json in wireshark, does that normal...?
index.php/fa/36266/0/

Best regards

Sean
Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1814413 is a reply to message #1814407] Wed, 11 September 2019 07:17 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Sean,

However I look at it, I see the JSON, see bottom right.
:)

BR
Elemer

[Updated on: Wed, 11 September 2019 07:20]

Report message to a moderator

Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1814503 is a reply to message #1814413] Thu, 12 September 2019 11:57 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

I know why wireshark recognized that dataframe frome server is json!
It's because the content-type in the continuation frame!

Best regards

Sean
Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1814511 is a reply to message #1814503] Thu, 12 September 2019 13:02 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Sean,

Hope you got some sleep :)
Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1814548 is a reply to message #1814511] Fri, 13 September 2019 04:32 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Ok, I realized that you were missing wireshark to identify
Explicitly the payload of the data frame as JSON.

Right, content-type is the indicator here. Good work.

Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1814660 is a reply to message #1814548] Mon, 16 September 2019 07:48 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

Thanks for your reply & concern, I got some sleep by finished my ttcn progression with your generous help.
Current state of my work is, ttcn test is temporarily put aside due to some new tasks out of ttcn.
Most of my question is currently answered in [message #1814461], but still great thanks to you for confirming my conclusion about the type stuff in wireshark.
Probably won't be here for a couple days, or weeks ( hope that I can get back here soon, since I finally get familiar with titan and ttcn).

P.S. Sorry for replying sooo late, Fri 13th ( oof but I'm a christian so it is fine) Sep was Moon Fest. in our country that most labor could get a paid day off.
So, the Fri and weekends forms a little vacation.
And I have to express my gratitude again, I had a very sweet vacation without any overtime working due to so many of your helps and answers.

Best regards

Sean

[Updated on: Mon, 16 September 2019 07:55]

Report message to a moderator

Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1821757 is a reply to message #1814660] Wed, 19 February 2020 16:22 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Hi Elemer and all,

First of all thanks for the SimpleREST exaqmple package! I just used it successfully and it is really easy to work over the HTTP-PCOs sending and receiving HTTP messages!

Just one question: If I start Wireshark and try to capture the HTTP stream I don't see the HTTP messages at all. Does the HTTP-PCOs implementation set some strange flags or something,
so that wireshark can not decode? May be some of you had the same problem?

Olaf



Re: Using JSON in TTCN-3 and Titan Part I: Testing REST / JSON web services: [message #1821784 is a reply to message #1821757] Thu, 20 February 2020 06:06 Go to previous message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Olaf,

glad to be of service;

TCP Packets do not carry any intrinsic, surefire information about the nature of the protocol in the payload, so Wireshark has to make
an educated guess about that; for instance if you use destination port 80, Wireshark will assume HTTP, but if other port appears as destination,
Wireshark will have to be nudged in the right direction.
One can do this by right-clicking on a TCP packet that is part of the HTTP message, then opt for Decode As..., and then for HTTP (or any other relevant payload protocol for that matter).


Best regards
Elemer

Previous Topic:Trouble getting SCTP to work with IPL4
Next Topic:Trouble getting SCTP to work with IPL4
Goto Forum:
  


Current Time: Thu Jul 09 21:11:42 GMT 2020

Powered by FUDForum. Page generated in 0.04181 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top