Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » Java code generator/executor of Titan(Java, C/C++, TTCN-3, codegenerator , workflow)
Java code generator/executor of Titan [message #1816935] Tue, 12 November 2019 12:54 Go to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Dear all,


we have been discussing before on this forum the introduction of a new workflow,
based on Java code generation from TTCN-3, as opposite to C/C++ code generation Titan builds upon today.

Of course there is no intention to abandon the legacy line, we see these two as being complementary to each other:

-the legacy line is fully featured, and has an ecosystem of thoroughly tested subproducts (test ports, protocol modules etc.);
it also can be easily used in minimalistic embedded environments, automated regression test suites etc.
-the Java code generator/executor Eclipse plug-in yet lacks the coverage and support of the legacy line;
most of the standard is covered to a degree similar to the classical Titan; RAW encoding also works and JSON is in the making, however test ports etc. will have to be adapted.
(BTW, the intention is not to go back and migrate every subproduct to Java-friendliness; we'd prefer to see the new workflow adopted in new technologies, such as IoT, where the burden of legacy is less there.)

We intend to publish small demo projects to exemplify a migration to the Java workflow.


This means that several workflows will be possible with Titan:

-classical development/execution based on the command line executor, and possibly an independent text editor
-command line executor plus Eclipse based IDE, with the full range of editing/assisting features added by the latter
-Java code generator and executor, with the same full featured IDE

Note that that last workflow adds full portability to Windows as well.

In the video linked below

https://www.youtube.com/watch?v=lZpq2qKAWW0

Kristof Szabados, architect/lead designer of Titan presents a few impressions after the initial exposure with the Java code generator/executor.

One thing that came as a surprise to us too is that compile times are spectacularly improved in Java. Also, execution times are not so bad either, which makes the Java code generator a worthy proposition not only during TTCN-3 development, but possibly during regression etc. test execution as well.


So please explore this new possibility and let us know of your impression.
The Java code generator has been made available in the previous Titan release, new parts will be added in the upcoming one.



Best regards
Elemer

[Updated on: Wed, 13 November 2019 09:06]

Report message to a moderator

Re: Java code generator/executor of Titan [message #1817083 is a reply to message #1816935] Fri, 15 November 2019 09:50 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Just to understand, Elemer:

This may be interesting for new projects, but not for existing projects, right? All codecs and external functions need to be rewritten in Java?

And do you have a Hello World project with a simple external function just to understand better?

Regards,
Olaf
Re: Java code generator/executor of Titan [message #1817151 is a reply to message #1817083] Sat, 16 November 2019 11:09 Go to previous messageGo to next message
Kristof Szabados is currently offline Kristof SzabadosFriend
Messages: 54
Registered: July 2015
Member
Dear Olaf,

"This may be interesting for new projects, but not for existing projects, right?"
It might also be interesting for existing projects, as they also might welcome help with the build times.
As I show in the video in the case of full builds we have seen times of ~180 seconds on the C side drop to ~10 seconds on the Java side ... and incremental build times of ~40 seconds drop to ~0.1 second.
The speedup this way of working can offer could benefit both old and new projects.
There are many ways to make this work:
- you could just simply try out the Java side and decide to keep using it.
- you could have both a C side and a Java side project in your eclipse, iterate fast on the Java side version and do the final builds on the C side version.
- you could also have only the Java side on your machine to speed up your work ... and use the C side in your background systems (for example CI) to produce C binaries
etc...

"All codecs and external functions need to be rewritten in Java?"
Yes, all codes and external functions used in your project need to be rewritten in Java.
But please note 2 points.
1)
only the codecs and external functions you are using.
For example we have already converted the RAW codec .. if you are only using that, you are good to go.
(we will eventually get to convert all codecs and testports provided with Titan ... in time)

2)
Conversion of existing external functions is usually quite simple as we tried to keep very close to the original architecture.

Lets say you have an external function that returns a field of a record type.
That might look something like this in TTCN-3:
"
type record temprec {
integer field1,
float field2
}

external function ef(in temprec p_roi) return integer;
"

The C side code to this function would look like this:
"
INTEGER ef(const temprec& p__rec) {
return p__rec.field1();
}
"
(to keep simple I left out the namespace, import and surrounding parts)

On the Java side this would look like this:
"
public static TitanInteger ef(final temprec p__rec) {
return p__rec.get_field_field1();
}
"

As you can see the function has to be written in Java.
But:
- the code on the abstract level can be the same, you just have to pay attention to different (but similar names).
These had to be done, as Java does not support for example operator overloading, which the C side uses extensively

- If you do this in Eclipse, using Quick fixes and code completion can make writing such functions quite fast.
On the C side, you would have to write the .hh and .cc files containing the external function, take care of using the proper namespace, header guards, how the signature will look like in C after translation, etc...
On the Java side in Eclipse:
1) write your external function in TTCN-3 code and invoke build.
2) Java will warn that Java class where the external function should be in does not exist ... bring the cursor over the warning, right click + Quick fix and ask Eclipse to create class.
3) Than it will warn that the function does not exist ... bring he cursor over the warning, right click + quick fix and ask eclipse to create it for you.
And you alreaady have a proper skeleton for your external function that you only need to fill in.
(When I was converting a large internal project to see how the build performance scales I was creating these external function skeletons by the dozens/minute)

Regards
Kristof
Re: Java code generator/executor of Titan [message #1817152 is a reply to message #1817151] Sat, 16 November 2019 11:13 Go to previous messageGo to next message
Kristof Szabados is currently offline Kristof SzabadosFriend
Messages: 54
Registered: July 2015
Member
Hi Olaf,

Just one more thing.
For external functions used for conversion where the platform dependent code is generated automatically on the C side, the Java side also generates the code automatically (if the encoding is supported).

for example:
"
type integer RAW_PDU_1
with { encode "RAW"; variant "FIELDLENGTH(16)"};
external function enc_RAW_PDU_1(in RAW_PDU_1 pdu) return octetstring with { extension "prototype(convert) encode(RAW)" }
"

This example should compile and work automatically.

Regards
Kristof
Re: Java code generator/executor of Titan [message #1817185 is a reply to message #1817152] Mon, 18 November 2019 08:47 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Thanks, Kristof, for your explanations.
I see: you are a Java freak :-)

My question to you: Do you think that the future of Titan will be Java based? Or will it be Java in some niche area, but not in the telecom area where current test systems use excelent open source libraries like OpenSSL?

There is also another point: Old C programmers like myself are afraid of Java :-)
In my case I even don't use Eclispse and I am very happy with Emacs, gcc and gdb :-)
I work on a small old Dell with 4G memory, which is enough for compiling and running the ETSI MCPTT ATS.

Regards,
Olaf

By the way: Is there an easy tutorial for installing Eclipse for Titan? May be I will try (probably I need more than 4G memory)
Re: Java code generator/executor of Titan [message #1817236 is a reply to message #1817185] Mon, 18 November 2019 19:31 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Olaf,

just my two cents:

-Java or C/C++ are just technological labels; users are not interested in labels but performance, be that small footprint of a tool or familiarity with the environment it assumes .
So we are not committed to labels but to offer tools with satisfactory performance.
Realistically one cannot expect limitless performance in every technological context, so that is why
we are trying to offer several technologies to address this from multiple angles.

There's a chance classical Titan will be continued to be used in the legacy telecom area while the Java workflow will find itself new applications possibly
in the service oriented realm, perhaps more related to java.

But let's not forget that the language that powers this is the same TTCN-3.

BR
Elemer
Re: Java code generator/executor of Titan [message #1827983 is a reply to message #1817236] Thu, 28 May 2020 09:17 Go to previous messageGo to next message
Jan S is currently offline Jan SFriend
Messages: 4
Registered: May 2020
Junior Member
Hi Guys,

as a newcomer I just wanted to ask if this is use case could be executed using the available/new Java Code Generator.
Sorry for eventual offtoping:
Let's say I have the ATS already defined in TTCN and I have the Communication Protocol stack written in JAVA.
Can I compile them together to run the ETS?
As far as I understood it, it will be necessary to implement the ports - TCI and TRI, but is there any additional effort to be done which I don't see right now?


Re: Java code generator/executor of Titan [message #1828017 is a reply to message #1827983] Fri, 29 May 2020 08:27 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Jan,



-the first thing you need to consider is the codecs your project assumes.
The Java code generator does not yet cover all codecs the legacy part does: only RAW is implemented, and JSON is on its way.

-next , your test port needs are to be charted:

we started to implement some test ports in Java, basic TCP/UDP behaviour, see https://github.com/eclipse/titan.misc/tree/master/JavaTestPorts
(location might change in the future) ; if you need something else, you might need to convert existing C/C++ test port code to Java.

Other than that , please be aware that the Java code generator/executor is not yet at the same level of functionality as the legacy part; the core language is more or less covered,
but for instance the Java main controller is under development; a new release is due soon, please keep an eye on the postings .

Meanwhile maybe you can get more familiar with the C/C++ compiler/executor and the Eclipse IDE, each of them having a certain learning curve.


Best regards
Elemer
Re: Java code generator/executor of Titan [message #1828671 is a reply to message #1828017] Tue, 16 June 2020 10:31 Go to previous messageGo to next message
Jan S is currently offline Jan SFriend
Messages: 4
Registered: May 2020
Junior Member
Hi Elemer,

i tried to figure out some stuff on my own, but there is a lot of documentation.
Quote:

-the first thing you need to consider is the codecs your project assumes.
The Java code generator does not yet cover all codecs the legacy part does: only RAW is implemented, and JSON is on its way.

Are you refering to protocol modules mentioned here https://projects.eclipse.org/projects/tools.titan protocol modules:

index.php/fa/38338/0/Prot_Mod.png

As far as I understood correctly, those are specified by various templates?
The standard which I using specifies a lot of them. In a common template module i have a line importing some kind of lib:

import from XSDAUX language "TTCN-3:2012" all;

is there are source where I could obtain various common libs or are they part of the titan core git hub intallation?

  • Attachment: Prot_Mod.png
    (Size: 16.19KB, Downloaded 70 times)
Re: Java code generator/executor of Titan [message #1828682 is a reply to message #1828671] Tue, 16 June 2020 14:07 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Jan,

protocol modules are structured representations of the messages expressed in TTCN-3 ( or ASN.1 etc. but let's stick to TTCN-3 for simplicity).

These structures are generic, but yes, specific templates can be generated based on them.
(For instance say there is a structure describing a message HELLO; by attributing certain values to different fields of this structure, a number of templates can be generated based on this structure)

Here

https://github.com/eclipse/titan.core

you can see a number of protocol modules listed ; and yes, these are the same ones as mentioned in https://projects.eclipse.org/projects/tools.titan. Protocol modules are stand -alone products, they have to be downloaded separately and are not part of the Titan installation itself.

Now the structures described by the protocol modules will have to encoded into a bitstream in order to be sent as a payload.
This can be done in a number of ways, depending on the representation of the payload:

-encoding in binary , also called as RAW encoding
-encoding in XML or JSON , wherever these formats are required (ultimately these are binary too, but with an internal structure according to XML or JSON specs)
-encoding in ascii ( or TEXT)
-binary encoding in BER/PER etc. for ASN.1

and so on

Encoding/decoding (that is back and forth conversion between the structured and binary representation ) can be done with purposely-written external functions
or using the capability of Titan to generate these codecs automatically;

to be able to do so , the TTCN-3 code has to be decorated with encoding instructions (which are non-standard) which will inform Titan about
how encoding/decoding should be done; Titan will process this info and generate the codec functions.

The legacy C/C++ Titan has a number of such supported codecs; the JAVA counterpart only supports RAW so far, and JSON is under construction.
We have published posts with detailed descriptions of the codecs, see earlier part of this forum,
and also the Titan reference guide, downloadable separately or packed with every Titan binary.

So you should look into your needs of protocol messages, see which protocol matches of the ones published so far.
If none, than you should write your own protocol module, which may be simpler or more complex task depending on the complexity of the protocol itself.
If there is a match you are in luck.


Now if the required encoding is purely binary, it's even better as you can use the RAW encoder and autogenerated codecs.
If not then for a JAVA workflow you will probably have to write your codecs yourself as they are not yet covered by the JAVA version
(Mind that codecs are not standardized, but implementation dependent, hence the setup).

If you don't stick to a JAVA workflow, than you can use the C/C++ workflow with a broader coverage;
in this case the Eclipse plug-ins will work as an IDE to the C/C++ based executor.


Based on this
 import from XSDAUX language "TTCN-3:2012" all;

you probably have some kind of an XML encoding, which is not yet supported by the JAVA variant and it will not be supported soon either, so your best bet is to go with the legacy version.
If I'm right than probably you should have a number of XSD files (XML schema documents) that are published by the same body that published the protocols.

They can be converted to TTCN-3 + encoding instructions using the xsd2ttcn utility that comes with Titan.

(see for example https://github.com/eclipse/titan.ProtocolModules.XMPP )


I hope this makes it somewhat more clearer.
However please be aware that there is an amount of specific knowledge that has to be acquired


Best regards

Elemer

Re: Java code generator/executor of Titan [message #1829274 is a reply to message #1828682] Tue, 30 June 2020 08:07 Go to previous messageGo to next message
Jan S is currently offline Jan SFriend
Messages: 4
Registered: May 2020
Junior Member
Hi Elemer,

first of all - thank you for your elaborate responses.

I have a communication stack implemented in JAVA.
It simulates the communication between client and server using a UDP/TCP.
I also have the TTCN test cases (from a standard) which based on the PICS test the client and the server side.

As far as I understood it, the Component interface on the TTCN side is one thing, but than I also need to take care on the interface of the SUT side.
Now I'm wondering if there is a JAVA package I could use to define the interface on the protocol side - something like this org.etsi.ttcn.tri.
I've been looking and could only find the classes.

Am I getting this all wrong?
Do I have to implement the JAVA package on my own or is there a repo?



Re: Java code generator/executor of Titan [message #1829279 is a reply to message #1829274] Tue, 30 June 2020 11:48 Go to previous message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Jan,
architecturally Titan is not aligned with the standard;
for reasons and details see:


Architecture and speed
https://www.eclipse.org/forums/index.php/t/1083552/

However this, as it is a matter internal to an implementation, should cause no problems in executing test cases based on the standard.

When testing a server functionality , this being the more common scenario, Titan implementing a client behavior , all you need is the TTCN-3 code of the test suite and an appropriate test port.
Test cases are triggered from the client/TTCN-3 side, as in a real network situation.

Things are a bit different when testing client functionality; Titan will implement server behavior which usually means waiting for a message that initiates communication. In such a case you will need an extraneous component called "upper tester" which is connected both to test suite (via an appropriate test port) and to the SUT representing the client functionality.
For this layout, the messaging is triggered also by the TTCN-3 side, which sends a request to the SUT (something like "send me messageA" ) which is relayed by the upper tester .
The SUT then will send messageA according to its client functionality which is received and analyzed by the simulated TTCN-3 server and so on.

This upper tester is specific to the SUT and will have to be written by the user. Again, this is only needed if you have to test client functionality.

BTW, the fact that your SUT is written in Java does not mean that the test suite has to be also written in Java, the C/C++ based legacy Titan should be fine, as these components communicate with each other over TCP/UDP ports;
the upper tester will have to be written in Java however, and it will have to talk to the Java code of the SUT on one side, and to the test suite , over some test port (that could be UDP or TCP) on the other side.

Best regards
Elemer



[Updated on: Tue, 30 June 2020 11:54]

Report message to a moderator

Previous Topic:Running TTCN 3 Test cases (WebSocket ) on windows machine using Eclipse TITAN plugin
Next Topic:Relaxing the C/C++ compiler version check for Titan 7.1.1
Goto Forum:
  


Current Time: Wed Jul 08 15:30:45 GMT 2020

Powered by FUDForum. Page generated in 0.02533 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top