Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » RIoT - the IoT load/performance generator based on TTCN-3 and Titan
RIoT - the IoT load/performance generator based on TTCN-3 and Titan [message #1790220] Thu, 07 June 2018 08:44 Go to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 853
Registered: January 2015
Senior Member
Dear all,

Before getting into the thick of it, there are a number of theoretical questions we should look into.


TTCN-3 is a language that is primarily targeting testers and testing; requirements against test code are essentially different from requirements against application code:
first and foremost , test code is distributed in source and it should be as easily readable and straightforward as possible.
Test code should never be a haiku , it should be a nursery rhyme, when it comes to simplicity and intelligibility.
Testers may or may not have in-depth programming expertise; their primary focus is the domain knowledge of the SUT.

Applications are typically distributed in binary, hence any obfuscation, abbreviation , structure detrimental to readability is permitted to a point.
Also, abort conditions differ: test code should terminate immediately as it encounters a fault; it makes no sense to continue as the test case failed already;
in comparison, an application should preferably never abort , it should try to survive and repair itself , if possible without the knowledge and involvement
of the user( a good example in this sense are browsers that try to survive any crappy code one feeds them). And so on.

From this it becomes apparent that Titan may not be the optimal tool for writing applications, and also that any comparison between
mainstream languages used fro writing applications and TTCN-3 might be misleading unless the different approaches above are factored in.


But a performance test generator is closer to a classical application than to a test suite:
it's likely distributed in binary and it's important that it survives errors.

So why then write one in TTCN-3? The answer is that some advantages of the language ( capability of message synthesis/analysis/sending/receiving) combined with some Titan features ( capability
to execute quickly, distributed execution) surpass other linguistic disadvantages ( lack of OOP for example - something that is already in the pipeline of standardization by the way).

Right. So let's try to clarify first the concept of a load/test generator.

A classical example is a phone call generator which simulates a high number of users and calls (signalling and payload) generated by them on the network interface.
Such a generator is able to answer questions such as: what are the call handling limits of the network? What happens if we surpass this limit? How will the network behave then?
Will it simply reject the calls or become unstable? and so on.
These are essential questions to be answered by the equipment manufacturer or maintainer of the network.

Please mind that a load generator may not only simulate the user population but the whole or part of a network as well;
In this case traffic of network A (simulated) can be directed over network B ( the system under test ) to chart its capabilities.


So what are the questions that can be answered by a load/traffic generator in the IoT world. Here the traffic generated by devices is very small, and typically does not present a
challenge to the network (well there might be exceptions here; as long as we consider ITS-intelligent transport system- as an IoT concept, traffic here might be significant; but this is
more an exception than the rule). In the IoT world the challenge is more in the large number of devices that are to be served by the network.


The load generated presented below was designed along the above principles, and:

-is written mainly in TTCN-3
-can simulate the presence and stateful application level protocol traffic of millions of IoT devices
based on a scalable architecture
-the traffic profile can change over time, both in intensity and traffic structure
-each simulated device is instantiated and monitored separately, with appropriate states
-besides generating load, it verifies the details of the messaging as well with a granularity particular to function tests only

-the traffic generated can be used to chart the capabilities of the network
to be used to control the IoT devices:
-handling a multitude of such devices
-capability to cope with the traffic profile generated by such a multitude

-can generate statistics, counters logs. etc.
-modularly extensible with any IoT protocol
-currently supporting LwM2M, CoAP , MQTT
-as platform, generic Linux and off-the-shelf hardware can be used
-if needed, any payload type can be added to signaling traffic
-can be extended in the direction of protocol fuzzing to test resilience of IoT applications

For the code, see

http://git.eclipse.org/gitroot/titan/titan.Applications.RIoT


Architecturally, the load generator consists of three layers:

index.php/fa/33029/0/


1. Core Load Library (CLL)

http://git.eclipse.org/gitroot/titan/titan.Libraries.CLL



This library realizes a common base foundation for the whole framework and
provides project, SUT and protocol independent functionality

2. The various Application Libraries (AppLib)

currently:

http://git.eclipse.org/gitroot/titan/titan.ApplicationLibraries.CoAP
http://git.eclipse.org/gitroot/titan/titan.ApplicationLibraries.LWM2M
http://git.eclipse.org/gitroot/titan/titan.ApplicationLibraries.MQTT


They are usually protocol, or application-area dependent, but can be reused
across many applications

3. The Application level code (often called as Control Logic) that "glues"
together the various framework components:

http://git.eclipse.org/gitroot/titan/titan.Applications.IoT_Loadtest_Framework


3.1 Configuration logic (what can and must be configured and what is set
implicitly, what is configured statically and what can be set interactively,
etc.)
3.2 Statistics generation and collection logic (what data is generated, how
the data is reported and which data is recorded in logs and which is
displayed during execution, etc.)
3.3 Deployment logic (which software component is deployed to which
PTC, whether distributed execution of a given Entity Group is
supported, or not, etc.



Also part of the package is a WebGui that connects to te application over a REST interface:

http://git.eclipse.org/gitroot/titan/titan.Libraries.Web_GUI


This architecture , and elements of it , are being used in other high-performance load generators for different telecom areas;
they are proven and tested to and industrial strength level.

see also :

https://ucaat.etsi.org/images/Presentations/UCAAT2017_Csondes_Test-platform-IoT-testing.pdf

To install and quick-start RIoT , please check the attached guide.
If you experiment with it, we'd be delighted to hear back from you.



Finally, for the delight of us, non-English speakers, a fun list of English collective nouns:

a murder of crows
an exaltation of larks
an unkindness of ravens 
a clattering of choughs
a murmuration of starlings
a charm of goldfinches


a feast of brewers 
a goring of butchers
an observance of hermits
a school of clerks
a doctrine of doctors
a tabernacle of bakers
a prudence of vicars
a state of princes
a congregation of people
a diligence of messengers
a discretion of priests
an execution of officers
an eloquence of lawyers
a drunkenship of cobblers
a proud showing of tailors
a skulk of thieves

a tuxedo of penguins 
a bask of crocodiles 
a destruction of cats 
a troop of baboons 
 a barrel of monkeys 
a band of gorillas
a tower of giraffes 
a parliament of owls 
a cackle of hyenas
a smack of jellyfish
an ambush of tigers
a wisdom of wombats




Best regards
Elemer


[Updated on: Thu, 07 June 2018 10:14]

Report message to a moderator

Re: RIoT - the IoT load/performance generator based on TTCN-3 and Titan [message #1790618 is a reply to message #1790220] Wed, 13 June 2018 15:56 Go to previous messageGo to next message
Harald Welte is currently offline Harald WelteFriend
Messages: 50
Registered: July 2017
Location: Berlin, Germany
Member

Elemer Lelik wrote on Thu, 07 June 2018 10:44
Dear all,

For the code, see

http://git.eclipse.org/gitroot/titan/titan.Applications.RIoT



Unfortunately that link returns a 404. I guess you wanted to refer to http://git.eclipse.org/c/titan/titan.Applications.RIoT.git/ ?
Re: RIoT - the IoT load/performance generator based on TTCN-3 and Titan [message #1790620 is a reply to message #1790618] Wed, 13 June 2018 16:00 Go to previous messageGo to next message
Harald Welte is currently offline Harald WelteFriend
Messages: 50
Registered: July 2017
Location: Berlin, Germany
Member

btw: the riot_quickstart_guide.pdf mentions "git submodule update -init", which is missing a "-" as it should be "--init". Unfortuately your manuals are not provided in editable source code format but only as docx files, so I couldn't simply send a diff / patch in gerrit, sorry.
Re: RIoT - the IoT load/performance generator based on TTCN-3 and Titan [message #1790633 is a reply to message #1790620] Wed, 13 June 2018 17:33 Go to previous message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 853
Registered: January 2015
Senior Member
Hi Harald,

thanks for the feedback;

I should have clarified that
http://git.eclipse.org/gitroot/titan/titan.Applications.RIoT

can be used to 'git clone';

http://git.eclipse.org/c/titan/titan.Applications.RIoT.git
works as an URL, but cannot be used in git operations.

We are aware that most of our docs are in legacy formats, we are working on it , but due to the sheer volume of work, and of course priorities , it will take a while.

Best regards
Elemer
Previous Topic:Titan toolchain and Jira
Next Topic:TITAN manuals are hard to find / not publicly searchable/indexed
Goto Forum:
  


Current Time: Wed Nov 14 22:22:23 GMT 2018

Powered by FUDForum. Page generated in 0.02036 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top