Home » Eclipse Projects » Eclipse Titan » RIoT - the IoT load/performance generator based on TTCN-3 and Titan
| RIoT - the IoT load/performance generator based on TTCN-3 and Titan [message #1790220] |
Thu, 07 June 2018 08:44  |
|
Dear all,
Before getting into the thick of it, there are a number of theoretical questions we should look into.
TTCN-3 is a language that is primarily targeting testers and testing; requirements against test code are essentially different from requirements against application code:
first and foremost , test code is distributed in source and it should be as easily readable and straightforward as possible.
Test code should never be a haiku , it should be a nursery rhyme, when it comes to simplicity and intelligibility.
Testers may or may not have in-depth programming expertise; their primary focus is the domain knowledge of the SUT.
Applications are typically distributed in binary, hence any obfuscation, abbreviation , structure detrimental to readability is permitted to a point.
Also, abort conditions differ: test code should terminate immediately as it encounters a fault; it makes no sense to continue as the test case failed already;
in comparison, an application should preferably never abort , it should try to survive and repair itself , if possible without the knowledge and involvement
of the user( a good example in this sense are browsers that try to survive any crappy code one feeds them). And so on.
From this it becomes apparent that Titan may not be the optimal tool for writing applications, and also that any comparison between
mainstream languages used fro writing applications and TTCN-3 might be misleading unless the different approaches above are factored in.
But a performance test generator is closer to a classical application than to a test suite:
it's likely distributed in binary and it's important that it survives errors.
So why then write one in TTCN-3? The answer is that some advantages of the language ( capability of message synthesis/analysis/sending/receiving) combined with some Titan features ( capability
to execute quickly, distributed execution) surpass other linguistic disadvantages ( lack of OOP for example - something that is already in the pipeline of standardization by the way).
Right. So let's try to clarify first the concept of a load/test generator.
A classical example is a phone call generator which simulates a high number of users and calls (signalling and payload) generated by them on the network interface.
Such a generator is able to answer questions such as: what are the call handling limits of the network? What happens if we surpass this limit? How will the network behave then?
Will it simply reject the calls or become unstable? and so on.
These are essential questions to be answered by the equipment manufacturer or maintainer of the network.
Please mind that a load generator may not only simulate the user population but the whole or part of a network as well;
In this case traffic of network A (simulated) can be directed over network B ( the system under test ) to chart its capabilities.
So what are the questions that can be answered by a load/traffic generator in the IoT world. Here the traffic generated by devices is very small, and typically does not present a
challenge to the network (well there might be exceptions here; as long as we consider ITS-intelligent transport system- as an IoT concept, traffic here might be significant; but this is
more an exception than the rule). In the IoT world the challenge is more in the large number of devices that are to be served by the network.
The load generated presented below was designed along the above principles, and:
-is written mainly in TTCN-3
-can simulate the presence and stateful application level protocol traffic of millions of IoT devices
based on a scalable architecture
-the traffic profile can change over time, both in intensity and traffic structure
-each simulated device is instantiated and monitored separately, with appropriate states
-besides generating load, it verifies the details of the messaging as well with a granularity particular to function tests only
-the traffic generated can be used to chart the capabilities of the network
to be used to control the IoT devices:
-handling a multitude of such devices
-capability to cope with the traffic profile generated by such a multitude
-can generate statistics, counters logs. etc.
-modularly extensible with any IoT protocol
-currently supporting LwM2M, CoAP , MQTT
-as platform, generic Linux and off-the-shelf hardware can be used
-if needed, any payload type can be added to signaling traffic
-can be extended in the direction of protocol fuzzing to test resilience of IoT applications
For the code, see
http://git.eclipse.org/gitroot/titan/titan.Applications.RIoT
Architecturally, the load generator consists of three layers:
1. Core Load Library (CLL)
http://git.eclipse.org/gitroot/titan/titan.Libraries.CLL
This library realizes a common base foundation for the whole framework and
provides project, SUT and protocol independent functionality
2. The various Application Libraries (AppLib)
currently:
http://git.eclipse.org/gitroot/titan/titan.ApplicationLibraries.CoAP
http://git.eclipse.org/gitroot/titan/titan.ApplicationLibraries.LWM2M
http://git.eclipse.org/gitroot/titan/titan.ApplicationLibraries.MQTT
They are usually protocol, or application-area dependent, but can be reused
across many applications
3. The Application level code (often called as Control Logic) that "glues"
together the various framework components:
http://git.eclipse.org/gitroot/titan/titan.Applications.IoT_Loadtest_Framework
3.1 Configuration logic (what can and must be configured and what is set
implicitly, what is configured statically and what can be set interactively,
etc.)
3.2 Statistics generation and collection logic (what data is generated, how
the data is reported and which data is recorded in logs and which is
displayed during execution, etc.)
3.3 Deployment logic (which software component is deployed to which
PTC, whether distributed execution of a given Entity Group is
supported, or not, etc.
Also part of the package is a WebGui that connects to te application over a REST interface:
http://git.eclipse.org/gitroot/titan/titan.Libraries.Web_GUI
This architecture , and elements of it , are being used in other high-performance load generators for different telecom areas;
they are proven and tested to and industrial strength level.
see also :
https://ucaat.etsi.org/images/Presentations/UCAAT2017_Csondes_Test-platform-IoT-testing.pdf
To install and quick-start RIoT , please check the attached guide.
If you experiment with it, we'd be delighted to hear back from you.
Finally, for the delight of us, non-English speakers, a fun list of English collective nouns:
a murder of crows
an exaltation of larks
an unkindness of ravens
a clattering of choughs
a murmuration of starlings
a charm of goldfinches
a feast of brewers
a goring of butchers
an observance of hermits
a school of clerks
a doctrine of doctors
a tabernacle of bakers
a prudence of vicars
a state of princes
a congregation of people
a diligence of messengers
a discretion of priests
an execution of officers
an eloquence of lawyers
a drunkenship of cobblers
a proud showing of tailors
a skulk of thieves
a tuxedo of penguins
a bask of crocodiles
a destruction of cats
a troop of baboons
a barrel of monkeys
a band of gorillas
a tower of giraffes
a parliament of owls
a cackle of hyenas
a smack of jellyfish
an ambush of tigers
a wisdom of wombats
Best regards
Elemer
[Updated on: Thu, 07 June 2018 10:14] Report message to a moderator |
|
| | | | | | | |
Goto Forum:
Current Time: Thu Apr 25 04:36:30 GMT 2024
Powered by FUDForum. Page generated in 0.03177 seconds |
] 
Search
Help
Register
Login
Home
