| Avoiding all initialization at every test case [message #1780366] |
Sun, 21 January 2018 16:20  |
|
In the TTCN-3 test suites I've written so far, and which I'm compiling+executing with TITAN, all initialization of state between IUT and testcase is performed completely for every test case. While I understand the rationale behind this and agree it is typically not a good idea to leak state between test cases in a test suite, there are for sure scenarios where it is a completely avoidable overhead.
Example (using GSM/3GPP language but given TITAN history I presume people are familiar with it):
* I want to test the Layer3 MM of a MSC., such as LocationUpdate procedure
* For every test case, TITAN brings up M3UA, establishes AS/ASP with the MSC, then brings up the A interface using BSSMAP RESET procedure, to then finally establish a SCCP/BSSAP/DTAP connection and preforming the L3 MM testing.
So I have to bring up all the lower layers for each and every test case, even though my test is only interested in the highest-level protocol. I have separate test suites for the lower-level protocols, and there's no need in doing a poor mans job at re-testing it at every L3 testcase.
So what I'm looking for is some mechanism by which I can keep some TITAN components running throughout the test suite run, started before the first test case and terminated only after the last testcsase. This way the M3UA/SCCP/BSSMAP-Reset could happen once.
Currently, this lower-layer bring-up (which actually involves more interfaces and protocols than in this example) is taking a multitude of the individual test case. So valuable test execution time is wasted with repeated bring-up of those parts that currently are not even bein tested.
Given that this type of problem basically exists in any complex/deep protocol stack, I would be surprised if there wasn't some kind of solution in either the TTCN-3 language or in TITAN.
One ugly hack would be to simply put many individual tests in one testcase. But then U loose valuable granularity in reporting, i.e. I don't see anymore in which of the many steps in one longer testase actually failed, which is not what I want.
Thanks in advance for any assistance.
|
|
|
| Re: Avoiding all initialization at every test case [message #1780398 is a reply to message #1780366] |
Mon, 22 January 2018 07:55  |
|
Hi Harald,
when a test case terminates , the MTC is destroyed , which means that the test suite cannot sustain signaling or communication links from test case to test case. The connected network does not cope well with such a situation, equivalent with the disappearance of a node ( the node simulated in TTCN-3) from the network:
alarms are raised, supervision screams etc. When re-established , the links need to stabilize which slows down test execution; supervision has memory so may isolate links that appear faulty and so on.
You need a separate executable to circumvent the negative effects; in fact we have a whole class of products we call "daemons" exactly for that : to survive termination of a test case.
These daemons implement the lower layer protocols and they are started at the beginning of a test execution and demolished at the end;
The components for the upper layer protocol/protocols are started and terminated as required by the test cases ; when started, the upper layers will connect to the daemon and the link to the SUT is established.
They are, as I mentioned, typically separate executables and they can be conveniently started /stopped in the [EXTERNAL_COMMANDS] section of the config file.
N.B. this behaviour replicates the TRI (TTCN-3 runtime interface) containing the test adapters as described in the standard.
Right now , we have the below dameonic products:
Please take a look and if interested I will check if I can make them open source.
Best regard
Elemer
Attachment: Daemon.png
(Size: 39.04KB, Downloaded 224 times)
|
|
|
Powered by
FUDForum. Page generated in 0.03136 seconds
] 
Search
Help
Register
Login
Home
