Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [escet-dev] How to test 3D digital twins?
  • From: Dennis Hendriks <dh_tue@xxxxxxxxxxx>
  • Date: Tue, 6 Jul 2021 19:53:29 +0000
  • Accept-language: nl-NL, en-US
  • Arc-authentication-results: i=1; 1; spf=none; dmarc=none; dkim=none; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed;; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XheF6jkHHM/ombQoPLdir2tIKaaqiCiANY9IfK3QhN4=; b=ntexEr3IdT0FTHKS7286yZBPvLROQ3hbegxi4sgYITA7Zejz7yIZV2JalP6nG+/GizkfvEsSdqIP9jrr1kVQfMd1bcYMFn96Yv5Eo72Sg8zmz6WQEUjcyTegXfN2FgJmtt6uNasZ+khmNl4OA3pV6wbOvh0k39iUzJ3QLLDFUHf11fUWbP9Q3M95AkXB3kEz3IP/E7HUeA8bEGRYNVf4JOBMJA1mw8eGzuI6SLpTsY2wzsK10bLzOjkCVb6R9dMIXx+PVlv4o5+t8nRyD1hvAy5OdksDmiixv2mtTh+dN/KQcysA+bugschjLF8MK93UO5Du5PIHerSt5GXPaEN56Q==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901;; cv=none; b=dOu4GC5pO+6pgTsHjZlbKnvfRlwjri+cVdoezFtaJyTQ8jbVkw9XtRKCBLG5KCP0q82WBepx1EKXY+gsydjF7bJS4z5oMrEUi+bw/6Nf2guto53LG+ks1vuk2zwnCoCpr/x9EhY3RKZHC4Qw7v6/iXov2svkeblOS5ARSMVGdZFpGurPxq2uNj4FFEbQdokJUP6p0p31CrjmHOyvQxMUjpjOwdmY10KvNX3myoQLj95V5fGndzB2XqfLYCq2CgSUh6Z4kuRKZmKsDNAqLAFRglxkEiHWhd2z/0MHTIAzdvXqGi4YPEWLYkxdGQaUNh/g1N99xaGRQvv6XDG44KtsmA==
  • Delivered-to: escet-dev@xxxxxxxxxxx
  • List-archive: <>
  • List-help: <>
  • List-subscribe: <>, <>
  • List-unsubscribe: <>, <>
  • Thread-index: AQHXcXuzCSQv2FVasUaOjy4iuiARYas0jGBLgAGfsQCAAC0Jzg==
  • Thread-topic: [escet-dev] How to test 3D digital twins?

Hi Bert,

Disclaimer: I'm not an expert on testing per se.

I assume you have to test the physical system manually? This is not automated? I assume you can test the digital twin fully automatically, if you provide the scenarios to execute?

One issue is how to get good coverage, I think. You could perform random simulations on the digital twin and observe the outcome (success, failure, states, or so). Then you could perform the same scenarios on the real system, and maybe observe the outcome as well. But to get good coverage, you'll need many scenarios. Some states may then be difficult to reach, as some states have higher chances of being reached than others. A classical problem is an automaton with as alphabet all 10 digits and which accepts only a specific password or phone number. The number of possible sequences is exponential in the sequence length and only one such sequence is accepted. It's highly unlikely you'll ever find it by black box testing. The same problem occurs in active automata learning, where conformance testing also plays a large role. There is quite some literature on this. This is not just theoretical, but a very practical problem as well.

But that assumes black box testing. If you know the model, you can use whitebox testing techniques. You could generate test scenarios from the model (digital twin model) and test them on the physical twin. You would need to generate both scenarios for behavior accepted by the digitial twin model, and for behavior not accepted by it. You would have to check that (non)acceptance against the (non)acceptance of the physical one. There are algorithms that can generate a 'complete' test set, under certain conditions. But still, the number of tests may be very large.

Another question is: what constitutes acceptance? It could be accepting states, if that is something you can define for your system. You could also define input and output subsets, and see for a sequence of inputs whether the sequence of outputs matches, similar to Mealy machines.

Does this help you in any way?


Van: escet-dev <escet-dev-bounces@xxxxxxxxxxx> namens Beek, Bert van <D.A.v.Beek@xxxxxx>
Verzonden: dinsdag 6 juli 2021 18:54
Aan: escet developer discussions <escet-dev@xxxxxxxxxxx>
Onderwerp: Re: [escet-dev] How to test 3D digital twins?
Hi Dennis,

The goal of this exercise is to find errors in the digital twins. Last year, due to the Corona restrictions, testing was optional. There was only a single group who tested on the festo workstations. Their controller worked without errors on the digital twin, but it did not work on the Festo workstation. The reason for this was that the actuator setting to immediately stop the sliding manipulator was true/true (pressure on both sides), whereas in the digital twin, the actuator setting for this was false/false. False/false (absence of pressure) on the Festo workstation caused the sliding manipulator to slowly stop => stopping at an incorrect location.

“Behave in the same way” means that a controller that works correctly on the twin should work correctly on the Festo system, and that a controller that shows erroneous behavior on the Festo workstation should show the same erroneous behavior on the twin.

Ideally, we would like to consider the full state space, but obviously that is impossible. The controller is hybrid (timed) so has infinite state space.

In principle we only look at the observable behavior at the festo workstation and at the twin (it a product picked up properly and release at the required target location), but we could probably generate certain graphs of behavior in TwinCAT.

  • Is full automation a requirement?
I don’t understand this question.


On 5 Jul 2021, at 18:09, Dennis Hendriks <dh_tue@xxxxxxxxxxx> wrote:

Hi Bert,

Some random questions that immediately pop into my mind:
  • What is the goal of this exercise? What do you hope to gain?
  • What does 'behave in the same way' mean for you?
  • Do you consider the full state space of behavior, or only a finite set of scenarios?
  • Do you consider the protocol only (order of events), or also timing?
  • Is full automation a requirement?

Van: escet-dev <escet-dev-bounces@xxxxxxxxxxx> namens Beek, Bert van <D.A.v.Beek@xxxxxx>
Verzonden: maandag 5 juli 2021 10:56
Aan: escet developer discussions <escet-dev@xxxxxxxxxxx>
Onderwerp: [escet-dev] How to test 3D digital twins?
How would you test the functionality of digital twins for the Festo workstations as shown in

A teacher assistant aims to test all CIF controllers submitted last year as final assignment for the course 4TC00 on the digital twin and on the Festo workstation to see if they behave in the same way.

I could also imagine that we write some small controllers that test specific functionality of specific sensor actuator relations/behavior.

Any thoughts or tips?

Thanks in advance
escet-dev mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit

Back to the top