[theia-dev] setTimeout() can lead to flaky tests

[Thomas Mäder <t.s.maeder@xxxxxxxxx>]

Hi folks,

I've recently been fixing a bunch of tests that have been failing on and off on CI and here's what I found: the pattern is the following:

setTimeout(doThingOne, 10);

setTimeout(doThingTwo, 20);

asssert.istTrue(thingOneHappenedBeforeThingTwo);

This test may fail in high-load environments (like CI)! Nodejs guarantees no order between different timeouts that are ready for execution. So if the test suite does not execute for 20ms at all because the test machine is overloaded, either "doThingOne" or "doThingTwo" may be executed first (https://nodejs.org/api/timers.html#settimeoutcallback-delay-args). So the lesson is:

Don't rely on timeouts to order things!

One way to solve this in the above example would be:

setTimeout(() => {

   doThingOne();

   setTimeout(doThingTwo, 10);

}, 10);

I know this is quite counterintuitive, so I thought I'd share this in case you're battling similar cases.

/Thomas