In general, I would say that seeing a test fail often is not a valid reason for excluding it.
To exclude a test, there needs to be a reason why something in the test is incorrect or if it is testing something beyond the specification requirements. You might have intended to include that information
in your email, but I don’t see it stated anywhere.
If I click on the results link, I see a more informative error message that goes with the
assertion,
[ Response code: 200, Response body: FAILURECaught exception attempting to call test method testScheduleWithCronTrigger on servlet ee.jakarta.tck.concurrent.spec.ManagedScheduledExecutorService.resourcedef.ManagedScheduledExecutorDefinitionServletjava.lang.AssertionError:
Must be able to cancel a repeating task after it executes a few times: org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$TriggerControllerFuture@475ae802[Cancelled] expected [true] but found [false]
The test itself looks reasonable to me. Are you certain that the implementation you are testing does not itself have a bug in sometimes reporting back a wrong result of future.cancel(true) ?
From:
cu-dev <cu-dev-bounces@xxxxxxxxxxx> on behalf of Scott Marlow via cu-dev <cu-dev@xxxxxxxxxxx>
Date: Wednesday, August 28, 2024 at 9:58 AM
To: cu developer discussions <cu-dev@xxxxxxxxxxx>
Cc: Scott Marlow <smarlow@xxxxxxxxxx>
Subject: [EXTERNAL] [cu-dev] Concurrency 3.0 TCK test failures in ManagedScheduledExecutorDefinitionTests.testScheduleWithCronTrigger
From https: //ci. eclipse. org/jakartaee-tck/job/10/job/jakarta-concurrency-tck-glassfish/138/consoleText:
" [INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR] ManagedScheduledExecutorDefinitionTests>Arquillian. run: 138->testScheduleWithCronTrigger: 147->TestClient. runTest: 52->TestClient. runTest: 60->TestClient. assertSuccessfulURLResponse: 143->ArquillianTests. assertTrue: 108
From
https://ci.eclipse.org/jakartaee-tck/job/10/job/jakarta-concurrency-tck-glassfish/138/consoleText:
"
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ManagedScheduledExecutorDefinitionTests>Arquillian.run:138->testScheduleWithCronTrigger:147->TestClient.runTest:52->TestClient.runTest:60->TestClient.assertSuccessfulURLResponse:143->ArquillianTests.assertTrue:108 Output did not contain successful
message: SUCCESS expected [true] but found [false]
[INFO]
[ERROR] Tests run: 148, Failures: 1, Errors: 0, Skipped: 0
"
We see this failure with WildFly a lot but it seems intermittent.
Should we consider excluding this test for Concurrency 3.0? If yes, it shouldn't be hard to open a TCK challenge for this test.
