Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [websocket-dev] refactored websocket tck test failures

On 07/04/2023 08:26, Gurunandan Rao wrote:
*

Hello Websocket-dev,


We are in the process of refactoring the WebSocket TCK to use Junit and Arquillian.

Who is "we"? This is the first mention of refactoring the WebSocket TCK I am aware of on this list. I would have expected some discussion or at least a heads-up first.

The WIP PR for refactoring can be found at https://github.com/jakartaee/platform-tck/pull/1160/files <https://github.com/jakartaee/platform-tck/pull/1160/files>.

I took a quick look at the PR.

Generally, it is extremely hard to see what has changed because of the large volume of format changes. I'm all for cleaning up the format, making it consistent and improving the readability of the code but I would do in a separate commit (or better still a separate PR) so reviewers can concentrate on the actual changes.

In its present form I would find the PR impossible to review.

Post refactor of the websocket tck tests, most of the tests are passing with glassfish 7 and JDK 11.

Great. Sounds like the migration is going well.

There are 16 negdep client tests [eg: com.sun.ts.tests.websocket.negdep.onclose.client.duplicate.WSCClientIT ], which are failing with `deployment to expected fail, but it succeeded` error.

Based on analysis of failing tests with Jtest, the test war deployment succeeds in glassfish, test client would report deployment error resulting in the test to pass.

OK. Given that all the client negative deployment tests are failing (and only those tests) that suggests a common root cause.

The CI run for the refactored code can be found at: https://ci.eclipse.org/jakartaee-tck/job/guru/job/jakarta-websocket-tck-glassfish/17/consoleFull <https://ci.eclipse.org/jakartaee-tck/job/guru/job/jakarta-websocket-tck-glassfish/17/consoleFull>

The failed test logs can be found at: https://ci.eclipse.org/jakartaee-tck/job/guru/job/jakarta-websocket-tck-glassfish/17/artifact/glassfish-runner/websocket-tck/target/failsafe-reports/ <https://ci.eclipse.org/jakartaee-tck/job/guru/job/jakarta-websocket-tck-glassfish/17/artifact/glassfish-runner/websocket-tck/target/failsafe-reports/>

Glassfish logs at: https://ci.eclipse.org/jakartaee-tck/job/guru/job/jakarta-websocket-tck-glassfish/17/artifact/glassfish-runner/websocket-tck/target/glassfish7/glassfish/domains/domain1/logs/server.log <https://ci.eclipse.org/jakartaee-tck/job/guru/job/jakarta-websocket-tck-glassfish/16/artifact/glassfish-runner/websocket-tck/target/glassfish7/glassfish/domains/domain1/logs/server.log>

There isn't actually a question for the members of this list but I am assuming the implied question is "Why is this failing?".

I'm still not that familiar with the WebSocket TCK. I've fixed a few bugs and done some of the version updates but I wouldn't claim to know or understand the TCK code that well.

It looks like the updated tests are expecting the WAR deployment to fail. That won't happen for these tests since the deliberate error is on the client side not the server side.

Looking at the results for last test run I have locally, the expected behaviour is that the jakarta.websocket.DeployException is thrown by the client during the call to WebSocketContainer.connectToServer()

I think adjusting the negdep.*.client tests so that they proceed to the initial WebSocket connection and then test if that fails with a DeployException should get the currently failing tests working as intended.

HTH,

Mark



Back to the top