[jetty-users] Concurrency issues with the WebSocket client

[Jörg Henne <hennejg@xxxxxxxxx>]

Hi,

with a WebSocket client implemented using Jetty 9.2.11 I am experiencing
strange concurrency behaviour with which I have difficulties telling "works
as designed" from "that's a bug".

The client in Question is implemented using the @ClientEndpoint annotation.
It has an onMessage method annotated with @OnMessage. The client is
initialized using a pattern along the lines of
ContainerProvider.getWebSocketContainer().connectToServer(theClient,
serverURI);.

In situations where the server sends a few messages in rapid succession, I
experienced unexpected application behaviour which I tracked down to
concurrent calls to the onMessage method being made. 

I verified that observation using something like this:

  AtomicInteger onMessageConcurrency = new AtomicInteger();

  @OnMessage
  public void onMessage(Serializable message) throws InterruptedException {
    final int c = onMessageConcurrency.incrementAndGet();
    try {
      if (c > 1)
        LOGGER.warn("Concurrent calls of onMessage >1: " + c);

      ... do something
    } finally {
      onMessageConcurrency.decrementAndGet();
    }
  }

To me this seems to be a violation of the
single-invocation-per-endpoint-instance contract WSC-5.1-2 in the
specification. To make things even worse, even if the onMessage method is
made synchronized, the calls may happen in a different order from the one in
which the messages were sent, thus destroying the message ordering.

A few things I already looked at are
- there are indeed different threads that call into onMethod. Their stacks
look something like this:

  WebSocketJobController.onMessage(Serializable) line: 332	
  NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not
available [native method]	
  NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57	
  DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
  Method.invoke(Object, Object...) line: 601	
  OnMessageTextStreamCallable(CallableMethod).call(Object, Object...) line: 70	
  OnMessageTextStreamCallable.call(Object, Reader) line: 60	
  JsrEvents<T,C>.callTextStream(RemoteEndpoint$Async, Object, Reader) line: 206	
  JsrAnnotatedEventDriver$2.run() line: 340	
  QueuedThreadPool.runJob(Runnable) line: 635	
  QueuedThreadPool$3.run() line: 555	
  Thread.run() line: 722	

- it is indeed the same client instance the calls are being made on
- the server side sends the messages in the correct order using a single
thread using synchronous delivery via session.getBasicRemote().sendObject(...)

So, I'd love to hear your opinion on this. Is this a bug or correct
behaviour? The specification doesn't explicitly mention the client with
respect to the single-thread-rule, but the client is certainly just an
endpoint so I think it should apply. 

Thanks
Jörg Henne