[jetty-users] Asynchronous servlets - thread safety/synchronization & re

["Bartek K." <kowalewski.bartosz@xxxxxxxxx>]

Hi Jetty gurus,

I've just started creating a simple asynchronous servlet running on
top of a Jetty8 server. I first tried using the Jetty native API
(continuations) as I used to employ a similar approach when creating
Jetty6-based server-side components. Then I switched to the Servlet3
API to make my code less coupled to Jetty classes. In both cases I had
some issues with implementing proper synchronization mechanisms that
would ensure that timeout tasks do not interfere with proper responses
that cause continuations to be resumed around the time timeout
happens. I also had problems with handling late responses - responses
that might potentially cause the continuation to be resumed long after
timeout happened.

Some facts:

I. Jetty reuses plenty of objects. This includes continuations,
servlet request objects, servlet response objects, ... Let's imagine
this scenario:
1. The servlet gets a new request (Req1), a continuation object
(Cont1) is retrieved  from ContinuationSupport
2. Continuation Cont1 is suspended
3. A timeout is triggered for Cont1, it causes a response to be sent
to the client
4. The client sends a new request (Req2) over the same TCP connection
(reuses the connection), Jetty knows that Req1 was finished, so it
resets Cont1 and reuses it
5. Cont1 for Req2 is suspended
6. Late response for request Req1 comes; it cannot use the
continuation that was assigned to Req1 as this continuation is already
used by Req2
7. When creating the servlet I need to implement my custom loigic for
checking if I'm still allowed to set the response or if timeout has
already expired. Just checking isExpired() is not enough as the
continuation has been reset

II. When expiring a continuation, Jetty first changes the state of the
continuation in a safe 'synchronized' block of code and then calls
listeners (continuation listeners & servlet3 async listeners) outside
of this block. In order to ensure that my servlet doesn't reach an
inconsistent state when handling a given request (i.e. proper response
sets HTTP code on the response while the listener is in its onTimeout
method), I need to put all o the actions responsible for setting the
response into a synchronized block.

Here's how I've done it when using Jetty native API:

1. Setting up the continuation:
{code}
UUID uuid = UUID.randomUUID();
continuation.setAttribute(UUID_ATTRIBUTE_NAME, uuid);
continuation.addContinuationListener(new ContinuationListener()
            {
                public void onTimeout(Continuation continuation) {
                    try {
                        synchronized (continuation) {
                            HttpServletResponse response =
(HttpServletResponse) continuation.getServletResponse();
                            response.setContentType("text/html");
                            response.setStatus(;
                            response.getWriter().println("");
                            continuation.complete();
                            continuation.removeAttribute(UUID_ATTRIBUTE_NAME);
                        }
                    } catch (Exception e) {
                      ...
                    }
                }

                public void onComplete(Continuation continuation) {
                     ...
                }
 });
continuation.suspend(response);
...
{code}

Handling respone:
{code}
// my internal code keeps track of the continuation assigned to the
given exchange and uuid of the exchange, so that I can resume/complete
the continuation when processing is done
synchronized (continuation) {
                    if
(uuid.equals(continuation.getAttribute(UUID_ATTRIBUTE_NAME)) &&
!continuation.isExpired()) {
                        HttpServletResponse response =
(HttpServletResponse) continuation.getServletResponse();
                        response.setContentType();
                        response.setStatus();
                        response.getWriter().println("");
                        continuation.complete();
                        continuation.removeAttribute(UUID_ATTRIBUTE_NAME);
                    }
                }
{code}

... so I need some advice. Does this code look safe? It's based on
initial static analysis o Jetty8 source code and some debugging, but I
haven't carried out thorough tests yet, so I'm not 100% sure that is
ok.

I also took a look at examples shipped with Jetty. I haven't found any
such mechanisms there. Shouldn't mechanism similar to the one I used
be introduced in these examples? I know that my approach is far from
being optimal :), but it seams that some synchronization is needed in
these examples.

As I mentioned at the very begining , I also tried the Servlet3 async
API. I did some tests and it seems that similiar issues apply here. My
simple AsyncContext-based servlet seemed to be called from the same
impl of continuation as in the Jetty native API based example. It was
called from AsyncContinuation. This suggests that similar
synchronization/uuid stuff needs to be introduced there. While
introducing uuids there is not a problem, synchronization seems to be
an issue. Jetty code suggests that the only way to make my response
handling code properly synchronized is to synchronize this block on
the continuation object that is used by Jetty underneath (similary to
what is done in the second snippet above). The issue is that
introducing such synchronization block would make my servlet
non-Jetty-agnostic. Any thoughts? Any good examples out there?

Sorry for a longish e-mail and thanks for your help!
  Bartek