| [jetty-users] Interleaving requests for fairness/starvation? |
|
Hi
I was wondering if anyone could confirm whether what we've done is sensible - it seems to work, but I'm not really confident and would like to know how righteous it is!
Our HTTP server processes requests that can take minutes at a time, the handling for which can easily be broken down into small chunks. We would like to interleave the handling of these requests so that none get starved and time out (and to help with fairness).
For each request, after 50ms of handling, we asynchronously suspend the request and (hope to) redispatch it to the queue behind newly arrived requests / other more-recently suspended requests.
With this approach, the issue seems to be in choosing where to call asyncContext.dispatch: too close to the suspend call (startAsync) and Jetty will not release the thread back to allow processing of other queued requests (presumably because Jetty thinks it's
more efficient to avoid the context switch and so immediately continues processing the current request).
We've currently settled on overriding QueuedThreadPool.execute(Runnable) and tryExecute(Runnable) to wrap the Runnable: any request that had registered itself while the original (unwrapped) Runnable was called is now redispatched. This seems to work in that:
(a) only one request attempts to register itself each time a Runnable is run, (b) the redispatch normally seems to happen very quickly after the request has registered itself, and (c) the request handling is indeed interleaved.
Does that sound like the right thing to do here? I'm thinking it would be as long as each request handle is guaranteed to be the tail call of a task submitted to the thread pool (I don't know enough about the use of ExecutionStrategy to know this).
Any suggestions/comments are highly appreciated.
Matt
P.S. I'm anticipating that the right thing to do might be to use a separate thread pool and manage the interleaving there? This of course would
incur another thread context switch / managing another pool. We're also using ThreadLimitHandler to avoid single clients taking out the whole server, so would need to duplicate that logic.
|