Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[jetty-users] Odd EventSource behavior

I’m trying to test the org.eclipse.jetty.servlets.EventSourceServlet (using v9.4.15), but am seeing some odd behavior.

I have a scheduled executor service update 1 in 10 random numbers every 333 millis. This is logged something like this:

19:20:56.876 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "4" to 4.293720580839897

Once updated, all subscribers are notified, like this:

  private def send(emitter: Emitter, id: Int)(value: Double): Unit = {
    try emitter.data(toJson(value)) catch {
      case th: Throwable =>
        System.err.println(s"Send of $id failed: ${th.getMessage} ($th)")
        emitter.close()
    }
  }

This all works fine, until I start randomly disconnecting clients. Many times the EventSource.onClose() is correctly called, but other times we hit a race condition and try to send to to a client that has just disappeared. As can be seen in the code above, all Throwables are caught in printed. I’ve tested both with and without the emitter.close() call, and things work fine if I don’t manually close. But this is not necessarily intuitive and is probably a bug. A send failure usually leads to two errors. It can be seen in the log as:

Send of 3 failed: null (org.eclipse.jetty.io.EofException)

then

Send of 3 failed: AsyncContext completed and/or Request lifecycle recycled (java.lang.IllegalStateException: AsyncContext completed and/or Request lifecycle recycled)

After the second failure, something odd happens, which can be seen by the timestamps, 10 seconds pauses start to happen, nothing being logged, not even from the scheduled executor service I have running every 333 millis.

Below is the log:

19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=HTTP/1.1{s=200,h=2,cl=-1},cb=org.eclipse.jetty.server.HttpChannel$SendCallback@c9939ef] generate: FLUSH ([p=0,l=85,c=8192,r=85],[p=0,l=0,c=0,r=0],false)@COMMITTED
19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [HeapByteBuffer@45a10322[p=0,l=85,c=8192,r=85]={<<<HTTP/1.1 200 OK\r\n...tion: close\r\n\r\n>>>ent-stream\r\nCache...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 85 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=6/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=DISPATCHED,uri=//localhost:8080/random/2,age=4}
19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=85 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=HTTP/1.1{s=200,h=2,cl=-1},cb=org.eclipse.jetty.server.HttpChannel$SendCallback@c9939ef] generate: DONE ([p=85,l=85,c=8192,r=0],[p=0,l=0,c=0,r=0],false)@COMMITTED
19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpChannelState - startAsync HttpChannelState@37d1bac3{s=DISPATCHED a=NOT_ASYNC i=true r=IDLE w=false}
19:20:54.020 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=38,c=32768,r=38]={<<<data: {"value":2....4052646617}\r\n\r\n>>>eta charset="utf-...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@d2ccf86] generate: FLUSH (null,[p=0,l=38,c=32768,r=38],false)@COMMITTED
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=38,c=32768,r=38]={<<<data: {"value":2....4052646617}\r\n\r\n>>>eta charset="utf-...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 38 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=1/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=DISPATCHED,uri=//localhost:8080/random/2,age=5}
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=38 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@d2ccf86] generate: DONE (null,[p=38,l=38,c=32768,r=0],false)@COMMITTED
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.server.Server - handled=true async=true committed=true on HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=DISPATCHED,uri=//localhost:8080/random/2,age=5}
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpChannelState - unhandle HttpChannelState@37d1bac3{s=DISPATCHED a=STARTED i=true r=IDLE w=false}
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=5} action WAIT
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=5} handle exit, result WAIT
19:20:54.021 [qtp564160838-20] DEBUG org.eclipse.jetty.server.HttpConnection - HttpConnection@2a55d598::SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=5} onFillable exit HttpChannelState@37d1bac3{s=ASYNC_WAIT a=STARTED i=false r=IDLE w=false} null
19:20:54.022 [qtp564160838-20] DEBUG org.eclipse.jetty.util.thread.ReservedThreadExecutor - ReservedThreadExecutor@4c762604{s=2/8,p=0}@1a30e2df waiting
19:20:54.199 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "8" to 8.913855132997274
19:20:54.532 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "1" to 1.248433380160289
19:20:54.866 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "3" to 3.7693731336796032
19:20:54.866 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@2393088e[p=0,l=38,c=32768,r=38]={<<<data: {"value":3....1336796032}\r\n\r\n>>>eta charset="utf-...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@16786285{null}
19:20:54.866 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@657fafbc[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@3503f0ff] generate: FLUSH (null,[p=0,l=38,c=32768,r=38],false)@COMMITTED
19:20:54.866 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@1762cc62{IDLE}->null [DirectByteBuffer@2393088e[p=0,l=38,c=32768,r=38]={<<<data: {"value":3....1336796032}\r\n\r\n>>>eta charset="utf-...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:20:54.866 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@1762cc62{WRITING}->null:IDLE-->WRITING
19:20:54.866 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 38 SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=1336/30000}{io=0/0,kio=0,kro=1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=11809}
19:20:54.866 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=38 remaining=0 WriteFlusher@1762cc62{WRITING}->null
19:20:54.866 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@1762cc62{IDLE}->null:WRITING-->IDLE
19:20:54.866 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@657fafbc[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@3503f0ff] generate: DONE (null,[p=38,l=38,c=32768,r=0],false)@COMMITTED
19:20:55.200 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "7" to 7.3874226404833605
19:20:55.535 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "4" to 4.333509109928213
19:20:55.869 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "2" to 2.713109756473337
19:20:55.869 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=37,c=32768,r=37]={<<<data: {"value":2....9756473337}\r\n\r\n>>>\neta charset="utf...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:20:55.869 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@3a8f8bb8] generate: FLUSH (null,[p=0,l=37,c=32768,r=37],false)@COMMITTED
19:20:55.869 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=37,c=32768,r=37]={<<<data: {"value":2....9756473337}\r\n\r\n>>>\neta charset="utf...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:20:55.869 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:20:55.869 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 37 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=1848/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=1853}
19:20:55.869 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=37 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:20:55.869 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:20:55.869 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@3a8f8bb8] generate: DONE (null,[p=37,l=37,c=32768,r=0],false)@COMMITTED
19:20:56.203 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "2" to 2.6343071213128537
19:20:56.203 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=38,c=32768,r=38]={<<<data: {"value":2....1213128537}\r\n\r\n>>>eta charset="utf-...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:20:56.203 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@15382099] generate: FLUSH (null,[p=0,l=38,c=32768,r=38],false)@COMMITTED
19:20:56.203 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=38,c=32768,r=38]={<<<data: {"value":2....1213128537}\r\n\r\n>>>eta charset="utf-...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:20:56.203 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:20:56.203 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 38 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=334/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=2187}
19:20:56.203 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=38 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:20:56.203 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:20:56.203 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@15382099] generate: DONE (null,[p=38,l=38,c=32768,r=0],false)@COMMITTED
19:20:56.537 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "3" to 3.4095049407020968
19:20:56.537 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@2393088e[p=0,l=38,c=32768,r=38]={<<<data: {"value":3....9407020968}\r\n\r\n>>>eta charset="utf-...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@16786285{null}
19:20:56.537 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@657fafbc[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@d83cc25] generate: FLUSH (null,[p=0,l=38,c=32768,r=38],false)@COMMITTED
19:20:56.537 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@1762cc62{IDLE}->null [DirectByteBuffer@2393088e[p=0,l=38,c=32768,r=38]={<<<data: {"value":3....9407020968}\r\n\r\n>>>eta charset="utf-...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:20:56.537 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@1762cc62{WRITING}->null:IDLE-->WRITING
19:20:56.537 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - write exception
org.eclipse.jetty.io.EofException: null
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:284)
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:393)
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:380)
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:826)
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:550)
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:850)
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:921)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:250)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:226)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: An established connection was aborted by the software in your host machine
    at sun.nio.ch.SocketDispatcher.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:51)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:51)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:262)
    ... 34 common frames omitted
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@1762cc62{FAILED}->null:WRITING-->FAILED
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - Commit failed
org.eclipse.jetty.io.EofException: null
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:284)
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:393)
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:380)
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:826)
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:550)
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:850)
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:921)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:250)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:226)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: An established connection was aborted by the software in your host machine
    at sun.nio.ch.SocketDispatcher.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:51)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:51)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:262)
    ... 34 common frames omitted
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - abort HttpConnection@3eb0a486::SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=F,to=1673/30000}{io=0/0,kio=0,kro=1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13482} {}
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.AbstractEndPoint - close SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=F,to=1673/30000}{io=0/0,kio=0,kro=1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13482}
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.AbstractEndPoint - close(null) SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=F,to=1673/30000}{io=0/0,kio=0,kro=1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13482}
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ChannelEndPoint - doClose SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,CLOSED,fill=-,flush=F,to=1673/30000}{io=0/0,kio=0,kro=1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13482}
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.WriteFlusher - ignored: WriteFlusher@1762cc62{FAILED}->null
java.nio.channels.ClosedChannelException: null
    at org.eclipse.jetty.io.WriteFlusher.onClose(WriteFlusher.java:502)
    at org.eclipse.jetty.io.AbstractEndPoint.onClose(AbstractEndPoint.java:353)
    at org.eclipse.jetty.io.ChannelEndPoint.onClose(ChannelEndPoint.java:215)
    at org.eclipse.jetty.io.AbstractEndPoint.doOnClose(AbstractEndPoint.java:225)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:192)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:175)
    at org.eclipse.jetty.server.HttpConnection.abort(HttpConnection.java:567)
    at org.eclipse.jetty.server.HttpChannel.abort(HttpChannel.java:966)
    at org.eclipse.jetty.server.HttpChannelOverHttp.abort(HttpChannelOverHttp.java:495)
    at org.eclipse.jetty.server.HttpChannel$SendCallback.failed(HttpChannel.java:1314)
    at org.eclipse.jetty.io.AbstractConnection.lambda$failedCallback$0(AbstractConnection.java:92)
    at org.eclipse.jetty.io.AbstractConnection.failedCallback(AbstractConnection.java:115)
    at org.eclipse.jetty.server.HttpConnection.access$1300(HttpConnection.java:53)
    at org.eclipse.jetty.server.HttpConnection$SendCallback.onCompleteFailure(HttpConnection.java:880)
    at org.eclipse.jetty.util.IteratingCallback.failed(IteratingCallback.java:401)
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:302)
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:380)
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:826)
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:550)
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:850)
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:921)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:250)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:226)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.FillInterest - onClose FillInterest@7c08b270{null}
19:20:56.539 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ManagedSelector - Wakeup ManagedSelector@3c130745{STARTED} id=0 keys=1 selected=0 updates=0
19:20:56.540 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.util.thread.QueuedThreadPool - queue org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint@27ad5162
19:20:56.540 [qtp564160838-14] DEBUG org.eclipse.jetty.util.thread.QueuedThreadPool - run org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint@27ad5162
19:20:56.540 [qtp564160838-19] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.WindowsSelectorImpl@d20b5b5 woken with none selected
19:20:56.540 [qtp564160838-19] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.WindowsSelectorImpl@d20b5b5 woken up from select, 0/0/0 selected
19:20:56.540 [qtp564160838-19] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.WindowsSelectorImpl@d20b5b5 processing 0 keys, 0 updates
19:20:56.540 [qtp564160838-19] DEBUG org.eclipse.jetty.io.ManagedSelector - updateable 0
19:20:56.540 [qtp564160838-19] DEBUG org.eclipse.jetty.io.ManagedSelector - updates 0
19:20:56.540 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpOutput - 
org.eclipse.jetty.io.EofException: null
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:284)
    at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:393)
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:380)
    at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:826)
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
    at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:550)
    at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:850)
    at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:921)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:250)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:226)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: An established connection was aborted by the software in your host machine
    at sun.nio.ch.SocketDispatcher.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:51)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:51)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:262)
    ... 34 common frames omitted
19:20:56.540 [qtp564160838-19] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector sun.nio.ch.WindowsSelectorImpl@d20b5b5 waiting with 0 keys
19:20:56.540 [qtp564160838-14] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
    at org.eclipse.jetty.io.ManagedSelector.safeInterestOps(ManagedSelector.java:308)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:432)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:955)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.540 [qtp564160838-14] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
    at org.eclipse.jetty.io.ManagedSelector.safeReadyOps(ManagedSelector.java:294)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:433)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:955)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.540 [qtp564160838-14] DEBUG org.eclipse.jetty.io.ManagedSelector - Destroyed SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,CLOSED,fill=-,flush=F,to=1674/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13483}
19:20:56.540 [qtp564160838-14] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
    at org.eclipse.jetty.io.ManagedSelector.safeInterestOps(ManagedSelector.java:308)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:432)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at java.util.Formatter$FormatSpecifier.printString(Formatter.java:2886)
    at java.util.Formatter$FormatSpecifier.print(Formatter.java:2763)
    at java.util.Formatter.format(Formatter.java:2520)
    at java.util.Formatter.format(Formatter.java:2455)
    at java.lang.String.format(String.java:2940)
    at org.eclipse.jetty.io.AbstractConnection.toString(AbstractConnection.java:290)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.io.AbstractConnection.onClose(AbstractConnection.java:221)
    at org.eclipse.jetty.server.HttpConnection.onClose(HttpConnection.java:520)
    at org.eclipse.jetty.io.SelectorManager.connectionClosed(SelectorManager.java:345)
    at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:958)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.540 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
    at org.eclipse.jetty.io.ManagedSelector.safeInterestOps(ManagedSelector.java:308)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:432)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at java.util.Formatter$FormatSpecifier.printString(Formatter.java:2886)
    at java.util.Formatter$FormatSpecifier.print(Formatter.java:2763)
    at java.util.Formatter.format(Formatter.java:2520)
    at java.util.Formatter.format(Formatter.java:2455)
    at java.lang.String.format(String.java:2940)
    at org.eclipse.jetty.io.AbstractConnection.toString(AbstractConnection.java:290)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.server.HttpConnection.abort(HttpConnection.java:564)
    at org.eclipse.jetty.server.HttpChannel.abort(HttpChannel.java:966)
    at org.eclipse.jetty.server.HttpChannelOverHttp.abort(HttpChannelOverHttp.java:495)
    at org.eclipse.jetty.server.HttpOutput.abort(HttpOutput.java:256)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:233)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.540 [qtp564160838-14] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
    at org.eclipse.jetty.io.ManagedSelector.safeReadyOps(ManagedSelector.java:294)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:433)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at java.util.Formatter$FormatSpecifier.printString(Formatter.java:2886)
    at java.util.Formatter$FormatSpecifier.print(Formatter.java:2763)
    at java.util.Formatter.format(Formatter.java:2520)
    at java.util.Formatter.format(Formatter.java:2455)
    at java.lang.String.format(String.java:2940)
    at org.eclipse.jetty.io.AbstractConnection.toString(AbstractConnection.java:290)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.io.AbstractConnection.onClose(AbstractConnection.java:221)
    at org.eclipse.jetty.server.HttpConnection.onClose(HttpConnection.java:520)
    at org.eclipse.jetty.io.SelectorManager.connectionClosed(SelectorManager.java:345)
    at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:958)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.540 [qtp564160838-14] DEBUG org.eclipse.jetty.io.AbstractConnection - onClose HttpConnection@3eb0a486::SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,CLOSED,fill=-,flush=F,to=1674/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13483}
19:20:56.540 [qtp564160838-14] DEBUG org.eclipse.jetty.util.thread.QueuedThreadPool - ran org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint@27ad5162
19:20:56.540 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
    at org.eclipse.jetty.io.ManagedSelector.safeReadyOps(ManagedSelector.java:294)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:433)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at java.util.Formatter$FormatSpecifier.printString(Formatter.java:2886)
    at java.util.Formatter$FormatSpecifier.print(Formatter.java:2763)
    at java.util.Formatter.format(Formatter.java:2520)
    at java.util.Formatter.format(Formatter.java:2455)
    at java.lang.String.format(String.java:2940)
    at org.eclipse.jetty.io.AbstractConnection.toString(AbstractConnection.java:290)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.server.HttpConnection.abort(HttpConnection.java:564)
    at org.eclipse.jetty.server.HttpChannel.abort(HttpChannel.java:966)
    at org.eclipse.jetty.server.HttpChannelOverHttp.abort(HttpChannelOverHttp.java:495)
    at org.eclipse.jetty.server.HttpOutput.abort(HttpOutput.java:256)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:233)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.541 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpConnection - abort HttpConnection@3eb0a486::SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,CLOSED,fill=-,flush=F,to=1674/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13484} {}
19:20:56.541 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
    at org.eclipse.jetty.io.ManagedSelector.safeInterestOps(ManagedSelector.java:308)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:432)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:174)
    at org.eclipse.jetty.server.HttpConnection.abort(HttpConnection.java:567)
    at org.eclipse.jetty.server.HttpChannel.abort(HttpChannel.java:966)
    at org.eclipse.jetty.server.HttpChannelOverHttp.abort(HttpChannelOverHttp.java:495)
    at org.eclipse.jetty.server.HttpOutput.abort(HttpOutput.java:256)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:233)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.541 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
    at org.eclipse.jetty.io.ManagedSelector.safeReadyOps(ManagedSelector.java:294)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:433)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:174)
    at org.eclipse.jetty.server.HttpConnection.abort(HttpConnection.java:567)
    at org.eclipse.jetty.server.HttpChannel.abort(HttpChannel.java:966)
    at org.eclipse.jetty.server.HttpChannelOverHttp.abort(HttpChannelOverHttp.java:495)
    at org.eclipse.jetty.server.HttpOutput.abort(HttpOutput.java:256)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:233)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.541 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.AbstractEndPoint - close SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,CLOSED,fill=-,flush=F,to=1675/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13484}
19:20:56.541 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
    at org.eclipse.jetty.io.ManagedSelector.safeInterestOps(ManagedSelector.java:308)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:432)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:181)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:175)
    at org.eclipse.jetty.server.HttpConnection.abort(HttpConnection.java:567)
    at org.eclipse.jetty.server.HttpChannel.abort(HttpChannel.java:966)
    at org.eclipse.jetty.server.HttpChannelOverHttp.abort(HttpChannelOverHttp.java:495)
    at org.eclipse.jetty.server.HttpOutput.abort(HttpOutput.java:256)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:233)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.541 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.ManagedSelector - 
java.nio.channels.CancelledKeyException: null
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
    at org.eclipse.jetty.io.ManagedSelector.safeReadyOps(ManagedSelector.java:294)
    at org.eclipse.jetty.io.ChannelEndPoint.toEndPointString(ChannelEndPoint.java:433)
    at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:447)
    at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:299)
    at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:271)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:233)
    at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:173)
    at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:680)
    at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:224)
    at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:97)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:181)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:175)
    at org.eclipse.jetty.server.HttpConnection.abort(HttpConnection.java:567)
    at org.eclipse.jetty.server.HttpChannel.abort(HttpChannel.java:966)
    at org.eclipse.jetty.server.HttpChannelOverHttp.abort(HttpChannelOverHttp.java:495)
    at org.eclipse.jetty.server.HttpOutput.abort(HttpOutput.java:256)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:233)
    at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:408)
    at org.eclipse.jetty.server.Response.flushBuffer(Response.java:1233)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.flush(EventSourceServlet.java:217)
    at org.eclipse.jetty.servlets.EventSourceServlet$EventSourceEmitter.data(EventSourceServlet.java:172)
    at test.eventsource.web.TestServlet.test$eventsource$web$TestServlet$$send(TestServlet.scala:48)
    at test.eventsource.web.TestServlet$ValueEmitter.$anonfun$onOpen$1(TestServlet.scala:64)
    at test.eventsource.web.TestServlet.$anonfun$init$2(TestServlet.scala:37)
    at test.eventsource.web.TestServlet.$anonfun$init$2$adapted(TestServlet.scala:37)
    at scala.collection.Iterator.foreach(Iterator.scala:941)
    at scala.collection.Iterator.foreach$(Iterator.scala:941)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scuff.concurrent.MultiMap$Values.foreach(MultiMap.scala:26)
    at test.eventsource.web.TestServlet.$anonfun$init$1(TestServlet.scala:37)
    at scuff.concurrent.package$ScuffScheduledExecutor$$anon$9.run(package.scala:68)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
19:20:56.542 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.io.AbstractEndPoint - close(null) SocketChannelEndPoint@10d8ea5b{/0:0:0:0:0:0:0:1:52806<->/0:0:0:0:0:0:0:1:8080,CLOSED,fill=-,flush=F,to=1675/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@3eb0a486[p=HttpParser{s=END,0 of -1},g=HttpGenerator@696d0b39{s=COMMITTED}]=>HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/3,age=13485}
Send of 3 failed: null (org.eclipse.jetty.io.EofException)
19:20:56.542 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannelState - complete HttpChannelState@3890f8d{s=ASYNC_WAIT a=STARTED i=false r=IDLE w=false}
19:20:56.542 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=ASYNC_WOKEN,uri=//localhost:8080/random/3,age=13485} handle //localhost:8080/random/3 
19:20:56.542 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannelState - handling HttpChannelState@3890f8d{s=ASYNC_WOKEN a=COMPLETE i=false r=IDLE w=false}
19:20:56.542 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@3fd554b0{r=3,c=true,c=false/false,a=COMPLETING,uri=//localhost:8080/random/3,age=13485} action COMPLETE
19:20:56.542 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannelState - onComplete HttpChannelState@3890f8d{s=COMPLETING a=COMPLETE i=false r=IDLE w=false}
19:20:56.542 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - COMPLETE for /random/3 written=154
19:20:56.542 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannelState - recycle HttpChannelState@3890f8d{s=COMPLETED a=NOT_ASYNC i=false r=IDLE w=false}
19:20:56.543 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.http.HttpParser - close HttpParser{s=END,0 of -1}
19:20:56.543 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.http.HttpParser - END --> CLOSE
19:20:56.543 [scuff.DefaultScheduler.7] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@3fd554b0{r=3,c=false,c=false/false,a=IDLE,uri=null,age=0} handle exit, result COMPLETE
19:20:56.876 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "4" to 4.293720580839897
19:20:57.211 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "8" to 8.303425371945819
19:20:57.545 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "4" to 4.262528716089454
19:20:57.879 [scuff.DefaultScheduler.7] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "9" to 9.56345647393559
19:20:58.212 [scuff.DefaultScheduler.1] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "2" to 2.933189162859997
19:20:58.212 [scuff.DefaultScheduler.1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=37,c=32768,r=37]={<<<data: {"value":2....9162859997}\r\n\r\n>>>\neta charset="utf...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:20:58.212 [scuff.DefaultScheduler.1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@3bccc9ca] generate: FLUSH (null,[p=0,l=37,c=32768,r=37],false)@COMMITTED
19:20:58.212 [scuff.DefaultScheduler.1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=37,c=32768,r=37]={<<<data: {"value":2....9162859997}\r\n\r\n>>>\neta charset="utf...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:20:58.212 [scuff.DefaultScheduler.1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:20:58.212 [scuff.DefaultScheduler.1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 37 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=2009/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=4196}
19:20:58.212 [scuff.DefaultScheduler.1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=37 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:20:58.212 [scuff.DefaultScheduler.1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:20:58.212 [scuff.DefaultScheduler.1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@3bccc9ca] generate: DONE (null,[p=37,l=37,c=32768,r=0],false)@COMMITTED
19:20:58.547 [scuff.DefaultScheduler.1] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "8" to 8.442157840963805
19:20:58.881 [scuff.DefaultScheduler.1] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "6" to 6.093339935773123
19:20:59.216 [scuff.DefaultScheduler.1] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "9" to 9.692938144529164
19:20:59.551 [scuff.DefaultScheduler.1] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "7" to 7.96541363725892
19:20:59.885 [scuff.DefaultScheduler.1] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT - test.eventsource.web.Main$$anon$1-6950e31: Updating "3" to 3.319576828083396
Send of 3 failed: AsyncContext completed and/or Request lifecycle recycled (java.lang.IllegalStateException: AsyncContext completed and/or Request lifecycle recycled)
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@21f1d5c3] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=5809/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=10005}
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@21f1d5c3] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@5595c92e] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=10005}
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:04.021 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@5595c92e] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@7b8cb2e3] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=10002/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=20007}
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@7b8cb2e3] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@cba89ab] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=20007}
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:14.023 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@cba89ab] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:23.844 [Scheduler-785992331] DEBUG org.eclipse.jetty.io.IdleTimeout - SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=-,to=9821/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=29828} idle timeout check, elapsed: 9821 ms, remaining: 20179 ms
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@234a0a32] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=10001/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=30008}
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@234a0a32] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@586d6bcc] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=30008}
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:24.024 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@586d6bcc] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@46e8c188] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=10001/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=40009}
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@46e8c188] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@1e47ef2f] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=40009}
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:34.025 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@1e47ef2f] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:44.025 [Scheduler-785992331] DEBUG org.eclipse.jetty.io.IdleTimeout - SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=-,to=10000/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=50009} idle timeout check, elapsed: 10000 ms, remaining: 20000 ms
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@6d2f6d4f] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\r>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=10002/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=50011}
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@6d2f6d4f] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=false committing=false callback=Blocker@79a66c61{null}
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@5eaf582d] generate: FLUSH (null,[p=0,l=1,c=32768,r=1],false)@COMMITTED
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - write: WriteFlusher@716b6e38{IDLE}->null [DirectByteBuffer@3a40f8d[p=0,l=1,c=32768,r=1]={<<<\n>>>ata: {"value":2.9...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}]
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{WRITING}->null:IDLE-->WRITING
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.ChannelEndPoint - flushed 1 SocketChannelEndPoint@5ac1bde9{/0:0:0:0:0:0:0:1:52821<->/0:0:0:0:0:0:0:1:8080,OPEN,fill=-,flush=W,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@2a55d598[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7dac1bce{s=COMMITTED}]=>HttpChannelOverHttp@36d171a7{r=3,c=true,c=false/false,a=ASYNC_WAIT,uri=//localhost:8080/random/2,age=50011}
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - Flushed=true written=1 remaining=0 WriteFlusher@716b6e38{WRITING}->null
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.io.WriteFlusher - update WriteFlusher@716b6e38{IDLE}->null:WRITING-->IDLE
19:21:44.027 [pool-1-thread-1] DEBUG org.eclipse.jetty.server.HttpConnection - org.eclipse.jetty.server.HttpConnection$SendCallback@763bd997[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@5eaf582d] generate: DONE (null,[p=1,l=1,c=32768,r=0],false)@COMMITTED

Back to the top