Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Stopping a server that is servicing requests
OK, I left this sit for a bit and I am finally getting back to it.  I wanted to make sure that I don't have anything trying to write which it looks like I might have in the previous trace I sent.  My servlet works like this (I am sure there are better ways to do this but I am not sure it matters in this case):

1. service() registers an appender with log4j.  The appender writer is set to an OutputStreamWriter that wraps response.getOutputStream().
2. after the appender is registered, my servlet sleeps waking every so often to see if the client connection has dropped.
3. when the connection drops, the appender is removed from the log4j logger and closed.  Exceptions are thrown in this case but log4j handles them fine:

org.eclipse.jetty.io.EofException
     at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:892)
     at org.eclipse.jetty.server.HttpConnection.flushResponse(HttpConnection.java:742)
     at org.eclipse.jetty.server.HttpConnection$Output.close(HttpConnection.java:1080)
     at sun.nio.cs.StreamEncoder.implClose(Unknown Source)
     at sun.nio.cs.StreamEncoder.close(Unknown Source)
     at java.io.OutputStreamWriter.close(Unknown Source)
     at java.io.FilterWriter.close(Unknown Source)
     at org.apache.log4j.WriterAppender.closeWriter(WriterAppender.java:213)
     at org.apache.log4j.WriterAppender.reset(WriterAppender.java:337)
     at org.apache.log4j.WriterAppender.close(WriterAppender.java:204)
     at com.stoneware.swlogviewer.HtmlLogger.connect(HtmlLogger.java:79)
     at com.stoneware.swlogviewer.LogViewer.service(LogViewer.java:35)

Where I have the issue is if, between step 2 and 3, the client connection has dropped and I attempt to restart the container, I get the hung thread:

   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00007fcc7492ccb0> (a org.eclipse.jetty.io.nio.SelectChannelEndPoint)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.blockWritable(SelectChannelEndPoint.java:324)
    - locked <0x00007fcc7492ccb0> (a org.eclipse.jetty.io.nio.SelectChannelEndPoint)
    at org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:480)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:147)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:86)
    at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source)
    at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source)
    at sun.nio.cs.StreamEncoder.implFlush(Unknown Source)
    at sun.nio.cs.StreamEncoder.flush(Unknown Source)
    - locked <0x00007fcc7496c5c0> (a java.io.OutputStreamWriter)
    at java.io.OutputStreamWriter.flush(Unknown Source)
    at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:57)
    at org.apache.log4j.WriterAppender.writeFooter(WriterAppender.java:352)
    at org.apache.log4j.WriterAppender.close(WriterAppender.java:203)
    - locked <0x00007fcc74969ab8> (a org.apache.log4j.WriterAppender)
    at com.stoneware.swlogviewer.HtmlLogger.connect(HtmlLogger.java:79)
    at com.stoneware.swlogviewer.LogViewer.service(LogViewer.java:35)

It appears the log4j code should fail if it gets the exception but it is locked in a wait() in SelectChannelEndPoint.  The only time I have seen this behavior is when I am shutting the container down.  It seems like SelectChannelEndPoint is not sequencing things down correctly if a client connection has dropped and things are shutting down but I am guessing (obviously).  Does any of this makes sense or am I still doing something stupid (please be kind!)??

Thanks
Tony


Greg Wilkins wrote: 
Tony,

the server does wack all the threads during the stop.  That is
what causes the InterrupedException.     Something in your servlet
must be in a loop and continues to try to write.

Jetty notices that the thread is not stopping and logs that fact.

I think we also close all the connections, so there should also be
IOExceptions... but if you use a writer, most people don't call
checkError - so the exception can be ignored.


cheers



Tony Thompson wrote:
  
I am using Jetty 7.0.2 snapshot code.  In a web application, I have a
servlet that is sending content back to a client (as most servlets do). 
If I stop the Jetty instance while the servlet is busy doing the
"generating content for a client" thing, I get this exception:

java.lang.InterruptedException
    at java.lang.Object.wait(Native Method)
    at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.blockWritable(SelectChannelEndPoint.java:324)
    at
org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:480)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:147)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:98)
    at com.stoneware.servlets.HtmlLogger.write(HtmlLogger.java:133)
    at com.stoneware.servlets.HtmlLogger.writeln(HtmlLogger.java:142)
    at com.stoneware.servlets.HtmlLogger.connect(HtmlLogger.java:89)
    at com.stoneware.swconsole.SWLog.service(SWLog.java:35)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:530)
....

and then this message:  1 threads could not be stopped

Of course, the thread is still hanging out after the stop has finished
which, in my case, is causing a classloader leak:

"qtp165789330-15" prio=10 tid=0x0000000041c9e800 nid=0x7f24 in
Object.wait() [0x00007f1fb79fb000..0x00007f1fb79fcb70]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00007f1fc34885d0> (a
org.eclipse.jetty.io.nio.SelectChannelEndPoint)
    at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.blockWritable(SelectChannelEndPoint.java:324)
    - locked <0x00007f1fc34885d0> (a
org.eclipse.jetty.io.nio.SelectChannelEndPoint)
    at
org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:480)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:147)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:98)
    at com.stoneware.servlets.HtmlLogger.write(HtmlLogger.java:133)
    at com.stoneware.servlets.HtmlLogger.writeln(HtmlLogger.java:142)
    at com.stoneware.servlets.HtmlLogger.connect(HtmlLogger.java:89)
    at com.stoneware.swconsole.SWLog.service(SWLog.java:35)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:530)
....

Since I am stopping the Jetty instance, shouldn't it whack all of those
threads that are busy servicing clients at the time?

Thanks
Tony
 
This message (and any associated files) is intended only for the
use of the individual or entity to which it is addressed and may
contain information that is confidential, subject to copyright or
constitutes a trade secret. If you are not the intended recipient
you are hereby notified that any dissemination, copying or
distribution of this message, or files associated with this message,
is strictly prohibited. If you have received this message in error,
please notify us immediately by replying to the message and deleting
it from your computer. Messages sent to and from Stoneware, Inc.
may be monitored.


------------------------------------------------------------------------

_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jetty-users
    
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jetty-users
 
This message (and any associated files) is intended only for the
use of the individual or entity to which it is addressed and may
contain information that is confidential, subject to copyright or
constitutes a trade secret. If you are not the intended recipient
you are hereby notified that any dissemination, copying or
distribution of this message, or files associated with this message,
is strictly prohibited. If you have received this message in error,
please notify us immediately by replying to the message and deleting
it from your computer. Messages sent to and from Stoneware, Inc.
may be monitored.

Back to the top