[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [jetty-users] Jetty 7 memory leak
|
Tony,
what needs to be done is to run the SslSelectChannelEndPoint on it's
own and stream data through it endlessly and see if the problem can be
reproduced in a simpler test case.
We have an EndPointTest that could be used as the basis of this.
Could you have a stab at creating this test? It will probably be next
week before we can get somebody onto this.
cheers
On 23 September 2010 12:30, Tony Thompson <Tony.Thompson@xxxxxxxxxxxxxx> wrote:
> ok, we have been running with Jetty 7.2 for about 18 hours now and this is
> what I have found. The memory leak still appears to be there but, it is
> different. It appears that maybe Jetty is not holding on to anything
> anymore and possibly it has shifted to a bug in the JVM (?). We have been
> comparing heap dumps today and the retained byte arrays do continue to grow
> with a GC root that looks like this:
>
> +--------------------------------------------------------------------------------------
> |
> +--------------------------------------------------------------------------------------
> |
> +---byte[16665]
> |
> |
> | +---buf of
> com.sun.net.ssl.internal.ssl.InputRecord
> |
> |
> | +---r of
> com.sun.net.ssl.internal.ssl.AppInputStream
> |
> |
> | +---input of
> com.sun.net.ssl.internal.ssl.SSLSocketImpl
> |
> |
> | +---referent of
> java.lang.ref.Finalizer
> |
> |
> | +---prev of
> java.lang.ref.Finalizer
> |
> |
> | +---prev of
> java.lang.ref.Finalizer
> |
> |
> | +---prev of
> java.lang.ref.Finalizer
> (and on like that...truncated for brevity)
>
> We are also still experiencing high CPU utilization but it seems to be
> better as well. When I say better, I just mean the server has been usable.
> Before it would get to a point where it wouldn't respond to requests any
> more.
>
> Any suggestions at this point?
> Tony
>
> Jan Bartel wrote:
>
> Tony,
>
> We're getting read to ship 7.2 out the door. It would be great if you
> could try the latest 7.2-SNAPSHOT out:
>
> https://oss.sonatype.org/content/groups/jetty-with-staging/org/eclipse/jetty/jetty-distribution/7.2.0-SNAPSHOT/jetty-distribution-7.2.0-20100922.104837-6.zip
>
> thanks,
> Jan
>
> On 23/09/10 08:55, Tony Thompson wrote:
>
> This particular issue happens with both 7.1.4 and 7.1.6. We were
> attempting to upgrade to 7.1.6 to see if it fixed the issue but it has
> not. Of course, you saw my other post with the high utilization issues
> so it has been a battle.
>
> Tony
>
> Jan Bartel wrote:
>
> Tony,
>
> From your other posts, you seem to be upgrading to 7.1.6 - can
> you confirm whether this is happening also on 7.1.6 or only 7.1.4?
>
> thanks
> Jan
>
> On 23/09/10 01:08, Tony Thompson wrote:
>
> I am also seeing this same issue with the 1.6.0_21 JVM on a 32 bit Linux
> server so it is not because of the older JVM. I still need some help
> with this one.
>
> Thanks
> Tony
>
> Tony Thompson wrote:
>
> I am running Jetty 7.1.4.v20100610 on a Linux server with this JVM:
> Java HotSpot(TM) 64-Bit Server VM; 1.6.0; 1.6.0-b105; mixed mode
>
> The server will run for a period of time and then run out of memory
> and crash. I have bumped the number of file handles so the server can
> run longer and I
> could get some heap dumps to compare. What I have found is literally
> millions of byte arrays hanging around in memory. As you can see
> below, it appears to
> be related to SSL. I know I am on an older JVM so it could be a JVM
> issue but, I found this which could indicate that maybe Jetty is not
> shutting things
> down properly:
>
> http://stackoverflow.com/questions/1623676/suns-java-ssl-implementation-is-leaking-memory
>
>
>
> Not quite my exact situation but similar. Can anyone provide some
> insight?
>
> Hoping the server can last until a reboot tonight....
> Tony
>
>
>
>
> Paths from "GC Roots" to "Object byte[] #00002AAAB9C201E0"
>
> +-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+
>
>
> | Name | Retained Size | Shallow Size |
> +-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+
>
>
> | +---byte[16665] | 16,688 | 16,688 |
> | | | | |
> | +---buf of com.sun.net.ssl.internal.ssl.EngineInputRecord | 17,792 |
> 96 |
> | | | | |
> | +---inputRecord of com.sun.net.ssl.internal.ssl.SSLEngineImpl |
> 29,496 | 192 |
> | | | | |
> | +---_engine of org.eclipse.jetty.io.nio.SslSelectChannelEndPoint |
> 53,568 | 184 |
> | | | | |
> | +---attachment of sun.nio.ch.SelectionKeyImpl | 64 | 64 |
> | | | | |
> | +---value of java.util.HashMap$Entry | 72 | 48 |
> | | | | |
> | +---[506] of java.util.HashMap$Entry[1024] | 50,696 | 8,216 |
> | | | | |
> | +---table of java.util.HashMap | 50,760 | 64 |
> | | | | |
> | +---fdToKey of sun.nio.ch.EPollSelectorImpl [Stack Local] | 51,184 |
> 120 |
> | | | | |
> | +---<local variable> of java.lang.Thread [Stack Local, Thread]
> "qtp1891328597-44 - Acceptor0 c@xxxxxxxxxxxxxxxxxxxxxxxxxx:443" |
> 1,216 | 168 |
> +-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+
>
>
>
> This message (and any associated files) is intended only for the use
> of the individual or entity to which it is addressed and may contain
> information that is confidential, subject to copyright or
> constitutes a trade secret. If you are not the intended recipient you
> are hereby notified that any dissemination, copying or distribution of
> this message, or files associated with this message, is strictly
> prohibited. If you have received this message in error, please notify
> us immediately by replying to the message and deleting it from your
> computer. Messages sent to and from Stoneware, Inc.
> may be monitored.
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
> This message (and any associated files) is intended only for the use of
> the individual or entity to which it is addressed and may contain
> information that is confidential, subject to copyright or
> constitutes a trade secret. If you are not the intended recipient you
> are hereby notified that any dissemination, copying or distribution of
> this message, or files associated with this message, is strictly
> prohibited. If you have received this message in error, please notify us
> immediately by replying to the message and deleting it from your
> computer. Messages sent to and from Stoneware, Inc.
> may be monitored.
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
> This message (and any associated files) is intended only for the
> use of the individual or entity to which it is addressed and may
> contain information that is confidential, subject to copyright or
> constitutes a trade secret. If you are not the intended recipient
> you are hereby notified that any dissemination, copying or
> distribution of this message, or files associated with this message,
> is strictly prohibited. If you have received this message in error,
> please notify us immediately by replying to the message and deleting
> it from your computer. Messages sent to and from Stoneware, Inc.
> may be monitored.
>
>
>
> This message (and any associated files) is intended only for the
> use of the individual or entity to which it is addressed and may
> contain information that is confidential, subject to copyright or
> constitutes a trade secret. If you are not the intended recipient
> you are hereby notified that any dissemination, copying or
> distribution of this message, or files associated with this message,
> is strictly prohibited. If you have received this message in error,
> please notify us immediately by replying to the message and deleting
> it from your computer. Messages sent to and from Stoneware, Inc.
> may be monitored.
>
>
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
>
