Re: [jetty-users] Help with Jetty crash log

[Benson Margulies <bimargulies@xxxxxxxxx>]

It is perhaps worth noting that this sort of crash is, ipso facto, a
defect in the JRE. At worst you should have received an OutOfMemory.
So, upgrading to a newer JVM is always worth a shot. Or complaining to
oracle.

On Mon, Apr 4, 2011 at 7:12 AM, Thomas Becker <tbecker@xxxxxxxxxxx> wrote:
> HI Matthieu,
>
> do you have any more log information from before that crash? It looks like
> it happened during a young generation garbage collection. Your eden space
> (where all new objects are placed in memory) was full and there was not much
> space (probably not enough) left in the old generation for objects which
> survived the young generation gc. The interesting part of your crash dump
> indicating this, is:
>
> Heap
>  PSYoungGen      total 417088K, used 417056K [0x00007f04eb080000,
> 0x00007f050a3f0000, 0x00007f0515b20000)
>   eden space 416960K, 100% used
> [0x00007f04eb080000,0x00007f05047b0000,0x00007f05047b0000)
>   from space 128K, 75% used
> [0x00007f05055d0000,0x00007f05055e8000,0x00007f05055f0000)
>   to   space 47232K, 0% used
> [0x00007f05075d0000,0x00007f05075e8000,0x00007f050a3f0000)
>  PSOldGen        total 529920K, used 377402K [0x00007f0495b20000,
> 0x00007f04b60a0000, 0x00007f04eb080000)
>   object space 529920K, 71% used
> [0x00007f0495b20000,0x00007f04acbae9b0,0x00007f04b60a0000)
>  PSPermGen       total 21504K, used 20440K [0x00007f048b320000,
> 0x00007f048c820000, 0x00007f0495b20000)
>   object space 21504K, 95% used
> [0x00007f048b320000,0x00007f048c7163a8,0x00007f048c820000)
>
> As solr/lucene usage can be quite memory intensive it's likely that your app
> simply ran out of memory. Maybe during indexing. So your next steps will be
> to find out what the application was doing before the crash and think about
> what might have caused it to run oom. If you can't find the root cause this
> way, you should try to get a heap dump and analyze what was consuming
> memory.
>
> Try setting these JVM options
> (http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html)
> in your startup script:
>
> -XX:HeapDumpPath=./java_pid<pid>.hprof Path to directory or filename for
> heap dump. Manageable. (Introduced in 1.4.2 update 12, 5.0 update 7.)
> -XX:-HeapDumpOnOutOfMemoryError Dump heap to file when
> java.lang.OutOfMemoryError is thrown. Manageable. (Introduced in 1.4.2
> update 12, 5.0 update 7.)
> -XX:-PrintGCDetails Print more details at garbage collection. Manageable.
> (Introduced in 1.4.0.) -XX:-PrintGCTimeStamps Print timestamps at garbage
> collection. Manageable (Introduced in 1.4.0.)
>
> Again it might be normal and that your application simply needs a bigger
> heap. Your current settings are: "jvm_args: -Xms512M -Xmx2048M". If you
> don't have more memory available you might need to rethink your indexing
> strategy (if the oom happened during indexing). But it might as well be that
> you've got a memory leak of some kind. You'll have to read your log files
> and try to get a heap dump.
> As indexing is memory intensive, proper tuning of your jvm memory settings
> is crucial. Normally you want a big newsize and the overall heap space as
> big as you have physical memory. But that heavily depends on your indexing
> strategy.
>
> Hope I did not confuse you too much. Please not that this is most certainly
> NOT a jetty issue.
>
> Cheers,
> Thomas
>
> On 04/04/2011 12:20, Matthieu Huin wrote:
>
> Greetings all,
>
> I am currently using solr as the backend behind a log aggregation and
> search system my team is developing. All was well and good until I
> noticed a test server crashing quite unexpectedly. We'd like to dig more
> into the incident but none of us has much experience with Jetty crash
> logs - not to mention that our Java is very rusty.
>
> The crash log is joined as an attachment.
>
> Could anyone help us with understanding what went wrong there ?
>
> Also, would it be possible and/or wise to automatically restart the
> server in case of such a crash ?
>
>
> Thanks for your help. If you need any extra info about that case, do not
> hesitate to ask !
>
>
> Matthieu Huin
>
> ( Resending this message as I used to wrong e-mail to send it, sorry for the
> inconvenience)
>
>
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
>