Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] tycho-gmp.gmf.tooling builds consumes enormous amount of memory



On 2/22/12 12:55 PM, Matthias Sohn wrote:
2012/2/22 Winston Prakash <winston.prakash@xxxxxxxxx>
Denis,

Right now I'm not talking about memory leak, but memory consumption.

Hudson has a tendency to keep the info about entire artifacts of a build. For example, if a job contains JUnit test results then this info is kept in the memory. Currently Hudson is holding about 360 jobs with 6500 builds and 510,000 JUnit Case results in the memory. Average memory consumed by each JUnit Test case is about 200kb. So net memory consumption by JUnit results should be about 100 MB, however memory profiler reports 250 MB of memory occupied by Test cases.
 
why is Hudson keeping all that in memory ?

Looks like the original authors developed Hudson that way. May be at that time the authors did not envision Hudson to be so successful and  become an enterprise tool :-)

Looks like it should instead use a LRU cache 
with a fixed max cache size.
Yes, that would be the right architecture for a large enterprise installation. I will look in to that.

 
I noticed that the job hudson-test-harness (which is maintained by me) has the  debug turned on. Because of this the Junit Test standard outputs has about 15-20 MB of debug results and this entire result was in the memory. Just 10 JUnit Case results occupy about 150-200 MB of memory. Now I turned off the debug. However, old Junit test results are still in the memory. So next time when Hudson restarts it should consume less memory (about 150 MB less). I noticed some other jobs too have large JUnit standard output. Reducing those output should reduce over all Hudson memory consumption. I will try to list those jobs. The ideal solution would be to fix in Hudson to load JUnit Test results  lazily, but that is a long term solution.

Similarly, the job "tycho-gmp.gmf.tooling" is producing huge amount of maven artifacts which are also kept in the memory (about 128 MB). I'm not sure exactly what it is, could be JavaDoc as pointed out by Mickael. I'm trying to find from tycho team if we could reduce those maven artifacts in the build (temporarily at least), until we find proper solution in Hudson itself.
 
does it keep all the maven artifacts in memory or some metadata about these artifacts ?

I think it keeps only the metadata. But something is going weird, I'm trying to figure that out with the maven folks.
 
Of course we need to fix the memory leaks, however the quick solution is to reduce these huge memory footprints for now in a easy way.

could we throw more memory on Hudson to workaround these problems until
Hudson became smarter about caching data ?
Currently 2.5 GB of memory is set aside for the Hudson JVM. If we are able to get rid of these huge memory consumers, I guess that amount should be ok.

- Winston
 
--
Matthias


_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev

Back to the top