Eclipse Community Forums: Hudson » Running out of memory

Help

Home

Home » Archived » Hudson » Running out of memory - Perm Size(Hudson 3.0.1 - running out of memory.)

Show: Today's Messages :: Show Polls :: Message Navigator

Running out of memory - Perm Size [message #1071305]

Fri, 19 July 2013 19:45

Stuart Lorber

Messages: 45
Registered: June 2013

Member

I keep increasing my memory settings for Perm Size.

I'm now running at -XX:PermSize=3585 -XX:MaxPermSize=6144m

I will have to see if that fails over the next few hours but I've run at:

-XX:PermSize=2048 -XX:MaxPermSize=4096m and I run out.

This is after only about 2 hours.

At the time I was running 10 - 12 jobs on two different slaves.

I was also navigating - a lot - through the browser to check all of my jobs for junit test results - maybe 10.

I've also disabled all unneeded plugins. I think that helped but I still run out.

Are there any known memory issues with 3.0.1? I haven't seen this problem with 2.2.1.

Report message to a moderator

Re: Running out of memory - Perm Size [message #1071330 is a reply to message #1071305]

Fri, 19 July 2013 21:18

Stuart Lorber

Messages: 45
Registered: June 2013

Member

This ran out of memory with my higher memory settings:

-XX:PermSize=3585 -XX:MaxPermSize=6144m

The error I got this time is:

java.lang.OutOfMemoryError: PermGen space
Exception in thread "Channel reader thread: SLAVE_NAME" java.lang.OutOfMemoryError: PermGen space.

Report message to a moderator

Re: Running out of memory - Perm Size [message #1071410 is a reply to message #1071330]

Sat, 20 July 2013 02:37

Winston Prakash

Messages: 534
Registered: August 2011
Location: Fremont, CA USA

Senior Member

For running Hudson you don't need permspace more than 128M or 256M. If you have to use more than -XX:MaxPermSize=256m, something else is going wrong here. PermSpace is used to store Java Classes & some String and static data. So need for more than 256m usually don't arise.

Winston Prakash
Eclipse Hudson team

Report message to a moderator

Re: Running out of memory - Perm Size [message #1071429 is a reply to message #1071410]

Sat, 20 July 2013 04:07

Winston Prakash

Messages: 534
Registered: August 2011
Location: Fremont, CA USA

Senior Member

Also, I guess, you are getting permGen error in slave. So adding to Hudson doesn't help. You might want to set in the Slave Configuraion. Set it in Launch Method -> Advanced -> JVM Options (This is for SSH slaves)

Winston Prakash
Eclipse Hudson team

Report message to a moderator

Re: Running out of memory - Perm Size [message #1071882 is a reply to message #1071429]

Sun, 21 July 2013 13:01

Stuart Lorber

Messages: 45
Registered: June 2013

Member

I'm running all of my slaves via the Java web start option.

Also, I do see the error with memory reported from the slave. I think I usually don't see this problem related to a slave however I will make note of this.

If this a problem with the slave why does the browser not connect to Hudson at all?

I try xxxxx:8080 with multiple browsers and I get no response until I restart.

Report message to a moderator

Re: Running out of memory - Perm Size [message #1071907 is a reply to message #1071882]

Sun, 21 July 2013 14:48

Stuart Lorber

Messages: 45
Registered: June 2013

Member

I was also running the same load on the same slave with 2.2.1 without this memory problem.

Report message to a moderator

Re: Running out of memory - Perm Size [message #1072533 is a reply to message #1071907]

Tue, 23 July 2013 01:13

Winston Prakash

Messages: 534
Registered: August 2011
Location: Fremont, CA USA

Senior Member

Very weird. http://ci.hudson-ci.org/ is running Hudson 3.0 and we don't see the issue.
What is the exact memory settings you have and the configuration of your machine, such as amount of physical memory etc.

BTW, don't specify -XX:PermSize=3585

Winston Prakash
Eclipse Hudson team

Report message to a moderator

Re: Running out of memory - Perm Size [message #1072737 is a reply to message #1072533]

Tue, 23 July 2013 11:59

Stuart Lorber

Messages: 45
Registered: June 2013

Member

My VM is running with 8 gb of RAM.

The latest memory settings I have are:

-XmX4096m -XX:MaxPermSize=1024m

So I'm actually running this:

java -DHUDSON_HOME=/home/e-team/hudson -DHUDSON_BUILDS=/home/e-team/hudson/data/builds -jar /home/e-team/hudson/hudson-3.0.1.war -Xmx4096m -XX:MaxPermSize=1024m

(Notice that I've now removed the PermSize setting.

The reason I'm running the way I am is that I need to specify HUDSON_BUILDS as it's a shared folder from a VirtualBox VM.

Also, I notice from the link provided in the previous post that there are maybe 20 jobs defined.

We have about 80 defined.

I'm also noticing, I believe, that this problem is most often occurring when the browser is on "refresh enabled".

Also, there are no jobs running on the master. They're all running on slaves.

One slave is another VM on the same machine. The other(s) are standalone machines.

I measured the used memory on my 8 gb Hudson VM and it's using less than 2 gb of RAM so I have 6 free.

I've also changed the .jnlp files on my slaves to add memory settings for the heap size after the java version:

<j2se version="1.5+" initial-heap-size="256m" max-heap-size="1024m"/>

[Updated on: Tue, 23 July 2013 18:22]

Report message to a moderator

Re: Running out of memory - Perm Size [message #1072892 is a reply to message #1072737]

Tue, 23 July 2013 18:23

Stuart Lorber

Messages: 45
Registered: June 2013

Member

These settings did not help.

I then tried to do the configuration through the slave configuration.

I changed the jnlp java options with various values. This is the last one:

-XX:PermSize=512m -XX:MaxPermSize=1536m

Report message to a moderator

Re: Running out of memory - Perm Size [message #1072934 is a reply to message #1072892]

Tue, 23 July 2013 19:58

Stuart Lorber

Messages: 45
Registered: June 2013

Member

2013-07-13 15:56:25.889:WARN:oejs.AbstractHttpConnection:/
java.lang.OutOfMemoryError: PermGen space

Latest memory error.

Report message to a moderator

Re: Running out of memory - Perm Size [message #1072935 is a reply to message #1072934]

Tue, 23 July 2013 20:00

Stuart Lorber

Messages: 45
Registered: June 2013

Member

It's possible this is also happening when I cancel a job.

Report message to a moderator

Re: Running out of memory - Perm Size [message #1073881 is a reply to message #1072935]

Thu, 25 July 2013 17:37

Winston Prakash

Messages: 534
Registered: August 2011
Location: Fremont, CA USA

Senior Member

Hi Stuart, let us take this discussion offline. When we have solution we can post it here. Write to me directly at winston DOT prakash At gmail DOT com

Winston Prakash
Eclipse Hudson team

Report message to a moderator

Re: Running out of memory - Perm Size [message #1074249 is a reply to message #1071305]

Fri, 26 July 2013 11:16

Stuart Lorber

Messages: 45
Registered: June 2013

Member

I believe this was my mistake.

I was starting Hudson from the command line with an incorrect order of parameters.

I've fixed this order and seem to be running fine for 2 days with the same loads I was running with 2.2.1.

I'm assuming this was my mistake and not a problem with the product.

Report message to a moderator

Re: Running out of memory - Perm Size [message #1074489 is a reply to message #1074249]

Fri, 26 July 2013 23:16

Winston Prakash

Messages: 534
Registered: August 2011
Location: Fremont, CA USA

Senior Member

Glad to hear, you got it working. FYI.. we are working on performance improvements for 3.1.0. We are looking at reducing the over all memory footprints by several folds, especially in large installations with several hundreds of jobs and several tens of thousands of builds. 3.10 will be released soon.

Winston Prakash
Eclipse Hudson team

Report message to a moderator

Previous Topic:	A very simple question
Next Topic:	authentication cancelled

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 19 21:46:37 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter