Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Hudson » Running out of memory - Perm Size(Hudson 3.0.1 - running out of memory.)
Running out of memory - Perm Size [message #1071305] Fri, 19 July 2013 19:45 Go to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
I keep increasing my memory settings for Perm Size.

I'm now running at -XX:PermSize=3585 -XX:MaxPermSize=6144m

I will have to see if that fails over the next few hours but I've run at:

-XX:PermSize=2048 -XX:MaxPermSize=4096m and I run out.

This is after only about 2 hours.

At the time I was running 10 - 12 jobs on two different slaves.

I was also navigating - a lot - through the browser to check all of my jobs for junit test results - maybe 10.

I've also disabled all unneeded plugins. I think that helped but I still run out.

Are there any known memory issues with 3.0.1? I haven't seen this problem with 2.2.1.
Re: Running out of memory - Perm Size [message #1071330 is a reply to message #1071305] Fri, 19 July 2013 21:18 Go to previous messageGo to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
This ran out of memory with my higher memory settings:

-XX:PermSize=3585 -XX:MaxPermSize=6144m

The error I got this time is:

java.lang.OutOfMemoryError: PermGen space
Exception in thread "Channel reader thread: SLAVE_NAME" java.lang.OutOfMemoryError: PermGen space.
Re: Running out of memory - Perm Size [message #1071410 is a reply to message #1071330] Sat, 20 July 2013 02:37 Go to previous messageGo to next message
Winston Prakash is currently offline Winston PrakashFriend
Messages: 434
Registered: August 2011
Location: Fremont, CA USA
Senior Member
For running Hudson you don't need permspace more than 128M or 256M. If you have to use more than -XX:MaxPermSize=256m, something else is going wrong here. PermSpace is used to store Java Classes & some String and static data. So need for more than 256m usually don't arise.



Winston Prakash
Eclipse Hudson team
Re: Running out of memory - Perm Size [message #1071429 is a reply to message #1071410] Sat, 20 July 2013 04:07 Go to previous messageGo to next message
Winston Prakash is currently offline Winston PrakashFriend
Messages: 434
Registered: August 2011
Location: Fremont, CA USA
Senior Member
Also, I guess, you are getting permGen error in slave. So adding to Hudson doesn't help. You might want to set in the Slave Configuraion. Set it in Launch Method -> Advanced -> JVM Options (This is for SSH slaves)

Winston Prakash
Eclipse Hudson team
Re: Running out of memory - Perm Size [message #1071882 is a reply to message #1071429] Sun, 21 July 2013 13:01 Go to previous messageGo to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
I'm running all of my slaves via the Java web start option.

Also, I do see the error with memory reported from the slave. I think I usually don't see this problem related to a slave however I will make note of this.

If this a problem with the slave why does the browser not connect to Hudson at all?

I try xxxxx:8080 with multiple browsers and I get no response until I restart.
Re: Running out of memory - Perm Size [message #1071907 is a reply to message #1071882] Sun, 21 July 2013 14:48 Go to previous messageGo to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
I was also running the same load on the same slave with 2.2.1 without this memory problem.
Re: Running out of memory - Perm Size [message #1072533 is a reply to message #1071907] Tue, 23 July 2013 01:13 Go to previous messageGo to next message
Winston Prakash is currently offline Winston PrakashFriend
Messages: 434
Registered: August 2011
Location: Fremont, CA USA
Senior Member
Very weird. http://ci.hudson-ci.org/ is running Hudson 3.0 and we don't see the issue.
What is the exact memory settings you have and the configuration of your machine, such as amount of physical memory etc.

BTW, don't specify -XX:PermSize=3585


Winston Prakash
Eclipse Hudson team
Re: Running out of memory - Perm Size [message #1072737 is a reply to message #1072533] Tue, 23 July 2013 11:59 Go to previous messageGo to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
My VM is running with 8 gb of RAM.

The latest memory settings I have are:

-XmX4096m -XX:MaxPermSize=1024m

So I'm actually running this:

java -DHUDSON_HOME=/home/e-team/hudson -DHUDSON_BUILDS=/home/e-team/hudson/data/builds -jar /home/e-team/hudson/hudson-3.0.1.war -Xmx4096m -XX:MaxPermSize=1024m

(Notice that I've now removed the PermSize setting.

The reason I'm running the way I am is that I need to specify HUDSON_BUILDS as it's a shared folder from a VirtualBox VM.

Also, I notice from the link provided in the previous post that there are maybe 20 jobs defined.

We have about 80 defined.

I'm also noticing, I believe, that this problem is most often occurring when the browser is on "refresh enabled".

Also, there are no jobs running on the master. They're all running on slaves.

One slave is another VM on the same machine. The other(s) are standalone machines.

I measured the used memory on my 8 gb Hudson VM and it's using less than 2 gb of RAM so I have 6 free.

I've also changed the .jnlp files on my slaves to add memory settings for the heap size after the java version:

<j2se version="1.5+" initial-heap-size="256m" max-heap-size="1024m"/>

[Updated on: Tue, 23 July 2013 18:22]

Report message to a moderator

Re: Running out of memory - Perm Size [message #1072892 is a reply to message #1072737] Tue, 23 July 2013 18:23 Go to previous messageGo to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
These settings did not help.

I then tried to do the configuration through the slave configuration.

I changed the jnlp java options with various values. This is the last one:

-XX:PermSize=512m -XX:MaxPermSize=1536m
Re: Running out of memory - Perm Size [message #1072934 is a reply to message #1072892] Tue, 23 July 2013 19:58 Go to previous messageGo to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
2013-07-13 15:56:25.889:WARN:oejs.AbstractHttpConnection:/
java.lang.OutOfMemoryError: PermGen space

Latest memory error.
Re: Running out of memory - Perm Size [message #1072935 is a reply to message #1072934] Tue, 23 July 2013 20:00 Go to previous messageGo to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
It's possible this is also happening when I cancel a job.
Re: Running out of memory - Perm Size [message #1073881 is a reply to message #1072935] Thu, 25 July 2013 17:37 Go to previous messageGo to next message
Winston Prakash is currently offline Winston PrakashFriend
Messages: 434
Registered: August 2011
Location: Fremont, CA USA
Senior Member
Hi Stuart, let us take this discussion offline. When we have solution we can post it here. Write to me directly at winston DOT prakash At gmail DOT com

Winston Prakash
Eclipse Hudson team
Re: Running out of memory - Perm Size [message #1074249 is a reply to message #1071305] Fri, 26 July 2013 11:16 Go to previous messageGo to next message
Stuart Lorber is currently offline Stuart LorberFriend
Messages: 44
Registered: June 2013
Member
I believe this was my mistake.

I was starting Hudson from the command line with an incorrect order of parameters.

I've fixed this order and seem to be running fine for 2 days with the same loads I was running with 2.2.1.

I'm assuming this was my mistake and not a problem with the product.
Re: Running out of memory - Perm Size [message #1074489 is a reply to message #1074249] Fri, 26 July 2013 23:16 Go to previous message
Winston Prakash is currently offline Winston PrakashFriend
Messages: 434
Registered: August 2011
Location: Fremont, CA USA
Senior Member
Glad to hear, you got it working. FYI.. we are working on performance improvements for 3.1.0. We are looking at reducing the over all memory footprints by several folds, especially in large installations with several hundreds of jobs and several tens of thousands of builds. 3.10 will be released soon.

Winston Prakash
Eclipse Hudson team
Previous Topic:A very simple question
Next Topic:authentication cancelled
Goto Forum:
  


Current Time: Thu Nov 27 00:28:00 GMT 2014

Powered by FUDForum. Page generated in 0.05856 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software