| "java.lang.OutOfMemoryError: Java heap space" while launching slave over ssh [message #905801] |
Fri, 31 August 2012 01:01  |
Eclipse User |
|
|
|
Hello,
I've been using this linux slave node for quite some time. But suddenly I get an OOM error when I attempt to launch the slave (see below). Notably, I have an identical slave that seems to be working just fine. Any thoughts or debugging tips?
Thanks,
Brian
[08/30/12 21:56:15] [SSH] Opening SSH connection to riyadh:22.
[08/30/12 21:56:15] [SSH] Authenticating as hudson/******.
[08/30/12 21:56:15] [SSH] Authentication successful.
[08/30/12 21:56:15] [SSH] The remote users environment is:
BASH=/bin/bash
BASHOPTS=cmdhist:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
BASH_ALIASES=()
BASH_ARGC=()
BASH_ARGV=()
BASH_CMDS=()
BASH_EXECUTION_STRING=set
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="4" [1]="2" [2]="24" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu")
BASH_VERSION='4.2.24(1)-release'
DIRSTACK=()
EUID=2002
GROUPS=()
HOME=/home/hudson
HOSTNAME=riyadh
HOSTTYPE=x86_64
IFS=$' \t\n'
LANG=en_US.UTF-8
LOGNAME=hudson
MACHTYPE=x86_64-pc-linux-gnu
MAIL=/var/mail/hudson
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
PIPESTATUS=([0]="0")
PPID=7066
PS4='+ '
PWD=/home/hudson
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:interactive-comments
SHLVL=1
SSH_CLIENT='169.229.130.76 48992 22'
SSH_CONNECTION='169.229.130.76 48992 169.229.130.90 22'
TERM=dumb
UID=2002
USER=hudson
XDG_SESSION_COOKIE=39710d5371497900693104820000032c-1346388888.763341-1638202249
_=']'
[08/30/12 21:56:15] [SSH] Checking java version of java
[08/30/12 21:56:15] [SSH] java -version returned 1.7.0_06.
[08/30/12 21:56:15] [SSH] Starting sftp client.
[08/30/12 21:56:15] [SSH] Copying latest slave.jar...
[08/30/12 21:56:15] [SSH] Copied 236,330 bytes.
[08/30/12 21:56:15] [SSH] Starting slave process: cd '/var/hudson' && java -jar slave.jar
<===[HUDSON REMOTING CAPACITY]===>java.io.IOException: Remote call on riyadh failed
at hudson.remoting.Channel.call(Channel.java:659)
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:326)
at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:383)
at hudson.plugins.sshslaves.SSHLauncher.launch(SSHLauncher.java:203)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:194)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.OutOfMemoryError: Java heap space
[08/30/12 21:56:16] [SSH] Connection closed.
ERROR: Connection terminated
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1030)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2570)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1024)
ERROR: [08/30/12 21:56:16] slave agent was terminated
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1030)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2570)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1024)
|
|
|
|
|
|
| Re: "java.lang.OutOfMemoryError: Java heap space" while launching slave over ssh [message #906247 is a reply to message #906170] |
Fri, 31 August 2012 19:37  |
| Eclipse User |
|
|
|
Hey guys,
Thanks for the prompt help. I think the root cause was that I was running OOM on the master, not the slave. After writing my last post, hudson started failing to load the full console with OOM, and my jobs appeared to be frozen. This all may have been because I recently increased the number of slaves that we're using, and the number of concurrent jobs. The console output from these jobs gets really big (tens of megs, if you can believe that). And I may have many users viewing many different consoles at a time. Anyway, the heap was at 128M. I cranked it up to 512M and restarted. I guess this doesn't explain why it worked for several weeks. Probably just had fewer users at that time.
Anyway, thanks for your help.
Ciao,
Brian
|
|
|
Powered by
FUDForum. Page generated in 0.04035 seconds
Home