Eclipse Community Forums: Hudson » Stopping Maven3 build step leaves running processes

Help

Home

Home » Archived » Hudson » Stopping Maven3 build step leaves running processes

Show: Today's Messages :: Show Polls :: Message Navigator

Stopping Maven3 build step leaves running processes [message #1408203]

Thu, 14 August 2014 13:37

Ivan Inozemtsev

Messages: 0
Registered: January 2015

During our maven build we use a custom maven plugin which spawns external process, which executes UI tests for an application. Our maven plugin uses runtime shutdown hooks to gracefully terminate test runner and application-under-test.

Today I found that when I terminate a build, shutdown hooks are not executed, moreover all processes launched from maven plugin continue running.

What's interesting, if I use shell build step, which just invokes maven, everything works correctly -- shutdown hooks work and all processes exit.

I've prepared a tiny demo at github.com/komaz/test-cancel.
- run.sh just prints a message to stdout and a file every second during one minute.
- pom.xml uses exec-maven-plugin to launch this shell.

Here I have two jobs (I gave anonymous access to read configuration and run/stop build):
- https://hudson.eclipse.org/rcptt/view/test-cancel/job/direct-maven-invoke/ -- runs maven via Maven 3 build step. When I start build and then stop it, I can go to workspace and see that external process is not stopped (file is being appended -- https://hudson.eclipse.org/rcptt/view/test-cancel/job/direct-maven-invoke/ws/out.txt).
- https://hudson.eclipse.org/rcptt/view/test-cancel/job/shell-maven-invoke/ -- runs maven via shell build step. When I start and stop it, child process is killed correctly.

Report message to a moderator

Re: Stopping Maven3 build step leaves running processes [message #1408697 is a reply to message #1408203]

Fri, 15 August 2014 20:50

Winston Prakash

Messages: 534
Registered: August 2011
Location: Fremont, CA USA

Senior Member

When an external process is spawned from Hudson, all the internal Hudson environment variables are passed to it. That is how Hudson knows the process was started by Hudson. So when a Job Completes or killed Hudson finds all external processes it started when the job was building and forcefully kills them to avoid any zombie process left behind.

Your observation looks like a bug in maven3 builder plugin. Means it is not passing all the internal Hudson variables to the process it spawns. The Shell Script builder seems to be doing the right thing.

We will investigate the issue.

Winston Prakash
Eclipse Hudson team

Report message to a moderator

Re: Stopping Maven3 build step leaves running processes [message #1409819 is a reply to message #1408697]

Tue, 19 August 2014 04:09

Ivan Inozemtsev

Messages: 0
Registered: January 2015

Thank you for a reply, Winston. I slightly modified an example so that launched script dumps environment variables to a file, and found that in both cases variables HUDSON_COOKIE and HUDSON_SERVER_COOKIE are set, so issue must be in something else. What's also interesting, conlose output for maven3 build plugin (https://hudson.eclipse.org/rcptt/view/test-cancel/job/direct-maven-invoke/6/console) shows an error, which is not displayed when maven is invoked via shell command: [ERROR] Failure: java.lang.InterruptedException.

I launched Hudson locally and found that when I cancel a shell-maven-invoke, I see the following in Hudson console:

Aug 19, 2014 11:04:55 AM hudson.util.ProcessTree killAll
INFO: killAll (process started by build): process=java.lang.UNIXProcess@25c0b221 and envs={HUDSON_COOKIE=f2bdc2d5-51e4-4161-b2bf-acb8bae931f0}
Aug 19, 2014 11:04:55 AM hudson.model.Run run
INFO: shell-maven-invoke #2 aborted
java.lang.InterruptedException
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:503)
	at java.lang.UNIXProcess.waitFor(UNIXProcess.java:210)
	at hudson.Proc$LocalProc.join(Proc.java:222)
	at hudson.Launcher$ProcStarter.join(Launcher.java:273)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:79)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:54)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:34)
	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:646)
	at hudson.model.Build$RunnerImpl.build(Build.java:181)
	at hudson.model.Build$RunnerImpl.doRun(Build.java:136)
	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:434)
	at hudson.model.Run.run(Run.java:1390)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:40)
	at hudson.model.ResourceController.execute(ResourceController.java:82)
	at hudson.model.Executor.run(Executor.java:137)

When I launch and stop direct-maven-invoke, initially I get the following in console:

Aug 19, 2014 11:05:57 AM sun.reflect.NativeMethodAccessorImpl invoke0
INFO: Using bundled Maven 3 installation
Aug 19, 2014 11:05:57 AM sun.reflect.NativeMethodAccessorImpl invoke0
INFO: Checking Maven 3 installation environment
Aug 19, 2014 11:05:57 AM sun.reflect.GeneratedMethodAccessor223 invoke
INFO: Checking Maven 3 installation version
Aug 19, 2014 11:05:57 AM sun.reflect.NativeMethodAccessorImpl invoke0
INFO: Detected Maven 3 installation version: 3.1.1
Aug 19, 2014 11:06:08 AM hudson.model.Run run
INFO: direct-maven-invoke #4 aborted
java.lang.InterruptedException
	at java.lang.Object.wait(Native Method)
	at hudson.remoting.Request.call(Request.java:121)
	at hudson.remoting.Channel.call(Channel.java:643)
	at org.hudsonci.maven.plugin.builder.internal.PerformBuild.doExecute(PerformBuild.java:198)
	at org.hudsonci.utils.tasks.PerformOperation.execute(PerformOperation.java:50)
	at org.hudsonci.maven.plugin.builder.MavenBuilder.perform(MavenBuilder.java:169)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:34)
	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:646)
	at hudson.model.Build$RunnerImpl.build(Build.java:181)
	at hudson.model.Build$RunnerImpl.doRun(Build.java:136)
	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:434)
	at hudson.model.Run.run(Run.java:1390)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:40)
	at hudson.model.ResourceController.execute(ResourceController.java:82)
	at hudson.model.Executor.run(Executor.java:137)

But then, after a while, when spawned processes have exited by themselves, extra errors appear:

Aug 19, 2014 11:07:03 AM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel EventSpy:[/Users/ivaninozemtsev/.hudson/maven/slavebundle/bundled-maven/bin/mvn, clean, install, -V, -B, -Dmaven.ext.class.path=/Users/ivaninozemtsev/.hudson/maven/slavebundle/resources:/Users/ivaninozemtsev/.hudson/maven/slavebundle/lib/maven3-eventspy-3.1.jar:/Users/ivaninozemtsev/.hudson/war/webapp/WEB-INF/lib/hudson-remoting-3.0.0.jar, -Dhudson.eventspy.port=56594, -f, pom.xml]
java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1023)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2596)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1316)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1017)

Aug 19, 2014 11:07:03 AM hudson.util.ProcessTree killAll
INFO: killAll (process started by build): process=java.lang.UNIXProcess@193b2405 and envs={HUDSON_COOKIE=e72a873e-1592-4966-9339-36acddbfba7a}

[Updated on: Tue, 19 August 2014 04:12] by Moderator

Report message to a moderator

Previous Topic:	Maven encrypt password
Next Topic:	Hudson upgrade

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sat Apr 27 04:29:42 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter