Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cdt-dev] Frequent cdt-verify job aborts

You can use this job to run jstack on all Java processes

https://hudson.eclipse.org/cdt/job/jstack-all/


I just ran it and it looks like this (skipping irrelevant threads):



"Worker-7" #3293 prio=5 os_prio=0 tid=0x00007f2114051800 nid=0x728d in Object.wait() [0x00007f20f7efd000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.core.internal.jobs.ThreadJob.waitForRun(ThreadJob.java:304)
    - locked <0x00000000f74bfde0> (a java.lang.Object)
    at org.eclipse.core.internal.jobs.ThreadJob.joinRun(ThreadJob.java:206)
    at org.eclipse.core.internal.jobs.ImplicitJobs.begin(ImplicitJobs.java:92)
    at org.eclipse.core.internal.jobs.JobManager.beginRule(JobManager.java:324)
    at org.eclipse.core.internal.resources.WorkManager.checkIn(WorkManager.java:121)
    at org.eclipse.core.internal.resources.Workspace.prepareOperation(Workspace.java:2188)
    at org.eclipse.core.internal.resources.Resource.refreshLocal(Resource.java:1552)
    at org.eclipse.cdt.core.settings.model.CProjectDescriptionStorageTests$1.run(CProjectDescriptionStorageTests.java:100)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:56)


"main" #1 prio=5 os_prio=0 tid=0x00007f220400b000 nid=0x5167 in Object.wait() [0x00007f220db8f000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.cdt.internal.core.pdom.PDOMIndexerJob.cancelJobs(PDOMIndexerJob.java:207)
    - locked <0x00000000e0edd920> (a java.lang.Object)
    at org.eclipse.cdt.internal.core.pdom.PDOMManager.cancelIndexerJobs(PDOMManager.java:939)
    at org.eclipse.cdt.internal.core.pdom.PDOMManager.stopIndexer(PDOMManager.java:922)
    at org.eclipse.cdt.internal.core.pdom.PDOMManager.preRemoveProject(PDOMManager.java:858)
    at org.eclipse.cdt.internal.core.pdom.PDOMManager.preCloseProject(PDOMManager.java:838)
    at org.eclipse.cdt.internal.core.model.CModelManager.preCloseProject(CModelManager.java:1417)
    at org.eclipse.cdt.internal.core.model.CModelManager.resourceChanged(CModelManager.java:913)
    at org.eclipse.core.internal.events.NotificationManager$1.run(NotificationManager.java:299)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.core.internal.events.NotificationManager.notify(NotificationManager.java:289)
    at org.eclipse.core.internal.events.NotificationManager.handleEvent(NotificationManager.java:253)
    at org.eclipse.core.internal.resources.Workspace.broadcastEvent(Workspace.java:369)
    at org.eclipse.core.internal.resources.Project.close(Project.java:189)
    at org.eclipse.cdt.core.settings.model.CProjectDescriptionStorageTests.testExternalCProjDescModification(CProjectDescriptionStorageTests.java:126)



"Worker-0" #46 prio=5 os_prio=0 tid=0x00007f2204812000 nid=0x51b4 in Object.wait() [0x00007f218a839000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.core.internal.jobs.ThreadJob.waitForRun(ThreadJob.java:304)
    - locked <0x00000000f74bfde0> (a java.lang.Object)
    at org.eclipse.core.internal.jobs.ThreadJob.joinRun(ThreadJob.java:206)
    at org.eclipse.core.internal.jobs.ImplicitJobs.begin(ImplicitJobs.java:92)
    at org.eclipse.core.internal.jobs.JobManager.beginRule(JobManager.java:324)
    at org.eclipse.cdt.internal.core.settings.model.CProjectDescriptionManager.runWspModification(CProjectDescriptionManager.java:469)
    at org.eclipse.cdt.internal.core.settings.model.CProjectDescriptionManager.runWspModification(CProjectDescriptionManager.java:443)
    at org.eclipse.cdt.internal.core.settings.model.CExternalSettingsManager.settingsChanged(CExternalSettingsManager.java:439)
    at org.eclipse.cdt.internal.core.settings.model.CExternalSettingContainerFactoryWithListener.notifySettingsChange(CExternalSettingContainerFactoryWithListener.java:48)
    at org.eclipse.cdt.internal.core.settings.model.CfgExportSettingContainerFactory.handleEvent(CfgExportSettingContainerFactory.java:253)
    at org.eclipse.cdt.internal.core.settings.model.CProjectDescriptionManager.notifyListeners(CProjectDescriptionManager.java:2180)
    at org.eclipse.cdt.internal.core.settings.model.AbstractCProjectDescriptionStorage.fireLoadedEvent(AbstractCProjectDescriptionStorage.java:268)
    at org.eclipse.cdt.internal.core.settings.model.xml.XmlProjectDescriptionStorage.getProjectDescription(XmlProjectDescriptionStorage.java:264)
    at org.eclipse.cdt.internal.core.settings.model.CProjectDescriptionManager.getProjectDescriptionInternal(CProjectDescriptionManager.java:426)
    at org.eclipse.cdt.internal.core.settings.model.CProjectDescriptionManager.getProjectDescription(CProjectDescriptionManager.java:408)
    at org.eclipse.cdt.internal.core.settings.model.CProjectDescriptionManager.getProjectDescription(CProjectDescriptionManager.java:402)
    at org.eclipse.cdt.internal.core.settings.model.CProjectDescriptionManager.getProjectDescription(CProjectDescriptionManager.java:395)
    at org.eclipse.cdt.core.CCorePlugin.getProjectDescription(CCorePlugin.java:1440)
    at org.eclipse.cdt.internal.core.model.CModelManager.getBinaryParser(CModelManager.java:636)
    at org.eclipse.cdt.internal.core.model.CModelManager.createBinaryFile(CModelManager.java:672)
    at org.eclipse.cdt.internal.core.model.CContainer.computeChild(CContainer.java:216)
    at org.eclipse.cdt.internal.core.model.CContainer.computeChildren(CContainer.java:182)
    at org.eclipse.cdt.internal.core.model.CContainer.buildStructure(CContainer.java:154)
    at org.eclipse.cdt.internal.core.model.Openable.generateInfos(Openable.java:261)
    at org.eclipse.cdt.internal.core.model.CElement.openWhenClosed(CElement.java:427)
    at org.eclipse.cdt.internal.core.model.CElement.getElementInfo(CElement.java:305)
    at org.eclipse.cdt.internal.core.model.CElement.getElementInfo(CElement.java:295)
    at org.eclipse.cdt.internal.core.model.Parent.getChildren(Parent.java:55)
    at org.eclipse.cdt.internal.core.model.CElement.accept(CElement.java:524)
    at org.eclipse.cdt.internal.core.model.CElement.accept(CElement.java:526)
    at org.eclipse.cdt.internal.core.pdom.indexer.PDOMRebuildTask.createDelegate(PDOMRebuildTask.java:118)
    at org.eclipse.cdt.internal.core.pdom.indexer.PDOMRebuildTask.run(PDOMRebuildTask.java:78)
    at org.eclipse.cdt.internal.core.pdom.PDOMIndexerJob.run(PDOMIndexerJob.java:155)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:56)



https://hudson.eclipse.org/cdt/job/jstack-all/31/consoleFull


Marc-André




From: cdt-dev-bounces@xxxxxxxxxxx <cdt-dev-bounces@xxxxxxxxxxx> on behalf of Sergey Prigogin <eclipse.sprigogin@xxxxxxxxx>
Sent: Friday, February 3, 2017 4:28 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Frequent cdt-verify job aborts
 
It looks like tests are hanging. Currently in both, https://hudson.eclipse.org/cdt/job/cdt-verify/7440/console and https://hudson.eclipse.org/cdt/job/cdt-master/1165/console. Any ideas on how to investigate that?

-sergey 

On Fri, Feb 3, 2017 at 12:56 PM, Marc-André Laperle <marc-andre.laperle@xxxxxxxxxxxx> wrote:

It used to be 2 executor. Maybe we can try switching it back. If there is no objection I can change it...


From: cdt-dev-bounces@xxxxxxxxxxx <cdt-dev-bounces@xxxxxxxxxxx> on behalf of Nathan Ridge <zeratul976@xxxxxxxxxxx>
Sent: Friday, February 3, 2017 3:37:57 PM
To: CDT Mailing List
Subject: [cdt-dev] Frequent cdt-verify job aborts
 
Lately I've noticed our cdt-verify jobs are being aborted very frequently, due to a timeout. In such cases, the following appears at the end of the build output:

   Build timed out (after 150 minutes). Marking the build as aborted.

Does anyone know the cause of these timeouts? Are we just expecting the tests to run more quickly than they actually do?

I also noticed that the aborts happen more frequently when there are more jobs being done in parallel. Are we perhaps allowing more jobs to run in parallel than the machine can handle?

Thanks,
Nate
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev


Back to the top