Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, & CDO's InvalidationRunna
[CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, & CDO's InvalidationRunna [message #1237458] Wed, 29 January 2014 19:34 Go to next message
Josh M is currently offline Josh MFriend
Messages: 22
Registered: July 2009
Junior Member
I am hoping that someone might be able to help me with a problem with deadlocks that we are having. This problem seems to present itself when the following state occurs:

[1] The CDOView/Transaction is locked by some process.
[2] A process executing on the display thread attempts to access the model, and subsequently must wait for the CDOView/Transaction lock. This process may be native to the display thread, or may be a Runnable queued by Display.asyncExec or Display.syncExec. This process can originate from anywhere in the application.
[3] Within the process of #1, a separate and synchronized process is set to run against the display thread.

Specific to CDO/Dawn, we are seeing this problem primarily within CDO's InvalidationRunnable and Dawn's DawnAtrtGMFHandler.handleViewInvalidationEvent. Within the handleViewInvalidationEvent, refreshes to the diagram result in calls to the following areas (there may be more), each of which demonstrate the conditions of #3 above:
a. org.eclipse.emf.transaction.impl.TransactionalEditingDomainImpl.runExclusive(Runnable) [called within DawnDiagramUpdater.refreshEditPart]
b. org.eclipse.gmf.runtime.diagram.ui.editparts.ConnectionEditPart.installBendpointEditPolicy() [called within ConnectionEditPart.refreshVisuals]


As this deadlock issue present's itself randomly (most likely due to timing), it is very hard to generate a test that is sufficiently reproducible, although I hope the logic described above suffices to outline the fundamental problem. We are seeing this issue more frequently in our application, compared to the ACore example, most likely due to the much higher occurrences of #2 above. I was however, able to get a thread trace of this deadlock occurring within the ACore example (pasted below).

Can you recommend a way forward to deal with this issue permanently? This problem is approaching critical for our use of CDO and Dawn. Thanks!

Example Acore Deadlock Thread trace:
org.eclipse.equinox.launcher.Main at localhost:57194    
      Thread [main] (Suspended)     
         owns: RunnableLock  (id=138)   
         waiting for: CDOTransactionImpl  (id=139)    
            owned by: Thread [CDOViewInvalidationRunner-Transaction 6] (Suspended)      
         CDOStoreImpl.remove(InternalEObject, EStructuralFeature, int) line: 479   
         CDOLegacyAdapter.notifyRemove(EStructuralFeature, int) line: 250    
         CDOLegacyAdapter.notifyChanged(Notification) line: 125 
         NodeImpl(BasicNotifierImpl).eNotify(Notification) line: 374   
         ENotificationImpl(NotificationImpl).dispatch() line: 1027   
         ConnectorImpl(EdgeImpl).setTarget(View) line: 215   
         DawnGMFHandler$2$1.doExecute() line: 181  
         DawnGMFHandler$2$1(RecordingCommand).execute() line: 135   
         EMFCommandOperation.doExecute(IProgressMonitor, IAdaptable) line: 119     
         EMFCommandOperation(AbstractEMFOperation).execute(IProgressMonitor, IAdaptable) line: 150  
         DefaultOperationHistory.execute(IUndoableOperation, IProgressMonitor, IAdaptable) line: 513      
         WorkspaceCommandStackImpl.doExecute(Command, Map<?,?>) line: 208      
         WorkspaceCommandStackImpl(AbstractTransactionalCommandStack).execute(Command, Map<?,?>) line: 165   
         WorkspaceCommandStackImpl(AbstractTransactionalCommandStack).execute(Command) line: 219      
         DawnGMFHandler$2.run() line: 174     
         RunnableLock.run() line: 35     
         UISynchronizer(Synchronizer).runAsyncMessages(boolean) line: 135   
         Display.runAsyncMessages(boolean) line: 4144  
         Display.readAndDispatch() line: 3761   
         Workbench.runEventLoop(Window$IExceptionHandler, Display) line: 2701      
         Workbench.runUI() line: 2665   
         Workbench.access$4(Workbench) line: 2499   
         Workbench$7.run() line: 679  
         Realm.runWithDefault(Realm, Runnable) line: 332     
         Workbench.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 668   
         PlatformUI.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 149    
         IDEApplication.start(IApplicationContext) line: 124    
         EclipseAppHandle.run(Object) line: 196  
         EclipseAppLauncher.runApplication(Object) line: 110    
         EclipseAppLauncher.start(Object) line: 79      
         EclipseStarter.run(Object) line: 353   
         EclipseStarter.run(String[], Runnable) line: 180   
         NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method] 
         NativeMethodAccessorImpl.invoke(Object, Object[]) line: not available   
         DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: not available    
         Method.invoke(Object, Object...) line: not available   
         Main.invokeFramework(String[], URL[]) line: 629  
         Main.basicRun(String[]) line: 584 
         Main.run(String[]) line: 1438   
         Main.main(String[]) line: 1414   
      Thread [CDOViewInvalidationRunner-Transaction 6] (Suspended)  
         owns: TransactionImpl  (id=219)  
         owns: CDOTransactionImpl  (id=139)   
         waiting for: Object  (id=220)   
         Object.wait(long) line: not available [native method]    
         Object.wait() line: 503  
         ThreadJob.waitForRun(ThreadJob, IProgressMonitor, InternalJob, Thread) line: 270      
         ThreadJob.joinRun(ThreadJob, IProgressMonitor) line: 197     
         ImplicitJobs.begin(ISchedulingRule, IProgressMonitor, boolean) line: 92  
         JobManager.beginRule(ISchedulingRule, IProgressMonitor) line: 286     
         Lock.uiSafeAcquire(boolean) line: 359   
         DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).acquire(InternalTransaction) line: 580      
         DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).activate(InternalTransaction) line: 508      
         TransactionImpl.start() line: 204  
         DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).startTransaction(boolean, Map<?,?>) line: 424   
         DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).runExclusive(Runnable) line: 321      
         DawnDiagramUpdater.refreshEditPart(EditPart, DiagramDocumentEditor) line: 80 
         DawnGMFHandler.handleObject(CDOObject) line: 236    
         DawnGMFHandler.handleViewInvalidationEvent(CDOViewInvalidationEvent) line: 80      
         DawnGMFHandler(BasicDawnListener).notifyEvent(IEvent) line: 54     
         Notifier.fireEventSafe(IEvent, IListener[]) line: 167     
         CDOTransactionImpl(Notifier).fireEvent(IEvent, IListener[]) line: 113     
         CDOTransactionImpl(CDOViewImpl).fireInvalidationEvent(long, Map<CDOObject,CDORevisionDelta>, Set<CDOObject>) line: 984  
         CDOTransactionImpl(CDOViewImpl).doInvalidate(CDOBranch, long, List<CDORevisionKey>, List<CDOIDAndVersion>, Map<CDOID,InternalCDORevision>, boolean) line: 873  
         CDOViewImpl$InvalidationRunnable.run() line: 1709     
         CDOViewImpl$2(QueueRunner).work(Worker$WorkContext, Runnable) line: 26   
         CDOViewImpl$2(QueueRunner).work(Worker$WorkContext, Object) line: 1    
         CDOViewImpl$2(QueueWorker<E>).doWork(Worker$WorkContext) line: 78   
         CDOViewImpl$2(QueueWorker<E>).work(Worker$WorkContext) line: 70   
         Worker$WorkerThread.run() line: 209   
Re: [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, &amp; CDO's Invalidat [message #1242285 is a reply to message #1237458] Sun, 09 February 2014 09:15 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6394
Registered: July 2009
Senior Member
Hi Josh,

Sorry, I'm not a Dawn expert (and especially not the responsible for Dawn). Display.syncExec() is often problematic and
can easily lead to deadlocks. If you can provide me with a test case that reproduces the deadlock after a reasonable
number of runs I'm certainly willing to analyze it.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper


Am 29.01.2014 20:34, schrieb Josh M:
> I am hoping that someone might be able to help me with a problem with deadlocks that we are having. This problem
> seems to present itself when the following state occurs:
>
> [1] The CDOView/Transaction is locked by some process.
> [2] A process executing on the display thread attempts to access the model, and subsequently must wait for the
> CDOView/Transaction lock. This process may be native to the display thread, or may be a Runnable queued by
> Display.asyncExec or Display.syncExec. This process can originate from anywhere in the application.
> [3] Within the process of #1, a separate and synchronized process is set to run against the display thread.
> Specific to CDO/Dawn, we are seeing this problem primarily within CDO's InvalidationRunnable and Dawn's
> DawnAtrtGMFHandler.handleViewInvalidationEvent. Within the handleViewInvalidationEvent, refreshes to the diagram
> result in calls to the following areas (there may be more), each of which demonstrate the conditions of #3 above:
> a. org.eclipse.emf.transaction.impl.TransactionalEditingDomainImpl.runExclusive(Runnable) [called within
> DawnDiagramUpdater.refreshEditPart]
> b. org.eclipse.gmf.runtime.diagram.ui.editparts.ConnectionEditPart.installBendpointEditPolicy() [called within
> ConnectionEditPart.refreshVisuals]
>
>
> As this deadlock issue present's itself randomly (most likely due to timing), it is very hard to generate a test that
> is sufficiently reproducible, although I hope the logic described above suffices to outline the fundamental problem.
> We are seeing this issue more frequently in our application, compared to the ACore example, most likely due to the
> much higher occurrences of #2 above. I was however, able to get a thread trace of this deadlock occurring within the
> ACore example (pasted below).
>
> Can you recommend a way forward to deal with this issue permanently? This problem is approaching critical for our
> use of CDO and Dawn. Thanks!
>
> Example Acore Deadlock Thread trace:
> org.eclipse.equinox.launcher.Main at localhost:57194 Thread [main] (Suspended) owns: RunnableLock
> (id=138) waiting for: CDOTransactionImpl (id=139) owned by: Thread
> [CDOViewInvalidationRunner-Transaction 6] (Suspended) CDOStoreImpl.remove(InternalEObject,
> EStructuralFeature, int) line: 479 CDOLegacyAdapter.notifyRemove(EStructuralFeature, int) line: 250
> CDOLegacyAdapter.notifyChanged(Notification) line: 125 NodeImpl(BasicNotifierImpl).eNotify(Notification) line:
> 374 ENotificationImpl(NotificationImpl).dispatch() line: 1027
> ConnectorImpl(EdgeImpl).setTarget(View) line: 215 DawnGMFHandler$2$1.doExecute() line: 181
> DawnGMFHandler$2$1(RecordingCommand).execute() line: 135 EMFCommandOperation.doExecute(IProgressMonitor, IAdaptable)
> line: 119 EMFCommandOperation(AbstractEMFOperation).execute(IProgressMonitor, IAdaptable) line: 150
> DefaultOperationHistory.execute(IUndoableOperation, IProgressMonitor, IAdaptable) line: 513
> WorkspaceCommandStackImpl.doExecute(Command, Map<?,?>) line: 208
> WorkspaceCommandStackImpl(AbstractTransactionalCommandStack).execute(Command, Map<?,?>) line: 165
> WorkspaceCommandStackImpl(AbstractTransactionalCommandStack).execute(Command) line: 219
> DawnGMFHandler$2.run() line: 174 RunnableLock.run() line: 35
> UISynchronizer(Synchronizer).runAsyncMessages(boolean) line: 135 Display.runAsyncMessages(boolean) line: 4144
> Display.readAndDispatch() line: 3761 Workbench.runEventLoop(Window$IExceptionHandler, Display) line: 2701
> Workbench.runUI() line: 2665 Workbench.access$4(Workbench) line: 2499 Workbench$7.run() line: 679
> Realm.runWithDefault(Realm, Runnable) line: 332 Workbench.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 668
> PlatformUI.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 149
> IDEApplication.start(IApplicationContext) line: 124 EclipseAppHandle.run(Object) line: 196
> EclipseAppLauncher.runApplication(Object) line: 110 EclipseAppLauncher.start(Object) line: 79
> EclipseStarter.run(Object) line: 353 EclipseStarter.run(String[], Runnable) line: 180
> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
> NativeMethodAccessorImpl.invoke(Object, Object[]) line: not available
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: not available Method.invoke(Object, Object...)
> line: not available Main.invokeFramework(String[], URL[]) line: 629 Main.basicRun(String[]) line: 584
> Main.run(String[]) line: 1438 Main.main(String[]) line: 1414 Thread
> [CDOViewInvalidationRunner-Transaction 6] (Suspended) owns: TransactionImpl (id=219) owns:
> CDOTransactionImpl (id=139) waiting for: Object (id=220) Object.wait(long) line: not available [native
> method] Object.wait() line: 503 ThreadJob.waitForRun(ThreadJob, IProgressMonitor, InternalJob, Thread) line:
> 270 ThreadJob.joinRun(ThreadJob, IProgressMonitor) line: 197 ImplicitJobs.begin(ISchedulingRule,
> IProgressMonitor, boolean) line: 92 JobManager.beginRule(ISchedulingRule, IProgressMonitor) line: 286
> Lock.uiSafeAcquire(boolean) line: 359
> DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).acquire(InternalTransaction)
> line: 580
> DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).activate(InternalTransaction)
> line: 508 TransactionImpl.start() line: 204
> DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).startTransaction(boolean,
> Map<?,?>) line: 424
> DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).runExclusive(Runnable) line:
> 321 DawnDiagramUpdater.refreshEditPart(EditPart, DiagramDocumentEditor) line: 80
> DawnGMFHandler.handleObject(CDOObject) line: 236 DawnGMFHandler.handleViewInvalidationEvent(CDOViewInvalidationEvent)
> line: 80 DawnGMFHandler(BasicDawnListener).notifyEvent(IEvent) line: 54 Notifier.fireEventSafe(IEvent,
> IListener[]) line: 167 CDOTransactionImpl(Notifier).fireEvent(IEvent, IListener[]) line: 113
> CDOTransactionImpl(CDOViewImpl).fireInvalidationEvent(long, Map<CDOObject,CDORevisionDelta>, Set<CDOObject>) line: 984
> CDOTransactionImpl(CDOViewImpl).doInvalidate(CDOBranch, long, List<CDORevisionKey>, List<CDOIDAndVersion>,
> Map<CDOID,InternalCDORevision>, boolean) line: 873 CDOViewImpl$InvalidationRunnable.run() line: 1709
> CDOViewImpl$2(QueueRunner).work(Worker$WorkContext, Runnable) line: 26
> CDOViewImpl$2(QueueRunner).work(Worker$WorkContext, Object) line: 1
> CDOViewImpl$2(QueueWorker<E>).doWork(Worker$WorkContext) line: 78
> CDOViewImpl$2(QueueWorker<E>).work(Worker$WorkContext) line: 70 Worker$WorkerThread.run() line: 209
Re: [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, &amp; CDO's Invalidat [message #1711921 is a reply to message #1242285] Tue, 20 October 2015 10:02 Go to previous messageGo to next message
Laurent Fasani is currently offline Laurent FasaniFriend
Messages: 136
Registered: October 2014
Senior Member
Hello

I'd like to revive this subject because it's still occurring on our application(sirius and sirius client) and even it is reproductible in particular conditions we are talking about deadlock so it is a major issue for us
I have similar stackTraces

Scenario:
1- execute an EMFCommand which lasts long enough to do step2 (for example open a DialogBox in the Command which require user to close it)
2- meanwhile do some modification on CDO server. Then the CDOViewInvalidationRunner thread will do some stuff on the client.

Step1 is executed on UIThread and will require CDOTransaction to apply changes

Step2 : In CDOViewInvalidationRunner thread, CDOViewImpl.doInvalidate takes the lock on CDOTransaction. If an EMFCommand is executed, during step2, it'll require the lock on UI because of TransactionalEditingDomainImpl.acquire which call Lock.uiAcquire as pointed out by Josh

The problem comes from that.
Why does TransactionalEditingDomainImpl.acquire need to call
transactionLock.uiSafeAcquire(!tx.isReadOnly());

EMFTransaction should not rely on the UIThread.

The https://bugs.eclipse.org/bugs/show_bug.cgi?id=375161 bug is opened to handle this issue.
In that ticket, Esteban DUGUEPEROUX, proposed to call
transactionLock.acquire(!tx.isReadOnly());


What do you think?
Thank a lot for your answer.


Laurent Fasani - Obeo
Need training or professional services for Sirius?
http://www.obeodesigner.com/sirius
Re: [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, &amp;amp; CDO's Inval [message #1711944 is a reply to message #1711921] Tue, 20 October 2015 11:44 Go to previous message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6394
Registered: July 2009
Senior Member
Am 20.10.2015 um 12:02 schrieb Laurent Fasani:
> Hello
>
> I'd like to revive this subject
It seems to me that you want to revive this discussion with the EMF Transaction team, right? If so, I'd suggest you
start a new thread in this forum and don't use the prefix [CDO]. That will hopefully attract their experts...

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper



> because it's still occurring on our application(sirius and sirius client) and even it is reproductible in particular
> conditions we are talking about deadlock so it is a major issue for us
> I have similar stackTraces
>
> Scenario: 1- execute an EMFCommand which lasts long enough to do step2 (for example open a DialogBox in the Command
> which require user to close it)
> 2- meanwhile do some modification on CDO server. Then the CDOViewInvalidationRunner thread will do some stuff on the
> client.
>
> Step1 is executed on UIThread and will require CDOTransaction to apply changes
>
> Step2 : In CDOViewInvalidationRunner thread, CDOViewImpl.doInvalidate takes the lock on CDOTransaction. If an
> EMFCommand is executed, during step2, it'll require the lock on UI because of TransactionalEditingDomainImpl.acquire
> which call Lock.uiAcquire as pointed out by Josh
>
> The problem comes from that.
> Why does TransactionalEditingDomainImpl.acquire need to call transactionLock.uiSafeAcquire(!tx.isReadOnly());
> EMFTransaction should not rely on the UIThread.
>
> The https://bugs.eclipse.org/bugs/show_bug.cgi?id=375161 bug is opened to handle this issue.
> In that ticket, Esteban DUGUEPEROUX, proposed to call transactionLock.acquire(!tx.isReadOnly());
>
> What do you think?
> Thank a lot for your answer.
>
Previous Topic: [XCore] xtend-maven-plugin used with tycho don't generate sources on JENKINS
Next Topic:Deadlock in EMF transaction
Goto Forum:
  


Current Time: Mon Sep 24 20:56:36 GMT 2018

Powered by FUDForum. Page generated in 0.03966 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top