Home » Modeling » EMF » [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, & CDO's InvalidationRunna
| [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, & CDO's InvalidationRunna [message #1237458] |
Wed, 29 January 2014 19:34  |
 Sed Mising name
Messages: 22
Registered: July 2009 |
Junior Member |
|
|
I am hoping that someone might be able to help me with a problem with deadlocks that we are having. This problem seems to present itself when the following state occurs:
[1] The CDOView/Transaction is locked by some process.
[2] A process executing on the display thread attempts to access the model, and subsequently must wait for the CDOView/Transaction lock. This process may be native to the display thread, or may be a Runnable queued by Display.asyncExec or Display.syncExec. This process can originate from anywhere in the application.
[3] Within the process of #1, a separate and synchronized process is set to run against the display thread.
Specific to CDO/Dawn, we are seeing this problem primarily within CDO's InvalidationRunnable and Dawn's DawnAtrtGMFHandler.handleViewInvalidationEvent. Within the handleViewInvalidationEvent, refreshes to the diagram result in calls to the following areas (there may be more), each of which demonstrate the conditions of #3 above:
a. org.eclipse.emf.transaction.impl.TransactionalEditingDomainImpl.runExclusive(Runnable) [called within DawnDiagramUpdater.refreshEditPart]
b. org.eclipse.gmf.runtime.diagram.ui.editparts.ConnectionEditPart.installBendpointEditPolicy() [called within ConnectionEditPart.refreshVisuals]
As this deadlock issue present's itself randomly (most likely due to timing), it is very hard to generate a test that is sufficiently reproducible, although I hope the logic described above suffices to outline the fundamental problem. We are seeing this issue more frequently in our application, compared to the ACore example, most likely due to the much higher occurrences of #2 above. I was however, able to get a thread trace of this deadlock occurring within the ACore example (pasted below).
Can you recommend a way forward to deal with this issue permanently? This problem is approaching critical for our use of CDO and Dawn. Thanks!
Example Acore Deadlock Thread trace:
org.eclipse.equinox.launcher.Main at localhost:57194
Thread [main] (Suspended)
owns: RunnableLock (id=138)
waiting for: CDOTransactionImpl (id=139)
owned by: Thread [CDOViewInvalidationRunner-Transaction 6] (Suspended)
CDOStoreImpl.remove(InternalEObject, EStructuralFeature, int) line: 479
CDOLegacyAdapter.notifyRemove(EStructuralFeature, int) line: 250
CDOLegacyAdapter.notifyChanged(Notification) line: 125
NodeImpl(BasicNotifierImpl).eNotify(Notification) line: 374
ENotificationImpl(NotificationImpl).dispatch() line: 1027
ConnectorImpl(EdgeImpl).setTarget(View) line: 215
DawnGMFHandler$2$1.doExecute() line: 181
DawnGMFHandler$2$1(RecordingCommand).execute() line: 135
EMFCommandOperation.doExecute(IProgressMonitor, IAdaptable) line: 119
EMFCommandOperation(AbstractEMFOperation).execute(IProgressMonitor, IAdaptable) line: 150
DefaultOperationHistory.execute(IUndoableOperation, IProgressMonitor, IAdaptable) line: 513
WorkspaceCommandStackImpl.doExecute(Command, Map<?,?>) line: 208
WorkspaceCommandStackImpl(AbstractTransactionalCommandStack).execute(Command, Map<?,?>) line: 165
WorkspaceCommandStackImpl(AbstractTransactionalCommandStack).execute(Command) line: 219
DawnGMFHandler$2.run() line: 174
RunnableLock.run() line: 35
UISynchronizer(Synchronizer).runAsyncMessages(boolean) line: 135
Display.runAsyncMessages(boolean) line: 4144
Display.readAndDispatch() line: 3761
Workbench.runEventLoop(Window$IExceptionHandler, Display) line: 2701
Workbench.runUI() line: 2665
Workbench.access$4(Workbench) line: 2499
Workbench$7.run() line: 679
Realm.runWithDefault(Realm, Runnable) line: 332
Workbench.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 668
PlatformUI.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 149
IDEApplication.start(IApplicationContext) line: 124
EclipseAppHandle.run(Object) line: 196
EclipseAppLauncher.runApplication(Object) line: 110
EclipseAppLauncher.start(Object) line: 79
EclipseStarter.run(Object) line: 353
EclipseStarter.run(String[], Runnable) line: 180
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: not available
Method.invoke(Object, Object...) line: not available
Main.invokeFramework(String[], URL[]) line: 629
Main.basicRun(String[]) line: 584
Main.run(String[]) line: 1438
Main.main(String[]) line: 1414
Thread [CDOViewInvalidationRunner-Transaction 6] (Suspended)
owns: TransactionImpl (id=219)
owns: CDOTransactionImpl (id=139)
waiting for: Object (id=220)
Object.wait(long) line: not available [native method]
Object.wait() line: 503
ThreadJob.waitForRun(ThreadJob, IProgressMonitor, InternalJob, Thread) line: 270
ThreadJob.joinRun(ThreadJob, IProgressMonitor) line: 197
ImplicitJobs.begin(ISchedulingRule, IProgressMonitor, boolean) line: 92
JobManager.beginRule(ISchedulingRule, IProgressMonitor) line: 286
Lock.uiSafeAcquire(boolean) line: 359
DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).acquire(InternalTransaction) line: 580
DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).activate(InternalTransaction) line: 508
TransactionImpl.start() line: 204
DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).startTransaction(boolean, Map<?,?>) line: 424
DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).runExclusive(Runnable) line: 321
DawnDiagramUpdater.refreshEditPart(EditPart, DiagramDocumentEditor) line: 80
DawnGMFHandler.handleObject(CDOObject) line: 236
DawnGMFHandler.handleViewInvalidationEvent(CDOViewInvalidationEvent) line: 80
DawnGMFHandler(BasicDawnListener).notifyEvent(IEvent) line: 54
Notifier.fireEventSafe(IEvent, IListener[]) line: 167
CDOTransactionImpl(Notifier).fireEvent(IEvent, IListener[]) line: 113
CDOTransactionImpl(CDOViewImpl).fireInvalidationEvent(long, Map<CDOObject,CDORevisionDelta>, Set<CDOObject>) line: 984
CDOTransactionImpl(CDOViewImpl).doInvalidate(CDOBranch, long, List<CDORevisionKey>, List<CDOIDAndVersion>, Map<CDOID,InternalCDORevision>, boolean) line: 873
CDOViewImpl$InvalidationRunnable.run() line: 1709
CDOViewImpl$2(QueueRunner).work(Worker$WorkContext, Runnable) line: 26
CDOViewImpl$2(QueueRunner).work(Worker$WorkContext, Object) line: 1
CDOViewImpl$2(QueueWorker<E>).doWork(Worker$WorkContext) line: 78
CDOViewImpl$2(QueueWorker<E>).work(Worker$WorkContext) line: 70
Worker$WorkerThread.run() line: 209
|
|
|
| Re: [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, & CDO's Invalidat [message #1242285 is a reply to message #1237458] |
Sun, 09 February 2014 09:15  |
|
Hi Josh,
Sorry, I'm not a Dawn expert (and especially not the responsible for Dawn). Display.syncExec() is often problematic and
can easily lead to deadlocks. If you can provide me with a test case that reproduces the deadlock after a reasonable
number of runs I'm certainly willing to analyze it.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Am 29.01.2014 20:34, schrieb Josh M:
> I am hoping that someone might be able to help me with a problem with deadlocks that we are having. This problem
> seems to present itself when the following state occurs:
>
> [1] The CDOView/Transaction is locked by some process.
> [2] A process executing on the display thread attempts to access the model, and subsequently must wait for the
> CDOView/Transaction lock. This process may be native to the display thread, or may be a Runnable queued by
> Display.asyncExec or Display.syncExec. This process can originate from anywhere in the application.
> [3] Within the process of #1, a separate and synchronized process is set to run against the display thread.
> Specific to CDO/Dawn, we are seeing this problem primarily within CDO's InvalidationRunnable and Dawn's
> DawnAtrtGMFHandler.handleViewInvalidationEvent. Within the handleViewInvalidationEvent, refreshes to the diagram
> result in calls to the following areas (there may be more), each of which demonstrate the conditions of #3 above:
> a. org.eclipse.emf.transaction.impl.TransactionalEditingDomainImpl.runExclusive(Runnable) [called within
> DawnDiagramUpdater.refreshEditPart]
> b. org.eclipse.gmf.runtime.diagram.ui.editparts.ConnectionEditPart.installBendpointEditPolicy() [called within
> ConnectionEditPart.refreshVisuals]
>
>
> As this deadlock issue present's itself randomly (most likely due to timing), it is very hard to generate a test that
> is sufficiently reproducible, although I hope the logic described above suffices to outline the fundamental problem.
> We are seeing this issue more frequently in our application, compared to the ACore example, most likely due to the
> much higher occurrences of #2 above. I was however, able to get a thread trace of this deadlock occurring within the
> ACore example (pasted below).
>
> Can you recommend a way forward to deal with this issue permanently? This problem is approaching critical for our
> use of CDO and Dawn. Thanks!
>
> Example Acore Deadlock Thread trace:
> org.eclipse.equinox.launcher.Main at localhost:57194 Thread [main] (Suspended) owns: RunnableLock
> (id=138) waiting for: CDOTransactionImpl (id=139) owned by: Thread
> [CDOViewInvalidationRunner-Transaction 6] (Suspended) CDOStoreImpl.remove(InternalEObject,
> EStructuralFeature, int) line: 479 CDOLegacyAdapter.notifyRemove(EStructuralFeature, int) line: 250
> CDOLegacyAdapter.notifyChanged(Notification) line: 125 NodeImpl(BasicNotifierImpl).eNotify(Notification) line:
> 374 ENotificationImpl(NotificationImpl).dispatch() line: 1027
> ConnectorImpl(EdgeImpl).setTarget(View) line: 215 DawnGMFHandler$2$1.doExecute() line: 181
> DawnGMFHandler$2$1(RecordingCommand).execute() line: 135 EMFCommandOperation.doExecute(IProgressMonitor, IAdaptable)
> line: 119 EMFCommandOperation(AbstractEMFOperation).execute(IProgressMonitor, IAdaptable) line: 150
> DefaultOperationHistory.execute(IUndoableOperation, IProgressMonitor, IAdaptable) line: 513
> WorkspaceCommandStackImpl.doExecute(Command, Map<?,?>) line: 208
> WorkspaceCommandStackImpl(AbstractTransactionalCommandStack).execute(Command, Map<?,?>) line: 165
> WorkspaceCommandStackImpl(AbstractTransactionalCommandStack).execute(Command) line: 219
> DawnGMFHandler$2.run() line: 174 RunnableLock.run() line: 35
> UISynchronizer(Synchronizer).runAsyncMessages(boolean) line: 135 Display.runAsyncMessages(boolean) line: 4144
> Display.readAndDispatch() line: 3761 Workbench.runEventLoop(Window$IExceptionHandler, Display) line: 2701
> Workbench.runUI() line: 2665 Workbench.access$4(Workbench) line: 2499 Workbench$7.run() line: 679
> Realm.runWithDefault(Realm, Runnable) line: 332 Workbench.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 668
> PlatformUI.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 149
> IDEApplication.start(IApplicationContext) line: 124 EclipseAppHandle.run(Object) line: 196
> EclipseAppLauncher.runApplication(Object) line: 110 EclipseAppLauncher.start(Object) line: 79
> EclipseStarter.run(Object) line: 353 EclipseStarter.run(String[], Runnable) line: 180
> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
> NativeMethodAccessorImpl.invoke(Object, Object[]) line: not available
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: not available Method.invoke(Object, Object...)
> line: not available Main.invokeFramework(String[], URL[]) line: 629 Main.basicRun(String[]) line: 584
> Main.run(String[]) line: 1438 Main.main(String[]) line: 1414 Thread
> [CDOViewInvalidationRunner-Transaction 6] (Suspended) owns: TransactionImpl (id=219) owns:
> CDOTransactionImpl (id=139) waiting for: Object (id=220) Object.wait(long) line: not available [native
> method] Object.wait() line: 503 ThreadJob.waitForRun(ThreadJob, IProgressMonitor, InternalJob, Thread) line:
> 270 ThreadJob.joinRun(ThreadJob, IProgressMonitor) line: 197 ImplicitJobs.begin(ISchedulingRule,
> IProgressMonitor, boolean) line: 92 JobManager.beginRule(ISchedulingRule, IProgressMonitor) line: 286
> Lock.uiSafeAcquire(boolean) line: 359
> DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).acquire(InternalTransaction)
> line: 580
> DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).activate(InternalTransaction)
> line: 508 TransactionImpl.start() line: 204
> DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).startTransaction(boolean,
> Map<?,?>) line: 424
> DawnDiagramEditingDomainFactory$DawnDiagramEditingDomain(TransactionalEditingDomainImpl).runExclusive(Runnable) line:
> 321 DawnDiagramUpdater.refreshEditPart(EditPart, DiagramDocumentEditor) line: 80
> DawnGMFHandler.handleObject(CDOObject) line: 236 DawnGMFHandler.handleViewInvalidationEvent(CDOViewInvalidationEvent)
> line: 80 DawnGMFHandler(BasicDawnListener).notifyEvent(IEvent) line: 54 Notifier.fireEventSafe(IEvent,
> IListener[]) line: 167 CDOTransactionImpl(Notifier).fireEvent(IEvent, IListener[]) line: 113
> CDOTransactionImpl(CDOViewImpl).fireInvalidationEvent(long, Map<CDOObject,CDORevisionDelta>, Set<CDOObject>) line: 984
> CDOTransactionImpl(CDOViewImpl).doInvalidate(CDOBranch, long, List<CDORevisionKey>, List<CDOIDAndVersion>,
> Map<CDOID,InternalCDORevision>, boolean) line: 873 CDOViewImpl$InvalidationRunnable.run() line: 1709
> CDOViewImpl$2(QueueRunner).work(Worker$WorkContext, Runnable) line: 26
> CDOViewImpl$2(QueueRunner).work(Worker$WorkContext, Object) line: 1
> CDOViewImpl$2(QueueWorker<E>).doWork(Worker$WorkContext) line: 78
> CDOViewImpl$2(QueueWorker<E>).work(Worker$WorkContext) line: 70 Worker$WorkerThread.run() line: 209
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, & CDO's Invalidat [message #1711921 is a reply to message #1242285] |
Tue, 20 October 2015 10:02 |
|
Hello
I'd like to revive this subject because it's still occurring on our application(sirius and sirius client) and even it is reproductible in particular conditions we are talking about deadlock so it is a major issue for us
I have similar stackTraces
Scenario:
1- execute an EMFCommand which lasts long enough to do step2 (for example open a DialogBox in the Command which require user to close it)
2- meanwhile do some modification on CDO server. Then the CDOViewInvalidationRunner thread will do some stuff on the client.
Step1 is executed on UIThread and will require CDOTransaction to apply changes
Step2 : In CDOViewInvalidationRunner thread, CDOViewImpl.doInvalidate takes the lock on CDOTransaction. If an EMFCommand is executed, during step2, it'll require the lock on UI because of TransactionalEditingDomainImpl.acquire which call Lock.uiAcquire as pointed out by Josh
The problem comes from that.
Why does TransactionalEditingDomainImpl.acquire need to call transactionLock.uiSafeAcquire(!tx.isReadOnly());
EMFTransaction should not rely on the UIThread.
The https://bugs.eclipse.org/bugs/show_bug.cgi?id=375161 bug is opened to handle this issue.
In that ticket, Esteban DUGUEPEROUX, proposed to call transactionLock.acquire(!tx.isReadOnly());
What do you think?
Thank a lot for your answer.
Laurent Fasani - Obeo
Need training or professional services for Sirius?
http://www.obeodesigner.com/sirius
|
|
|
| Re: [CDO/Dawn] Deadlocking w/ Display, Dawn's handleViewInvalidationEvent, &amp; CDO's Inval [message #1711944 is a reply to message #1711921] |
Tue, 20 October 2015 11:44 |
|
Am 20.10.2015 um 12:02 schrieb Laurent Fasani:
> Hello
>
> I'd like to revive this subject
It seems to me that you want to revive this discussion with the EMF Transaction team, right? If so, I'd suggest you
start a new thread in this forum and don't use the prefix [CDO]. That will hopefully attract their experts...
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
> because it's still occurring on our application(sirius and sirius client) and even it is reproductible in particular
> conditions we are talking about deadlock so it is a major issue for us
> I have similar stackTraces
>
> Scenario: 1- execute an EMFCommand which lasts long enough to do step2 (for example open a DialogBox in the Command
> which require user to close it)
> 2- meanwhile do some modification on CDO server. Then the CDOViewInvalidationRunner thread will do some stuff on the
> client.
>
> Step1 is executed on UIThread and will require CDOTransaction to apply changes
>
> Step2 : In CDOViewInvalidationRunner thread, CDOViewImpl.doInvalidate takes the lock on CDOTransaction. If an
> EMFCommand is executed, during step2, it'll require the lock on UI because of TransactionalEditingDomainImpl.acquire
> which call Lock.uiAcquire as pointed out by Josh
>
> The problem comes from that.
> Why does TransactionalEditingDomainImpl.acquire need to call transactionLock.uiSafeAcquire(!tx.isReadOnly());
> EMFTransaction should not rely on the UIThread.
>
> The https://bugs.eclipse.org/bugs/show_bug.cgi?id=375161 bug is opened to handle this issue.
> In that ticket, Esteban DUGUEPEROUX, proposed to call transactionLock.acquire(!tx.isReadOnly());
>
> What do you think?
> Thank a lot for your answer.
>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
Goto Forum:
Current Time: Fri Apr 19 06:27:41 GMT 2024
Powered by FUDForum. Page generated in 0.01925 seconds |
] 
Search
Help
Register
Login
Home
