Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Deadlock between Job and event listener on Display thread
Deadlock between Job and event listener on Display thread [message #1843245] Wed, 21 July 2021 11:18 Go to next message
Elie Richa is currently offline Elie RichaFriend
Messages: 44
Registered: February 2016
Member
Hello folks,

In the Eclipse integration of my language I set up an editor callback to listen to keystrokes and determine the semantic object where they occurred. I also have Eclipse Jobs that can simultaneously be trying to modify the document. I am reaching a dead-lock and I'm not sure how to resolve it.

The Job is something like this:

val editor = HandlerUtil.getActiveEditor(event) as XtextEditor
val doc = editor.document
val job = Job.create("Modifying document") [
    doc.modify [
        // Modify the semantic model
    ]
    return null
]
job.schedule


On the other hand I setup an SWT VerifyListener to intercept keystrokes in this way:

(Omitting a lot of injection and initialization code. Assume everything is working correctly with no runtime exceptions. The dead-lock is the only issue)

class BLEditorCallback extends IXtextEditorCallback.NullImpl {
    override afterCreatePartControl(XtextEditor editor) {
        super.afterCreatePartControl(editor)
        partVerifyListener = new VerifyListener() {
            override verifyText(VerifyEvent e) {
                val selProv = editor.selectionProvider.selection
                val currentObj = if (selProv instanceof ITextSelection) {
                        val offset = selProv.offset
                        editor.document.readOnly [
                            objAtOffsetHelper.resolveContainedElementAt(it, offset)
                        ]
               }
            }
        }
        editor.addVerifyListener(partVerifyListener)
    }
}


When I launch the job, and then hit a key in the editor while the job is running, I get the dead-lock. The Job thread and the Display thread stacks look like this:

"Worker-3: Modifying document" #44 prio=5 os_prio=31 cpu=15193.22ms elapsed=443.31s tid=0x00007fc49a827000 nid=0x16603 waiting on condition  [0x000070000c083000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@11.0.11/Native Method)
        - parking to wait for  <0x000000075e00a9b0> (a java.util.concurrent.Semaphore$NonfairSync)
        at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.11/LockSupport.java:234)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(java.base@11.0.11/AbstractQueuedSynchronizer.java:1079)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(java.base@11.0.11/AbstractQueuedSynchronizer.java:1369)
        at java.util.concurrent.Semaphore.tryAcquire(java.base@11.0.11/Semaphore.java:415)
        at org.eclipse.ui.internal.PendingSyncExec.acquire(PendingSyncExec.java:39)
        at org.eclipse.ui.internal.PendingSyncExec.waitUntilExecuted(PendingSyncExec.java:88)
        at org.eclipse.ui.internal.UISynchronizer.syncExec(UISynchronizer.java:142)
        at org.eclipse.swt.widgets.Display.syncExec(Display.java:5048)
        at org.eclipse.xtext.ui.util.DisplayRunnable.syncExec(DisplayRunnable.java:22)
        at org.eclipse.xtext.ui.editor.model.XtextDocument.fireRewriteSessionChanged(XtextDocument.java:741)
        at org.eclipse.jface.text.AbstractDocument.startRewriteSession(AbstractDocument.java:1456)
        at org.eclipse.jface.text.RewriteSessionEditProcessor.performEdits(RewriteSessionEditProcessor.java:97)
        at org.eclipse.xtext.ui.editor.model.edit.ReconcilingUnitOfWork.exec(ReconcilingUnitOfWork.java:63)
        at org.eclipse.xtext.ui.editor.model.edit.ReconcilingUnitOfWork.exec(ReconcilingUnitOfWork.java:1)
        at org.eclipse.xtext.resource.OutdatedStateManager.exec(OutdatedStateManager.java:70)
        at org.eclipse.xtext.ui.editor.model.XtextDocument$XtextDocumentLocker.modify(XtextDocument.java:432)
        - locked <0x000000075b0955f8> (a org.eclipse.xtext.resource.DerivedStateAwareResource)
        at org.eclipse.xtext.ui.editor.model.XtextDocument.internalModify(XtextDocument.java:165)
        at org.eclipse.xtext.ui.editor.model.XtextDocument.modify(XtextDocument.java:158)
        at org.irit.xtext.blocklibrary.ui.commands.CreateModelTemplateForTestCaseHandler$3.run(CreateModelTemplateForTestCaseHandler.java:89)
        at org.eclipse.core.runtime.jobs.Job$2.run(Job.java:187)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

"main" #1 prio=6 os_prio=31 cpu=70760.75ms elapsed=460.51s tid=0x00007fc495049800 nid=0x307 waiting for monitor entry  [0x00007ffeea45a000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.eclipse.xtext.ui.editor.model.XtextDocument$XtextDocumentLocker.internalReadOnly(XtextDocument.java:513)
        - waiting to lock <0x000000075b0955f8> (a org.eclipse.xtext.resource.DerivedStateAwareResource)
        at org.eclipse.xtext.ui.editor.model.XtextDocument$XtextDocumentLocker.priorityReadOnly(XtextDocument.java:489)
        at org.eclipse.xtext.ui.editor.model.XtextDocument.priorityReadOnly(XtextDocument.java:145)
        at org.irit.xtext.blocklibrary.ui.editor.BLBaselineProtectionListener.verifyText(BLBaselineProtectionListener.java:215)
        at org.irit.xtext.blocklibrary.ui.editor.BLEditorCallback$1.verifyText(BLEditorCallback.java:57)
        at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:277)
        at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
        at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4441)
        at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1512)


The Job thread owns the DerivedStateAwareResource lock and is waiting to acquire the Display lock, while the Display thread owns the Display lock and is waiting to acquire the DerivedStateAwareResource. It's the typical scenario when two threads acquire a sequence of locks in a different order.

On the Display thread we can't change the order of locking because the Display lock is necessarily already taken before the VerifyEvent is triggered.

On the Job thread we can't change the order of locking because we don't want to acquire the Display lock for the entirety of the document modification (it takes some time).

I saw that if I use a CancelableUnitOfWork on the Job side, I could use doc.priorityReadOnly() on the Display thread to cause the cancellation of the Job. However I don't think that will solve it because the stack trace shows that the Job thread is blocked during AbstractDocument.startRewriteSession() so I won't have the opportunity of checking a cancel indicator.

A last solution I'm considering is to do something like this in the event listener:

        partVerifyListener = new VerifyListener() {
            override verifyText(VerifyEvent e) {
                if (editor.document.hasModifyingUnitsOfWork()) {
                    // calling editor.document.readOnly() might cause a dead lock so
                    // display a dialog and reject the keystroke
                    e.doit = false
                    return;
                }

                val selProv = editor.selectionProvider.selection
                val currentObj = if (selProv instanceof ITextSelection) {
                        val offset = selProv.offset
                        editor.document.readOnly [
                            objAtOffsetHelper.resolveContainedElementAt(it, offset)
                        ]
               }
            }
        }


And I could subclass XTextDocument and override modify() to track when modification jobs are ongoing.

Thoughts? Ideas?

Thanks in advance!


Elie Richa, Ph.D
Software Engineer, AdaCore
https://www.adacore.com
Re: Deadlock between Job and event listener on Display thread [message #1843246 is a reply to message #1843245] Wed, 21 July 2021 11:30 Go to previous messageGo to next message
Elie Richa is currently offline Elie RichaFriend
Messages: 44
Registered: February 2016
Member
It just occurs to me that another solution would be to make that document-modifying Job prevent edits to the document being modified in some more fundamental way.

I could run the job in a modal dialog but that seems too invasive.

Ideally I would like users to be able to navigate in the UI while the modification is running in the background, and perhaps even make edits to other documents than the one being modified.


Elie Richa, Ph.D
Software Engineer, AdaCore
https://www.adacore.com
Re: Deadlock between Job and event listener on Display thread [message #1843252 is a reply to message #1843246] Wed, 21 July 2021 13:16 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 7265
Registered: July 2009
Senior Member
Hi

The Xtext concurrency is usually very simple with worker threads farmed off by the main thread, each locking out undesirable activity by an IUnitOfWork. If you have a deadlock, it would seem that either some activity is never completing or that you have cross-called such activities, or more likely as a newbie failed to observe the IUnitOfWor principles.

I suggest instrumenting IUnitOfWork to see exactly who starts and finishes what on what thread.

A possible 'deadlock' may actually be a livelock if a work thread is abused by having its iteration domain corrupted by a another thread that has neglected to invoke IUnitOfWork. Never hang on to a Resource or its content in your own state variables.

Regards

Ed Willink
Re: Deadlock between Job and event listener on Display thread [message #1843257 is a reply to message #1843252] Wed, 21 July 2021 15:29 Go to previous messageGo to next message
Elie Richa is currently offline Elie RichaFriend
Messages: 44
Registered: February 2016
Member
Hello Ed,

Thanks for the insight. I considered the points you raised but unfortunately I don't think the issue is there.

Regarding IUnitOfWork, my activities are not doing anything special or out of ordinary. I'm using the Xtext API and not defining my own IUnitOfWorks. The only particularity I see is the fact that one of them is running on the UI thread because it is part of an SWT event listener. That fact introduces a synchronization on Display.syncExec() in addition to the existing synchronization of IUnitOfWorks. The existence of two locks acquired in a different order creates the dead lock.

As for the possibility of a livelock, it's not the situation here. We can see in the stack trace that both threads are waiting on locks and not doing actual work.

I feel like a possible course of action would be to make that document-modifying Job UI-blocking somehow. I know that sometimes typing in an editor while a build job is running cause a dialog to appear with a suspended job called "Waiting User Operation". If I can make my job act this way, then perhaps I can avoid the SWT events from occurring during the job.


Elie Richa, Ph.D
Software Engineer, AdaCore
https://www.adacore.com
Re: Deadlock between Job and event listener on Display thread [message #1843260 is a reply to message #1843257] Wed, 21 July 2021 15:59 Go to previous message
Ed Willink is currently offline Ed WillinkFriend
Messages: 7265
Registered: July 2009
Senior Member
Hi

In principle anything that happens on the UI thread should be very quick and simple because anything more complicated can get unexpectedly slow and kill the whole UI. So your IUnitofWork on the UI thread is suspect. Either it should not happen at all, or it is not simple enough.

NB. something apparently trivial such as an EMF eGet() can be devastating if the underlying implementation uses delegates to some implementation language. If OCL is used, then the OCL environment will be started up and probably a nested Xtext parse or ten will happen for the OCL source.

Regards

Ed Willink
Previous Topic:Using LocalDateTime in Xtext Grammar
Next Topic:Performance issue when doing index of resourceset
Goto Forum:
  


Current Time: Sun Sep 19 13:48:27 GMT 2021

Powered by FUDForum. Page generated in 0.02805 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top