Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » VIATRA » ViatraQuery - Synchronisation of Threads
ViatraQuery - Synchronisation of Threads [message #1760549] Fri, 28 April 2017 08:45 Go to next message
Jakob Cammisar is currently offline Jakob CammisarFriend
Messages: 5
Registered: May 2015
Junior Member
Hi everyone,

firstly, thank you for creating and maintaining the Viatra Framework.

I am currently using ViatraQuery in a Project as a way to access the data and create Observables for the UI.

The used technologies include:
EMF version 2.11
CDO version 4.3.0
ViatraQuery version 1.5.0

ViatraQuery is given a global CDOView to get access to the database and create its network.
An Import is triggered during the startup of the Application, but sometimes an exception is thrown by the ViatraQueryEngine:

CDOViewInvalidationRunner-View 1 FATAL ViatraQueryEngine.1867072584 - VIATRA Query encountered an error in processing the EMF model. This happened while trying to add the object: Project@OID407
 java.util.NoSuchElementException
 at java.util.ArrayDeque.removeFirst(ArrayDeque.java:280)
 at org.eclipse.viatra.query.runtime.rete.network.ReteContainer.messageConsumptionSingleThreaded(ReteContainer.java:460)
 at org.eclipse.viatra.query.runtime.rete.network.Network.waitForReteTermination(Network.java:352)
 at org.eclipse.viatra.query.runtime.rete.boundary.ExternalInputEnumeratorNode.update(ExternalInputEnumeratorNode.java:99)
 at org.eclipse.viatra.query.runtime.emf.EMFQueryRuntimeContext$EStructuralFeatureInstancesKeyAdapter.featureInserted(EMFQueryRuntimeContext.java:588)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperImpl.notifyFeatureListeners(NavigationHelperImpl.java:752)
 at org.eclipse.viatra.query.runtime.base.core.EMFBaseIndexInstanceStore.insertFeatureTuple(EMFBaseIndexInstanceStore.java:198)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperVisitor.visitReference(NavigationHelperVisitor.java:358)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperVisitor.visitInternalContainment(NavigationHelperVisitor.java:342)
 at org.eclipse.viatra.query.runtime.base.comprehension.EMFModelComprehension.traverseFeatureInternal(EMFModelComprehension.java:276)
 at org.eclipse.viatra.query.runtime.base.comprehension.EMFModelComprehension.traverseFeatureTargets(EMFModelComprehension.java:216)
 at org.eclipse.viatra.query.runtime.base.comprehension.EMFModelComprehension.traverseObject(EMFModelComprehension.java:202)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter$2.call(NavigationHelperContentAdapter.java:207)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter$2.call(NavigationHelperContentAdapter.java:1)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperImpl.coalesceTraversals(NavigationHelperImpl.java:1216)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter.addAdapter(NavigationHelperContentAdapter.java:201)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter.handleContainment(NavigationHelperContentAdapter.java:490)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter.selfAdapt(NavigationHelperContentAdapter.java:425)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter.simpleNotifyChanged(NavigationHelperContentAdapter.java:369)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter$1.call(NavigationHelperContentAdapter.java:90)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter$1.call(NavigationHelperContentAdapter.java:1)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperImpl.coalesceTraversals(NavigationHelperImpl.java:1233)
 at org.eclipse.viatra.query.runtime.base.core.NavigationHelperContentAdapter.notifyChanged(NavigationHelperContentAdapter.java:87)
 at org.eclipse.emf.common.notify.impl.BasicNotifierImpl.eNotify(BasicNotifierImpl.java:374)
 at org.eclipse.emf.common.notify.impl.NotificationChainImpl.dispatch(NotificationChainImpl.java:98)
 at org.eclipse.emf.common.notify.impl.NotificationChainImpl.dispatch(NotificationChainImpl.java:86)
 at org.eclipse.emf.internal.cdo.view.CDOViewImpl.sendDeltaNotifications(CDOViewImpl.java:1016)
 at org.eclipse.emf.internal.cdo.view.CDOViewImpl.doInvalidate(CDOViewImpl.java:879)
 at org.eclipse.emf.internal.cdo.view.CDOViewImpl$InvalidationRunnable.run(CDOViewImpl.java:1710)
 at org.eclipse.net4j.util.concurrent.QueueRunner.work(QueueRunner.java:26)
 at org.eclipse.net4j.util.concurrent.QueueRunner.work(QueueRunner.java:1)
 at org.eclipse.net4j.util.concurrent.QueueWorker.doWork(QueueWorker.java:78)
 at org.eclipse.net4j.util.concurrent.QueueWorker.work(QueueWorker.java:70)
 at org.eclipse.net4j.util.concurrent.Worker$WorkerThread.run(Worker.java:209)
 


After some research I found the problem:
During the import of the data a check is performed to test whether the data is already in the DB or it needs to be added. So a Matcher from a Pattern is used to check the existence of the data.
If it's not existing, a runnable will be submitted to a SingleThreadedExecutor to add the data and not block the import Thread.

Now 2 Threads are actively using ViatraQuery. One uses the Matcher-Classes to retrieve data and one adds data to the database, therefore triggering update notifications to the ViatraQueryRuntime. At some point, both Threads are in the ReteContainer.messageConsumptionSingleThreaded - method and the exception above will be thrown.

Thankfully the exception is reproducable. It happens regularly during the import and can be mitigated with the use of CDO and its synchronization. But now the question pops up whether this situation can happen during the active use of the application. Multiple Users will acces the data and create/ alter/ delete objects and, from my point of view, this can cause this to happen.

I noticed an experimental feature while looking through the code of the ReteContainer: the reteThreads, a multithreaded approach to the message consumption.
For testing purposes I activated the feature and it solves the problem and works for normal queries of the Matchers.
But now, Observables created via ViatraObservables.observeMatchesAsList seem to stop reacting to removals/ additions.


To get to the Point:

1.) Could this situation happen during a multiuser/ multithread use of ViatraQuery?
2.) If so, is there a way to synchronise/ increase the robustness of the ViatraQueryRuntime under the assumption that multiple Threads use it simultaneously?
3.) What is the status of the experimental feature? It solves Question 2 and is a really nice and elegant way to handle multithreading.


Best Regards,

Jakob
Re: ViatraQuery - Synchronisation of Threads [message #1760552 is a reply to message #1760549] Fri, 28 April 2017 09:01 Go to previous messageGo to next message
Abel Hegedus is currently offline Abel HegedusFriend
Messages: 187
Registered: September 2015
Senior Member
Quote:
1.) Could this situation happen during a multiuser/ multithread use of ViatraQuery?
2.) If so, is there a way to synchronise/ increase the robustness of the ViatraQueryRuntime under the assumption that multiple Threads use it simultaneously?
3.) What is the status of the experimental feature? It solves Question 2 and is a really nice and elegant way to handle multithreading.


1. VIATRA is single threaded, as EMF itself is also single threaded (at least on the base level of EObject implementations). Usually, when multiple users/applications can access and modify a single EMF model, you should use a transactional editing domain that enforces modifications to be executed in transactions and to execute transactions one-by-one.

2. The feature that you found is not related to supporting multi threaded access to the Rete network, it is an experimental feature to process changes on multiple threads internally (when delivering Rete messages in the network), but the changes still have to arrive from single thread.

3. I think it was found that it does not bring any true performance improvements and as discussed, does not provide thread safety on the VIATRA Query framework level.

To sum up, make sure you only access a VIATRA Query engine from a single thread at the same time. You may get additional details from other responses. Also, we are happy to answer any further questions.
Re: ViatraQuery - Synchronisation of Threads [message #1760564 is a reply to message #1760552] Fri, 28 April 2017 11:28 Go to previous message
Jakob Cammisar is currently offline Jakob CammisarFriend
Messages: 5
Registered: May 2015
Junior Member
Hi Abel,

Thank you for your very quick response.
I think you answered my question already. Since VIATRA is single threaded, there is no other way than to synchronize the access on the application level and to make sure, only one Thread accesses the engine at a time, like you said.

Thank you very much,

Jakob
Previous Topic:VIATRA - Query Generic API
Next Topic:@OrderBy using an Integer parameter
Goto Forum:
  


Current Time: Fri Sep 22 13:46:01 GMT 2017

Powered by FUDForum. Page generated in 0.03489 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software