Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » OCL » Non-deterministic error with OCL validation in Xtext project(CompleteOCLEObjectValidator experiences semi-reproducible stack traces. Perhaps race condition during initialization?)
Non-deterministic error with OCL validation in Xtext project [message #1783696] Thu, 15 March 2018 16:09 Go to next message
Christian Burghard is currently offline Christian BurghardFriend
Messages: 2
Registered: March 2018
Junior Member
Hi,

I am working on a larger Xtext project in which I would like to incorporate OCL-based model validation.
The integration of OCL constraints in our validator has previously been discussed by my colleague in a previous topic.

However, when running our Eclipse product on a fresh workspace, the CompleteOCLEObjectValidator tends to fail non-deterministically.
Due to its size, sharing the whole project is not an option and I was unable to reproduce the error in a minimal example.
Nevertheless, I try to describe our problem as thoroughly as I can:

As mentioned above, the CompleteOCLEObjectValidator sometimes works but sometimes produces one of several different stack traces. The most prevalent of these stack traces (No OCL Standard Library content available) is given below. Due to this behavior I have to assume that I am dealing with some kind of race condition.

The Pivot Standalone Configuration tutorial suggests that this is a sign of the OCL standard library not being properly installed. I randomly tried several of the standalone setup commands mentioned in the tutorial but to no avail (in fact, calling OCLstdlib.install() before registering the CompleteOCLEObjectValidator seemed to only make matters worse).

Has anyone encountered similar problems when integrating OCL validation into an Xtext project?
Are there any known issues when integrating OCL with Xtext projects which were created with older versions (Xtext 2.7.2) ?

Thanks,
Chris

org.eclipse.ocl.pivot.internal.utilities.IllegalLibraryException: No OCL Standard Library content available
	at org.eclipse.ocl.pivot.internal.StandardLibraryImpl.getRequiredLibraryType(StandardLibraryImpl.java:852)
	at org.eclipse.ocl.pivot.internal.StandardLibraryImpl.resolveRequiredSimpleType(StandardLibraryImpl.java:989)
	at org.eclipse.ocl.pivot.internal.StandardLibraryImpl.getOclAnyType(StandardLibraryImpl.java:636)
	at org.eclipse.ocl.pivot.internal.manager.PivotMetamodelManager.getASmetamodel(PivotMetamodelManager.java:825)
	at org.eclipse.ocl.pivot.internal.manager.PivotMetamodelManager.getCompleteClass(PivotMetamodelManager.java:1035)
	at org.eclipse.ocl.pivot.internal.manager.PivotMetamodelManager.getPrimaryClass(PivotMetamodelManager.java:1616)
	at org.eclipse.ocl.pivot.internal.context.ClassContext.<init>(ClassContext.java:31)
	at org.eclipse.ocl.pivot.internal.manager.PivotMetamodelManager.createParserContext(PivotMetamodelManager.java:564)
	at org.eclipse.ocl.pivot.internal.manager.PivotMetamodelManager.parseSpecification(PivotMetamodelManager.java:2141)
	at org.eclipse.ocl.pivot.internal.validation.PivotEObjectValidator$ValidationAdapter.validate(PivotEObjectValidator.java:171)
	at org.eclipse.ocl.pivot.internal.validation.PivotEObjectValidator$ValidationAdapter.validate(PivotEObjectValidator.java:137)
	at org.eclipse.ocl.pivot.internal.validation.PivotEObjectValidator.validatePivot(PivotEObjectValidator.java:384)
	at org.eclipse.ocl.xtext.completeocl.validation.CompleteOCLEObjectValidator.validatePivot(CompleteOCLEObjectValidator.java:164)
	at org.eclipse.ocl.pivot.internal.validation.PivotEObjectValidator.validate(PivotEObjectValidator.java:348)
	at org.eclipse.xtext.validation.CompositeEValidator.validate(CompositeEValidator.java:151)
	at org.eclipse.emf.ecore.util.Diagnostician.doValidate(Diagnostician.java:171)
	at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:158)
	at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:137)
	at org.eclipse.xtext.validation.CancelableDiagnostician.validate(CancelableDiagnostician.java:39)
	at org.eclipse.emf.ecore.util.Diagnostician.doValidateContents(Diagnostician.java:181)
	at org.eclipse.xtext.validation.CancelableDiagnostician.doValidateContents(CancelableDiagnostician.java:70)
	at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:161)
	at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:137)
	at org.eclipse.xtext.validation.CancelableDiagnostician.validate(CancelableDiagnostician.java:39)
	at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:120)
	at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:146)
	at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:124)
	at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:90)
	at org.eclipse.xtext.ui.editor.validation.ValidationJob$1.exec(ValidationJob.java:91)
	at org.eclipse.xtext.ui.editor.validation.ValidationJob$1.exec(ValidationJob.java:1)
	at org.eclipse.xtext.util.concurrent.CancelableUnitOfWork.exec(CancelableUnitOfWork.java:26)
	at org.eclipse.xtext.resource.OutdatedStateManager.exec(OutdatedStateManager.java:91)
	at org.eclipse.xtext.ui.editor.model.XtextDocument$XtextDocumentLocker.internalReadOnly(XtextDocument.java:520)
	at org.eclipse.xtext.ui.editor.model.XtextDocument$XtextDocumentLocker.readOnly(XtextDocument.java:492)
	at org.eclipse.xtext.ui.editor.model.XtextDocument.readOnly(XtextDocument.java:133)
	at org.eclipse.xtext.ui.editor.validation.ValidationJob.createIssues(ValidationJob.java:86)
	at org.avl.mdml.core.ui.editors.MDMLValidationJob.createIssues(MDMLValidationJob.java:85)
	at org.eclipse.xtext.ui.editor.validation.ValidationJob.run(ValidationJob.java:67)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Re: Non-deterministic error with OCL validation in Xtext project [message #1783881 is a reply to message #1783696] Mon, 19 March 2018 19:31 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6040
Registered: July 2009
Senior Member
Hi

Your references to Xtext 2.7.2 make me wonder whether you are using very old code. I vaguely recall fixing an embarrassingly awful bug in CompleteOCLEObjectValidator a couple of months ago.

Using old Xtext is generally troublesome, but OCL has extra code to improve compatibility.

Narrowly OCLstdlib.install should soleve the missing library, but it should be called automatically from some higher level doSteup.

Please share the versions you use and at least your startup code.

Regards

Ed Willink
Re: Non-deterministic error with OCL validation in Xtext project [message #1783916 is a reply to message #1783696] Tue, 20 March 2018 10:44 Go to previous messageGo to next message
German Vega is currently offline German VegaFriend
Messages: 51
Registered: December 2015
Member
Hello

I have a standalone application that uses complete OCL validation, and I got very similar random crashes to yours, with exactly the same kind of exceptions (stdlib not available).

At the end (after some painful debugging) I discovered that my problem was caused by the environment factory (org.eclipse.ocl.pivot.utilities.EnvironmentFactory) used by the validator being detached in the middle of the execution.

After some more painful debugging to try to understand why it was disposed, it boiled down to the fact that the org.eclipse.ocl.pivot.utilities.OCL object used to create the EnvironmentFactory was being garbage collected, and it has a finalize method that detaches the factory.

To make a long history short, the solution was just to be sure to keep a hard reference to the OCL object used to create the factory.

Although I am not validating xtext models, maybe you have a similar problem to mine.

So I think that instead of something like this :

	public class StatesJavaValidator extends AbstractStatesJavaValidator {
		
		@Override
		public void register(EValidatorRegistrar registrar) {
			
			super.register(registrar);
			StatesPackage ePackage = StatesPackage.eINSTANCE;
			URI oclURI = URI.createPlatformResourceURI("/StatesProject/model/States.ocl", true);
		
			OCL ocl  = OCL.newInstance();
			registrar.register(ePackage, new CompleteOCLEObjectValidator(ePackage, oclURI,ocl.getEnvironmentFactory()));
		}
	}
	


You'd rather do something more like this :

	public class OCLReference {

		public static OCL ocl = OCL.newInstance();
		
		public static void dispose() {
			ocl.dispose();
		}
	}
	
	public class StatesJavaValidator extends AbstractStatesJavaValidator {
		
		
		@Override
		public void register(EValidatorRegistrar registrar) {
			
			super.register(registrar);
			StatesPackage ePackage = StatesPackage.eINSTANCE;
			URI oclURI = URI.createPlatformResourceURI("/StatesProject/model/States.ocl", true);

			registrar.register(ePackage, new CompleteOCLEObjectValidator(ePackage, oclURI, OCLReference.ocl.getEnvironmentFactory()));
		}
	}


And you explicitly dispose the OCL instance at a point in your application when you are sure you do not need validation any longer.

Hope it helps

German Vega

[Updated on: Tue, 20 March 2018 10:49]

Report message to a moderator

Re: Non-deterministic error with OCL validation in Xtext project [message #1783920 is a reply to message #1783916] Tue, 20 March 2018 12:10 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6040
Registered: July 2009
Senior Member
Hi

Thank you for sharing this pain and solution. Unfortunately what you describe is exactly the way it is intended to work. I think there are recommendations wrt OCL lifetime in the documentation already, so beefing those up won't make much difference; nobody reads documentation.

Any suggestions for improving this?

Perhaps it might be possible to detect the re-initialization of OCL post garbage collect to recommend re-use rather than churning of OCL EnvironmentFactory instances. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=532649

Regards

Ed Willink
Re: Non-deterministic error with OCL validation in Xtext project [message #1783939 is a reply to message #1783920] Tue, 20 March 2018 15:11 Go to previous messageGo to next message
German Vega is currently offline German VegaFriend
Messages: 51
Registered: December 2015
Member
Hello Ed,

Well, once I could diagnose the problem, I found that the lifecycle of the OCL instances was well documented in the source code and javadoc. But, I do not recall the documentation about CompleteOCL validation mentions it.

Concerning the bug you opened, you can imagine so many different scenarios for validation (batch/live/on-demand, by resource set, by resource, .....) that is difficult to imagine a lifecycle mechanism to suit them all. In any case, this problem mainly concern standalone users, so perhaps better documenting the issues in the programmer's guide is the only cost-effective deal.

One thing I kept wondering while I was investigating the problem was why it did not happen inside Eclipse, only in my standalone application. So I looked in the source code to see how it was handled when you do "OCl -> load document.." and discovered classes CompleteOCLLoader and OCLAdapter.

They suited perfectly well my application's needs, as they couple the lifecycle of the OCL object to the lifecycle of the resource set. So at the end I completely forgot about CompleteOCLEObjectValidator and instead used CompleteOCLLoader directly. I join the utility class I developed for performing validation, so others can get inspiration if they read this thread.

Perhaps this deserves some documentation in chapter "Unified or Pivot Programmers Guide - Validators"

However, I am not sure this can help in the scenario of "Complete OCL for Xtext Validation". I tried to look at how xtext handles the lifecycle of validators but it is too difficult to understand, as it uses guice injection. That's why I proposed this solution with the static field and the explicit dispose, even if it is far from satisfactory.

The ideal will be to dispose the OCL instance when the xtext validator is no longer required. Maybe if you or Christian Burghard, who seem more knowledgeable in xtext than me, can ask the question in their forum.

Regards

German
Re: Non-deterministic error with OCL validation in Xtext project [message #1783945 is a reply to message #1783939] Tue, 20 March 2018 15:35 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6040
Registered: July 2009
Senior Member
Hi

It is certainly best for an application to create one OCL and keep re-using it rather than relying on OCL to self-create.

Validation is challenging since the outer application may not be OCL aware. OCL therefore caches itself as an entry in the validation context, which will often match the lifecycle of declaration validity. But naive validation code may often have an outer loop that creates multiple contexts. But much better that you attach your one OCL to your one ResourceSet.

I spotted that EMF uses a concept of thread constants. Something similar might work for OCL.

Regards

Ed Willink
Re: Non-deterministic error with OCL validation in Xtext project [message #1783997 is a reply to message #1783696] Wed, 21 March 2018 11:29 Go to previous messageGo to next message
Christian Burghard is currently offline Christian BurghardFriend
Messages: 2
Registered: March 2018
Junior Member
Hi
Thank both of you for the swift reply. Although the lead provided by German Vega looked promising, it did not solve our problem (unless we are missing something).

We use the following code to initialize the OCL validator. As suggested, we created a static reference to an OCL instance.
public class OCLMDMLValidator extends MDML04Validator {

	static Logger log = Logger.getLogger(OCLMDMLValidator.class);
	static OCL ocl = OCL.newInstance();

	@Override
	public void register(EValidatorRegistrar registrar) {
		super.register(registrar);
		MDML04Package ePackage = MDML04Package.eINSTANCE;
		URI oclURI = URI.createPlatformResourceURI(
				"/org.avl.mdml.core/src/org/avl/mdml/mydsl/validation/MDML.ocl",
				true);
		registrar.register(ePackage,
				new CompleteOCLEObjectValidator(ePackage, oclURI,
						ocl.getEnvironmentFactory()));
	}
}


To give you an idea about the versions we use, I attach our target platform definition and the manifest of the main project.

Regards,
Chris
Re: Non-deterministic error with OCL validation in Xtext project [message #1783998 is a reply to message #1783997] Wed, 21 March 2018 12:09 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6040
Registered: July 2009
Senior Member
Hi

I don't see anything particularly concerning in your versions, except that by using Neon you miss 18 months of developments. (You also need to choose between the Neon and Neon++ releases to workaround a breaking Xtext API change.)

Using https://wiki.eclipse.org/OCL/New_and_Noteworthy/Oxygen to refresh my memory, the following might be very relevant
https://bugs.eclipse.org/484231Repeated Complete OCL validations are no longer ignored
many of the other fixes might contribute to a better diagnosis of something wrong with your OCL.

You can confirm that you only use one OCL instance by setting a breakpoint on the MetmodelManager constructor. Multiple instances are usually unnecessary, inefficient and an indication that the actual control flow is not as intended.

If your models evolve after you start validating, you may well detect that OCL's ability to keep its derived Pivot representations in sync with the primary Ecore/UML is poor; awaiting auto-generation from QVTr.

Even if you don't deliberately change your models, the models, particularly UML models, might evolve by lazy proxy resolution. An EcoreUtil.resolveAll prior to validation might help.

Regards

Ed Willink
Re: Non-deterministic error with OCL validation in Xtext project [message #1784063 is a reply to message #1783998] Thu, 22 March 2018 08:29 Go to previous messageGo to next message
German Vega is currently offline German VegaFriend
Messages: 51
Registered: December 2015
Member
Hello Ed, Christian

I think that the initial hunch of Christina that it is a race condition was right.

I did some tests. I created a xtext DSL (the hello world example), and started a new workspace in which I opened three model files in three different editors.

Then I put a breakpoint in CompleteOCLEObjectValidator.initialize and restarted the workspace. The initialization of each of three editors triggered OCL validation in different working threads.

All tried to initialize simultaneously the same CompleteOCLEObjectValidator as shown in this screenshot

index.php/fa/32401/0/

Depending on the interleaving of the initialization, some times it works, sometimes I got similar kind of exceptions (no ocl stdlib) as Christian has, as shown here

index.php/fa/32402/0/

the exception stack is not exactly the same as Christians, but I think that the CompleteOCLEObjectValidator is not really designed to be thread-safe, so it can happen in other situations.

I used neon with xtext 2.10 for testing
index.php/fa/32403/0/

Hope this helps clarifying this issue
Re: Non-deterministic error with OCL validation in Xtext project [message #1784071 is a reply to message #1784063] Thu, 22 March 2018 09:22 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6040
Registered: July 2009
Senior Member
Hi

The current status is that the Pivot OCL code is intended to be thread safe, but I will not claim thread safety until there is a level of auto-generation of shared state that gets close to guaranteeing thread safety. IMHO manual best endeavours is not an adequate design approach. NB EMF is not thread safe.

Could you clarify what three files you had open. I would expect three editors to be independent and so have no sharing. Perhaps Xtext is introducing some sharing that I have not considered.

Regards

Ed Willink

Re: Non-deterministic error with OCL validation in Xtext project [message #1784083 is a reply to message #1784071] Thu, 22 March 2018 10:35 Go to previous messageGo to next message
German Vega is currently offline German VegaFriend
Messages: 51
Registered: December 2015
Member
Hi Ed,

The problem is not really that xtext editors are introducing some hidden sharing, the bottom line is that all are using the same singleton EMF's EValidator.Registry.INSTANCE for performing validation.

So xtext registers the CompleteOCLEObjectValidator in the validation chain of EValidator.Registry.INSTANCE, and from then on it will be implicitly shared by any application (not only xtext editors) that uses the standard org.eclipse.emf.ecore.util.Diagnostician

Because initialization of CompleteOCLEObjectValidator is lazy, it can really happen in any thread (depending on how it is triggered : from a menu, from a builder, ..... ) and sometimes it can happen concurrently in several threads.

The scenario of having three opened files (you can see them at the right of the first screenshot test.mydsl test2.mydsl test3.mydsl) was just a way to systematically reproduce the problem. The files had nothing in particular.

The steps to reproduce the scenario are the following :

1) define your xtext DSL and add complete OCL validation
2) start a debugged eclipse with your DSL plugins
3) edit several DSL model files in different editors
4) let the editors opened and exit the debugged eclipse
5) put a breakpoint in CompleteOCLEObjectValidator.initialize
6) restart the debugged eclipse
7) the first thing each editor will do on reopening is try to perform a validation in a working thread, and you will get into the situation of the first screenshot

I agree with you that EMF itself is not thread safe, but validation needs just read-only access to the model so it should work .. the problem is all the side-effects that are not really read-only (loading of resources, CS2AS , calculation of the complete OCL model, ...)

I know you really did a great effort and wonderful job to make OCL work in isolated resource sets and as transparently as possible... but there are so many singletons in EMF that multi-thread sharing is almost inevitable ...
Re: Non-deterministic error with OCL validation in Xtext project [message #1784092 is a reply to message #1784083] Thu, 22 March 2018 12:10 Go to previous message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6040
Registered: July 2009
Senior Member
Hi

https://bugs.eclipse.org/bugs/show_bug.cgi?id=532757 raised

The need to share the single global EValidator.Registry.INSTANCE is a major pain, unlike the SettingDelegate etc registries that are per-ResourceSet delegating to global. I think there's an EMF WONTFIX somewhere. The workaround is supposed to use a per-ResourceSet sub-delegation of the single entry. Hopefully it's just a one-line bug in my best endeavors synchronization.

Regards

Ed Willink
Previous Topic:def / derive in interactive OCL console
Next Topic:Generic Enumeration Literal
Goto Forum:
  


Current Time: Sat Sep 22 17:09:11 GMT 2018

Powered by FUDForum. Page generated in 0.03488 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top