Eclipse Community Forums: TMF (Xtext) » Infinite Loop in DocumentTokenSource.computeDamageRegion

Help

Home

Home » Modeling » TMF (Xtext) » Infinite Loop in DocumentTokenSource.computeDamageRegion

Show: Today's Messages :: Show Polls :: Message Navigator

Infinite Loop in DocumentTokenSource.computeDamageRegion [message #1387792]

Wed, 25 June 2014 15:44

Marcus Höpfner

Messages: 53
Registered: February 2014

Member

Hi,

I changed my grammar for a mathematical expression a little bit and now I get an infinite loop.

Here is the exception:

java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:2760)
	at java.util.Arrays.copyOf(Arrays.java:2734)
	at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
	at java.util.ArrayList.add(ArrayList.java:370)
	at org.eclipse.xtext.ui.editor.model.DocumentTokenSource.computeDamageRegion(DocumentTokenSource.java:306)
	at org.eclipse.xtext.ui.editor.model.DocumentTokenSource.updateStructure(DocumentTokenSource.java:245)
	at org.eclipse.xtext.ui.editor.model.XtextDocument.fireDocumentChanged(XtextDocument.java:363)
	at org.eclipse.jface.text.AbstractDocument.replace(AbstractDocument.java:1191)
	at org.eclipse.jface.text.AbstractDocument.replace(AbstractDocument.java:1210)
	at org.eclipse.jface.text.projection.ProjectionTextStore.replace(ProjectionTextStore.java:111)
	at org.eclipse.jface.text.AbstractDocument.replace(AbstractDocument.java:1184)
	at org.eclipse.jface.text.AbstractDocument.replace(AbstractDocument.java:1210)
	at org.eclipse.jface.text.projection.ProjectionDocument.replace(ProjectionDocument.java:629)
	at org.eclipse.jface.text.DefaultDocumentAdapter.replaceTextRange(DefaultDocumentAdapter.java:248)
	at org.eclipse.swt.custom.StyledText.modifyContent(StyledText.java:7198)
	at org.eclipse.swt.custom.StyledText.sendKeyEvent(StyledText.java:8030)
	at org.eclipse.swt.custom.StyledText.doContent(StyledText.java:2474)
	at org.eclipse.swt.custom.StyledText.handleKey(StyledText.java:5928)
	at org.eclipse.swt.custom.StyledText.handleKeyDown(StyledText.java:5959)
	at org.eclipse.swt.custom.StyledText$7.handleEvent(StyledText.java:5636)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1057)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1081)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1066)
	at org.eclipse.swt.widgets.Widget.sendKeyEvent(Widget.java:1108)
	at org.eclipse.swt.widgets.Widget.sendKeyEvent(Widget.java:1104)
	at org.eclipse.swt.widgets.Widget.wmChar(Widget.java:1525)
	at org.eclipse.swt.widgets.Control.WM_CHAR(Control.java:4723)
	at org.eclipse.swt.widgets.Canvas.WM_CHAR(Canvas.java:344)
	at org.eclipse.swt.widgets.Control.windowProc(Control.java:4611)
	at org.eclipse.swt.widgets.Canvas.windowProc(Canvas.java:340)
	at org.eclipse.swt.widgets.Display.windowProc(Display.java:4977)

My grammar:

grammar com.sap.bw.qd.formula.QueryFormula /*with org.eclipse.xtext.common.Terminals*/ hidden(WS)  

import "platform:/resource/com.sap.bw.qd.model/model/query.ecore"
import "http://www.eclipse.org/emf/2003/XMLType" as type
import "http://www.w3.org/2005/Atom" as atom
import "platform:/resource/com.sap.bw.models/bwcore/bwcore.ecore" as bwcore
import "http://www.eclipse.org/emf/2002/Ecore" as ecore

// TODO: decide whether the  operations (from back end) which are not a function should be written 
//       as prefix (as currently) or better infix with one operand (like in the old query designer)
// TODO: operators used directly as code instead of referencing parser rule, otherwise serialization does not work properly
// refer to http://www.eclipse.org/forums/index.php/m/1273046/#msg_1273046
FormulaDefinition returns FormulaDefinition:
	{FormulaDefinition}
	formulaToken=FormulaToken?;

terminal WS:
	(' ' | '\t' | '\r' | '\n')+;

FormulaToken returns FormulaToken:
	Prio1SignOperation | Prio1Operation;

Prio1Operation returns FormulaToken: // like +
	Prio2Operation ({FormulaInfixOperator.childToken+=current} code=('+' | '-' | 'OR' | 'XOR')
	childToken+=Prio2Operation)*;

Prio2Operation returns FormulaToken: // like *
	Prio3Operation ({FormulaInfixOperator.childToken+=current} code=('%' | '%A' | '%_A' | '*' | '/' | 'AND' | 'DIV' |
	'MOD') childToken+=Prio3Operation)*;

Prio3Operation returns FormulaToken: // like ==
	InfixOperandIncludeSignOperation ({FormulaInfixOperator.childToken+=current} code=('**' | '<' | '<=' | '<>' | '==' |
	'>' | '>=') childToken+=InfixOperandIncludeSignOperation)*;

InfixOperandIncludeSignOperation returns FormulaToken:
	InfixOperand | Prio5SignOperation;

InfixOperand returns FormulaToken:
	'(' FormulaToken ')' | FormulaMemberOperand | FormulaConstant | PrefixOperation1Operand | PrefixOperation2Operands;

Prio1SignOperation returns FormulaToken:
	{FormulaSignOperator} code=('+' | '=') childToken+=Prio1Operation; // sign operation and prio1operation have the same priority

Prio5SignOperation returns FormulaToken:
	{FormulaSignOperator} code=('-') childToken+=InfixOperand;

PrefixOperation2Operands returns FormulaPrefixOperator:
	code=('MAX' | 'MIN') '(' childToken+=FormulaToken ';' childToken+=FormulaToken ')';

PrefixOperation1Operand returns FormulaPrefixOperator:
	code=('NOT' | '%CT' | '%GT' | '%RT' | '%XT' | '%YT' | 'ABS' | 'ACOS' | 'ASIN' | 'ATAN' | 'CEIL' | 'CMR' | 'COS' | 'COSH' |
	'COUNT' | 'DATE' | 'DELTA' | 'EXP' | 'FIX' | 'FLOOR' | 'FRAC' | 'ISNUL' | 'LEAF' | 'LOG' | 'LOG10' | 'MAX0' | 'MIN0' | 'NDIV0'
	| 'NODIM' | 'NOERR' | 'SCAL' | 'SIGN' | 'SIN' | 'SINH' | 'SQRT' | 'SUMCT' | 'SUMGT' | 'SUMRT' | 'TAN' | 'TANH' |
	'TIME' | 'TRUNC' | 'VECT') '(' childToken+=FormulaToken ')';

FormulaConstant returns FormulaConstant:
	value=FormulaConstantValue;

/*Id must be "[/S1/0AMOUNT] Amount" or "[/C2/HELPCELL] My Cell" or "[/F1/FORMULA1] My Formula" (done by ScopeProvider)*/
FormulaMemberOperand returns FormulaMemberOperand: 
	member=[Member|STRING];

FormulaConstantValue returns type::Double:
	DOUBLE;

/* this is the parent of formulaDefinition, for some reasons the serializer needs that rule */
MemberFormula returns MemberFormula:
	formulaDefinition=FormulaDefinition;

terminal DOUBLE returns ecore::EDouble:
	('0'..'9')+ (('.' | ',') ('0'..'9')+)?; // double or int 

terminal STRING	: 
			'"' ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|'"') )* '"'
		; 
	
terminal PREFIX_NUMERICAL_OPERATOR_TYPE:
	(('A'..'Z') |'%')*	;

This is the part I have added (in preparation for some further changes!

terminal PREFIX_NUMERICAL_OPERATOR_TYPE:
	(('A'..'Z') |'%')*	;

Why am I doing this? I want to reduce the list of keywords. So I want to replace all the keywords in rule PrefixOperation1Operand (code=...) by PREFIX_NUMERICAL_OPERATOR_TYPE.
I have heard that is better to have less keywords. I would provider syntax check , content assist and value converter then for the keywords.

Why does it end in infinite loop?

BTW: Here's my MWE file which I stripped down to what I need:

module com.sap.bw.qd.formula.QueryFormula

import org.eclipse.emf.mwe.utils.*
import org.eclipse.xtext.generator.*
import org.eclipse.xtext.ui.generator.*

var grammarURI = "classpath:/com/sap/bw/qd/formula/QueryFormula.xtext"
var fileExtensions = "qf"
var projectName = "com.sap.bw.qd.formula"
var runtimeProject = "../${projectName}"
var generateXtendStub = true

Workflow {
	bean = StandaloneSetup {
		scanClassPath  = true
		platformUri = "${runtimeProject}/.."
		registerGeneratedEPackage = "com.sap.bw.qd.model.query.QueryPackage"
		registerGenModelFile = "platform:/resource/com.sap.bw.qd.model/model/query.genmodel"
	}

	component = DirectoryCleaner {
		directory = "${runtimeProject}/src-gen"
	}

	component = DirectoryCleaner {
		directory = "${runtimeProject}.ui/src-gen"
	}

	component = Generator {
		pathRtProject = runtimeProject
		pathUiProject = "${runtimeProject}.ui"
		pathTestProject = "${runtimeProject}.tests"
		projectNameRt = projectName
		projectNameUi = "${projectName}.ui"
		language = auto-inject {
			uri = grammarURI

			// Java API to access grammar elements (required by several other fragments)
			fragment = grammarAccess.GrammarAccessFragment auto-inject {}

			// the Ecore2Xtext specific terminal converter
			fragment = ecore2xtext.Ecore2XtextValueConverterServiceFragment auto-inject {}

			// serializer 2.0
			fragment = serializer.SerializerFragment auto-inject {
				//generateStub = false
			}

			// a custom ResourceFactory for use with EMF 
			fragment = resourceFactory.ResourceFactoryFragment auto-inject {}

			// the Antlr parser
			fragment = parser.antlr.XtextAntlrGeneratorFragment auto-inject {
				options = {
					classSplitting = true
				}
			}

			// Xtend-based API for validation 
			fragment = validation.ValidatorFragment auto-inject {
			}

			// scoping and exporting API
			fragment = scoping.ImportURIScopingFragment auto-inject {
				generateXtendStub = false // we have a java class instead
			}
			fragment = exporting.SimpleNamesFragment auto-inject {}

			// formatter API 
			// fragment = formatting.FormatterFragment auto-inject {}
			fragment = ecore2xtext.FormatterFragment auto-inject {}

			// labeling API 
			fragment = labeling.LabelProviderFragment auto-inject {}
			
			//content assist API 
			fragment = contentAssist.ContentAssistFragment auto-inject {}

			// antlr parser generator tailored for content assist 
			fragment = parser.antlr.XtextAntlrUiGeneratorFragment auto-inject {
				options = {
					classSplitting = true
				}
			}
			
			// generates junit test support classes into Generator#pathTestProject
			fragment = junit.Junit4Fragment auto-inject {}
		}
	}
}

Thanks, Marcus

Report message to a moderator

Re: Infinite Loop in DocumentTokenSource.computeDamageRegion [message #1387983 is a reply to message #1387792]

Wed, 25 June 2014 21:48

Zoltan Ujhelyi

Messages: 360
Registered: July 2015

Senior Member

Hi,

I guess you might have introduced a non-deterministic case with regards to the keywords of the language: if your grammar uses the word 'MAX' or 'MIN' it could be either the keywords used in the rule
PrefixOperation2Operands or a newly introduced terminal, and that causes some issues during editing.

I am not entirely sure this is the issue, but it makes sense as before the parser runs, a lexer is used to identify these terminals, where no context information is available. This means, a single terminal/keyword should not conflict.

Additionally, I would check the output of the mwe execution: usually, some problems are only shown during generation instead of in the Xtext grammar editor. If you have no experience with grammars, they can be a bit cryptic, but they usually help determining the issue.

I hope I have helped a bit. Cheers,
Zoltán

Report message to a moderator

Re: Infinite Loop in DocumentTokenSource.computeDamageRegion [message #1388279 is a reply to message #1387983]

Thu, 26 June 2014 07:56

Marcus Höpfner

Messages: 53
Registered: February 2014

Member

Hi,

you are right.
So I changed the grammar and applied my planned change completely. It still hangs with infinite loop in DocumentTokenSource.computeDamageRegion. OutOfMemoryException is not thrown anymore.

MWE execution is fine. No errors.
Here is the grammar:

grammar com.sap.bw.qd.formula.QueryFormula /*with org.eclipse.xtext.common.Terminals*/ hidden(WS)  

import "platform:/resource/com.sap.bw.qd.model/model/query.ecore"
import "http://www.eclipse.org/emf/2003/XMLType" as type
import "http://www.w3.org/2005/Atom" as atom
import "platform:/resource/com.sap.bw.models/bwcore/bwcore.ecore" as bwcore
import "http://www.eclipse.org/emf/2002/Ecore" as ecore

// TODO: decide whether the  operations (from back end) which are not a function should be written 
//       as prefix (as currently) or better infix with one operand (like in the old query designer)
// TODO: operators used directly as code instead of referencing parser rule, otherwise serialization does not work properly
// refer to http://www.eclipse.org/forums/index.php/m/1273046/#msg_1273046
FormulaDefinition returns FormulaDefinition:
	{FormulaDefinition}
	formulaToken=FormulaToken?;

terminal WS:
	(' ' | '\t' | '\r' | '\n')+;

FormulaToken returns FormulaToken:
	Prio1SignOperation | Prio1Operation;

Prio1Operation returns FormulaToken: // like +
	Prio2Operation ({FormulaInfixOperator.childToken+=current} code=('+' | '-' | 'OR' | 'XOR')
	childToken+=Prio2Operation)*;

Prio2Operation returns FormulaToken: // like *
	Prio3Operation ({FormulaInfixOperator.childToken+=current} code=('%' | '%A' | '%_A' | '*' | '/' | 'AND' | 'DIV' |
	'MOD') childToken+=Prio3Operation)*;

Prio3Operation returns FormulaToken: // like ==
	InfixOperandIncludeSignOperation ({FormulaInfixOperator.childToken+=current} code=('**' | '<' | '<=' | '<>' | '==' |
	'>' | '>=') childToken+=InfixOperandIncludeSignOperation)*;

InfixOperandIncludeSignOperation returns FormulaToken:
	InfixOperand | Prio5SignOperation;

InfixOperand returns FormulaToken:
	'(' FormulaToken ')' | FormulaMemberOperand | FormulaConstant | PrefixOperation;

Prio1SignOperation returns FormulaToken:
	{FormulaSignOperator} code=('+' | '=') childToken+=Prio1Operation; // sign operation and prio1operation have the same priority

Prio5SignOperation returns FormulaToken:
	{FormulaSignOperator} code=('-') childToken+=InfixOperand;

PrefixOperation returns FormulaPrefixOperator:
	code=PREFIX_NUMERICAL_OPERATOR_TYPE '(' childToken+=FormulaToken (';' childToken+=FormulaToken)* ')';

FormulaConstant returns FormulaConstant:
	value=FormulaConstantValue;

/*Id must be "[/S1/0AMOUNT] Amount" or "[/C2/HELPCELL] My Cell" or "[/F1/FORMULA1] My Formula" (done by ScopeProvider)*/
FormulaMemberOperand returns FormulaMemberOperand: 
	member=[Member|STRING];

FormulaConstantValue returns type::Double:
	DOUBLE;

/* this is the parent of formulaDefinition, for some reasons the serializer needs that rule */
MemberFormula returns MemberFormula:
	formulaDefinition=FormulaDefinition;

terminal DOUBLE returns ecore::EDouble:
	('0'..'9')+ (('.' | ',') ('0'..'9')+)?; // double or int 

terminal STRING	: 
			'"' ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|'"') )* '"'
		; 
	
terminal PREFIX_NUMERICAL_OPERATOR_TYPE:
	(('A'..'Z') |'%')*	;

Report message to a moderator

Re: Infinite Loop in DocumentTokenSource.computeDamageRegion [message #1390007 is a reply to message #1388279]

Sat, 28 June 2014 19:06

Marcus Höpfner

Messages: 53
Registered: February 2014

Member

Sorry, forgot to mention where it hangs..
Entered "3+8" and afterwards " MAX".
Sometimes it works and hangs a little later only. E.g. after backspace a few times and " MAX" again.

Report message to a moderator

Previous Topic:	Using ANTLR or Xtext for YAML editor
Next Topic:	fullyQualifiedName is null when loading Resource from URI

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Tue Jan 21 12:19:51 GMT 2020

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter