Eclipse Community Forums: TMF (Xtext) » Handling semantic whitespace: forcefully avoiding it

Home » Modeling » TMF (Xtext) » Handling semantic whitespace: forcefully avoiding it('identifier$' shall be parsed whilst 'identifier $' shall not)

Show: Today's Messages :: Show Polls :: Message Navigator

Handling semantic whitespace: forcefully avoiding it [message #1768792]

Mon, 24 July 2017 00:12

David Black

Messages: 33
Registered: June 2017

Member

Hi,

I'm trying to apply what I've read in the official docs and on this forum to handle semantic whitespace, but I'm not able to make it work.

My case is simple: accepting strings like "variable_name$" and rejecting strings like "variable_name $". That is: there cannot be any blank between a variable's name and the "$" postfix operator.

(In reality, "variable_name" is a complex datatype rule, not a terminal, in my real language. I'm assuming it's a terminal here just for demo purposes.)

My demo-purposes grammar looks like this:

grammar org.xtext.example.mydsl.MyDsl
import "http://www.eclipse.org/emf/2002/Ecore" as ecore
generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"

Model hidden(WS):
	source+=Expression*;
Expression hidden():
	ID '$';
terminal ID  		: '^'?('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*;
terminal WS			: (' '|'\t'|'\r'|'\n')+;

But that grammar accepts e.g. both "variable1$" and "variable2 $", what is not what I want (accepting 'variable1$' is OK, but 'variable2 $' is illegal because it contains blanks between the identifier and the operator). And I don't know why.

Can anyone point out what I'm doing wrong?

Thanks in advance.

David

[Updated on: Mon, 24 July 2017 00:13]

Report message to a moderator

Re: Handling semantic whitespace: forcefully avoiding it [message #1768795 is a reply to message #1768792]

Mon, 24 July 2017 03:29

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

Hi,

i cannot reproduce this.
am getting

[XtextSyntaxDiagnostic: null:2 extraneous input ' ' expecting '$']

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Handling semantic whitespace: forcefully avoiding it [message #1768813 is a reply to message #1768795]

Mon, 24 July 2017 06:38

David Black

Messages: 33
Registered: June 2017

Member

Wow, that's interesting, Christian.

Maybe I am not checking it rightly?

The unit test I am submitting my grammar to is this:

/*
 * generated by Xtext 2.10.0
 */
package org.xtext.example.mydsl.tests

import com.google.inject.Inject
import org.eclipse.xtext.junit4.InjectWith
import org.eclipse.xtext.junit4.XtextRunner
import org.eclipse.xtext.junit4.util.ParseHelper
import org.eclipse.xtext.util.EmfFormatter
import org.junit.Assert
import org.junit.Test
import org.junit.runner.RunWith
import org.xtext.example.mydsl.myDsl.Model

@RunWith(XtextRunner)
@InjectWith(MyDslInjectorProvider)
class MyDslParsingTest{

	@Inject
	ParseHelper<Model> parseHelper

	@Test 
	def void loadModel() {
		val parsedModel = parseHelper.parse('''
			variable1$ variable2  $
		''')
		val parsedModelString = EmfFormatter.objToStr(parsedModel)
		println(parsedModelString)
		Assert.assertNotNull(parsedModel)
	}

}

And the output of the .objToStr method after that test execution is this:

Model {
    attr EString source [
        0: 'variable1$'
        1: 'variable2 $'
    ]
}

Could you please suggest any other/further checks to approach this issue?

Thanks in advance,

David

Report message to a moderator

Re: Handling semantic whitespace: forcefully avoiding it [message #1768815 is a reply to message #1768813]

Mon, 24 July 2017 06:58

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

hi,

i test this way:

@Test
def void loadModel() {
val result = parseHelper.parse('''
variable1$ variable2 $
''')
Assert.assertNotNull(result)
println(result.eResource.errors)
Assert.assertTrue(result.eResource.errors.isEmpty)
}

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator