Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Problem with ID Terminal Rule
Problem with ID Terminal Rule [message #1782074] Fri, 16 February 2018 18:44 Go to next message
Greg Lusk is currently offline Greg LuskFriend
Messages: 36
Registered: February 2017
Location: USA
Member
I have a problem identical to the one described in this thread:
https://www.eclipse.org/forums/index.php/t/166300/

I have solved it in exactly the way that Sven prescribed, however the solution creates another problem: the ID terminal seems to be treated as a generic string, allowing characters that my grammar considers illegal. Here is a relevant snippet from my grammar:


Variable_Primary returns PrimaryVariable:
	name=Variable_Name
	
;

Variable_Name: 
	ID | 'E' | 'e'
;

Real_Value returns ecore::EDouble:
	Signed_Int '.' UNSIGNED_INT ('E' Signed_Int)?
;

terminal UNSIGNED_INT returns ecore::ELong:
	DIGIT ('_'? DIGIT)*
;

terminal fragment LETTER: 'a'..'z' | 'A'..'Z' | '_';
terminal fragment DIGIT: '0'..'9';

terminal ID:
 	DIGIT? LETTER (LETTER | DIGIT)* 	
;




The requirements for the grammar are as follows:
1. The name feature of Variable_Primary cannot include parentheses
2. The name feature of Variable_Primary can be the string literal 'E'
3. The value 1.23E-1 should match the Real_Value data type rule.

With the grammar stated above, requirement #1 is not satisfied, but requirements 2 and 3 are satisfied. Can anyone recommend a way to fulfill all three requirements?
Re: Problem with ID Terminal Rule [message #1782079 is a reply to message #1782074] Fri, 16 February 2018 19:32 Go to previous messageGo to next message
Márcio Koch is currently offline Márcio KochFriend
Messages: 57
Registered: August 2013
Member
Sorry, I cannot see where rule 1# is infringed (include parentheses). But one thing that is possible to do when is not possible to do via the grammer is to use the validation approach.
Some thing like:
@Check
def checkVariableName(Variable_Primary toCheck) {
  if (toCheck.hasParentheses) {
      error('The name cannot has parentheses.', toCheck, ...)
  }
}
Re: Problem with ID Terminal Rule [message #1782081 is a reply to message #1782079] Fri, 16 February 2018 19:45 Go to previous messageGo to next message
Greg Lusk is currently offline Greg LuskFriend
Messages: 36
Registered: February 2017
Location: USA
Member
Thanks for the response. This should really be a parser rule, though. The detection of invalid characters should go there by design.

From my grammar, it is not evident to me, either, why a name like Y(x) passes through the parser and is not considered to be illegal.
Re: Problem with ID Terminal Rule [message #1782082 is a reply to message #1782079] Fri, 16 February 2018 19:47 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
can you share a complete grammar and parsing test from what you've posted i can only guess

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Problem with ID Terminal Rule [message #1782083 is a reply to message #1782082] Fri, 16 February 2018 20:04 Go to previous messageGo to next message
Greg Lusk is currently offline Greg LuskFriend
Messages: 36
Registered: February 2017
Location: USA
Member
Hi Christian, I created a Github Gist that contains my grammar and a related test:

https://gist.github.com/glusk/f7565cca0977b9ead21e6e0fbe1f7559
Re: Problem with ID Terminal Rule [message #1782085 is a reply to message #1782083] Fri, 16 February 2018 20:26 Go to previous messageGo to next message
Greg Lusk is currently offline Greg LuskFriend
Messages: 36
Registered: February 2017
Location: USA
Member
Interestingly enough, the following test fails (as expected):

@Test def testParseIllegalVarName(){
		parseFunctionBlock('''
			FUNCTION_BLOCK 3V0
			 VAR_INPUT
			  e:BOOL;
			  E:BOOL;
			  X:NUMERIC:=1.23E-23;
			 END_VAR
			 VAR_OUTPUT
			  B:BOOL;
			  D:BOOL;
			 END_VAR
			 VAR
			  C(1):BOOL;
			 END_VAR
			END_FUNCTION_BLOCK
		''').assertNoErrors
	}


But building a model with the Factory and trying to serialize an illegal variable name succeeds. I would expect that, if parsing faile, the semantic sequencer would throw an exception when backtracking during serialization too.

[Updated on: Fri, 16 February 2018 20:27]

Report message to a moderator

Re: Problem with ID Terminal Rule [message #1782086 is a reply to message #1782083] Fri, 16 February 2018 20:30 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
so your question is about serialization and not parsing?

you need to implement a value converter service for that.

package org.xtext.example.mydsl;

import org.eclipse.xtext.common.services.DefaultTerminalConverters;
import org.eclipse.xtext.conversion.IValueConverter;
import org.eclipse.xtext.conversion.ValueConverter;
import org.eclipse.xtext.conversion.ValueConverterException;
import org.eclipse.xtext.conversion.impl.KeywordAlternativeConverter;
import org.eclipse.xtext.nodemodel.INode;

import com.google.inject.Inject;

public class MyDslValueConverterService extends DefaultTerminalConverters {
	
	@Inject
	private VarNameVC varNameVC;

	@ValueConverter(rule = "Variable_Name")
	public IValueConverter<String> Variable_Name() {
		return varNameVC;
	}
	
	public static class VarNameVC implements IValueConverter<String> {

		@Override
		public String toValue(String string, INode node) throws ValueConverterException {
			return string;
		}

		@Override
		public String toString(String value) throws ValueConverterException {
			if (value.contains("(")) {
				throw new ValueConverterException("mimimi", null, null);
			}
			return value;
		}
		
	}

}



Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Problem with ID Terminal Rule [message #1782087 is a reply to message #1782086] Fri, 16 February 2018 20:39 Go to previous messageGo to next message
Greg Lusk is currently offline Greg LuskFriend
Messages: 36
Registered: February 2017
Location: USA
Member
Yes, the question is about serialization; the original problem surfaced during parsing, though. I will try the value converter service and post back how it works out. One question, though: since there are infinite combinations of possible illegal characters, in the toString() function, is there some means of accessing the grammar rule to see if it matches the expected regex to ensure that the value is valid instead of having to work through a bunch of specific exceptions?
Re: Problem with ID Terminal Rule [message #1782088 is a reply to message #1782087] Fri, 16 February 2018 21:05 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
yes and now. its not really easy.

you would have to build a combination of AbstractLexerBasedConverter
and QualifiedNameValueConverter and KeywordAlternativeConverter

calling the parser manually i huge overhead and might be extremely inperformant

public static class MyDslParser2 extends MyDslParser {
		public IParseResult doParseVarName(CharSequence sequence) {
			try {
				return parse("Variable_Name", new ANTLRReaderStream(new StringReader(sequence.toString())));
			} catch (IOException e) {
				throw new WrappedException(e);
			}
		}
	}
	
	public static class VarNameVC implements IValueConverter<String> {
		
		@Inject
		MyDslParser2 p;


		@Override
		public String toValue(String string, INode node) throws ValueConverterException {
			return string;
		}

		@Override
		public String toString(String value) throws ValueConverterException {
			IParseResult result = p.doParseVarName(value);
			if (result.hasSyntaxErrors()) {
				System.err.println(result.getSyntaxErrors());
				throw new ValueConverterException("mimimi", null, null);
			}
			return value;
		}




=> what about using a positive regex instead of opting out?


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Problem with ID Terminal Rule [message #1782089 is a reply to message #1782088] Fri, 16 February 2018 21:10 Go to previous messageGo to next message
Greg Lusk is currently offline Greg LuskFriend
Messages: 36
Registered: February 2017
Location: USA
Member
A positive regex was going to be my next option to ask about. I suppose that the only drawback might be that I will always have to manually synchronize the regex with the one that is in the grammar. But that would not be too big a price to pay. :)
Re: Problem with ID Terminal Rule [message #1782155 is a reply to message #1782089] Mon, 19 February 2018 15:45 Go to previous message
Greg Lusk is currently offline Greg LuskFriend
Messages: 36
Registered: February 2017
Location: USA
Member
Christian,

Your solution worked perfectly! We used a positive regex to ensure that the variable names contain only legal characters, and now serialization works correctly. Thanks!
Previous Topic:Lexical syntax highlighting
Next Topic:Using indent for more than one level of indentation
Goto Forum:
  


Current Time: Thu Mar 28 21:46:19 GMT 2024

Powered by FUDForum. Page generated in 0.02677 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top