Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Illegal escape character in json grammar for character that is in the grammar
Illegal escape character in json grammar for character that is in the grammar [message #1072331] Mon, 22 July 2013 14:37 Go to next message
Mr Manner is currently offline Mr MannerFriend
Messages: 26
Registered: January 2013
Junior Member
Consider this json grammar where the STRING terminal allows for the backslash "\" to escape a number of character including the solidus "/".

grammar com.test.example.Json hidden(WS)
import "http://www.eclipse.org/emf/2002/Ecore" as ecore

generate json "http://www.test.com/example/Json"

RootObject returns JsonObject:
	(JsonObject | EmptyObject)?; 

JsonObject:
	{JsonObject} '{' (members+=Member) (',' members+=Member)* '}';

EmptyObject returns JsonObject:
	{JsonObject} '{' '}';

Member:
	key=STRING ':' name=Value;
	
Value:
	JsonObject | EmptyObject | StringValue | Array | EmptyArray | Boolean | Null | NumberValue;

NumberValue:
	name=NUMBER;

StringValue:
	name=STRING;
	
Array:
	{Array} '[' ( => values+=Value) (',' values+=Value)* ']';
 
 EmptyArray returns Array:
 	{Array} '[' ']';
 
Boolean:
	{Boolean} (name='true' | 'false');

Null:
	{Null} name='null';

terminal NUMBER:
	'-'? INT ('.' INT)? (('E' | 'e') '-'? INT)?;
	
terminal INT returns ecore::EInt:
	('0'..'9')+;

terminal STRING returns ecore::EString:
	'"' ( '\\' ('"'|'/'|'\\'|'b'|'f'|'n'|'r'|'t') | !('\\'|'"') )* '"';

terminal WS:
	(' '|'\t'|'\r'|'\n')+;


The editor for this grammar puts an error mark in eclipse when a string contains the "\/" sub-string, but not for other escapable characters like \n, \t, \\ or \".

Ex.
This Works
{ 	
  "test": "\t\"\n\b\f\\\r" 
}

This does not work
{ 	
  "test": "\t\"\n\b\f\\\r\/" 
}



I covered the basics like made sure that I generate the latest version of my grammar and cleaned my build and stuff.
Why, could this be a bug in xtext?
Re: Illegal escape character in json grammar for character that is in the grammar [message #1072385 is a reply to message #1072331] Mon, 22 July 2013 16:44 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
Did you have a look at the default String terminal and its value
converter. It does basically the same.

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Illegal escape character in json grammar for character that is in the grammar [message #1072493 is a reply to message #1072385] Mon, 22 July 2013 22:15 Go to previous messageGo to next message
Mr Manner is currently offline Mr MannerFriend
Messages: 26
Registered: January 2013
Junior Member
I based by String terminal on the default. What do you mean by "its value converter", the default String has the same behaviour as the json String.
I don't want this behaviour from my String, it should comply with the json standard. Why does "\/" not match the terminal rule?
Re: Illegal escape character in json grammar for character that is in the grammar [message #1072590 is a reply to message #1072493] Tue, 23 July 2013 05:21 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
???????
it matches the terminal rule!

the error message is "illegal escape character" this comes from the org.eclipse.xtext.conversion.impl.STRINGValueConverter
that delegates to org.eclipse.xtext.util.Strings.convertFromJavaString(String, boolean)


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Illegal escape character in json grammar for character that is in the grammar [message #1073118 is a reply to message #1072331] Wed, 24 July 2013 07:12 Go to previous messageGo to next message
Claudio Heeg is currently offline Claudio HeegFriend
Messages: 75
Registered: April 2013
Member
If I may hijack that topic as I have a...somewhat...similar question..
Is it at all possible to rewrite the String Terminal itself to allow either '\','\_' or a regular '...'-String, without using the ValueConverter?
My attempts at doing this directly via that rule ended up in getting an "Illegal Escape Character" error message, which strikes me as something coded behind the scenes, but I may be mistaken.

[Updated on: Wed, 24 July 2013 07:14]

Report message to a moderator

Re: Illegal escape character in json grammar for character that is in the grammar [message #1073128 is a reply to message #1073118] Wed, 24 July 2013 07:26 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14661
Registered: July 2009
Senior Member
Hi just make sure you hook an appropriate valueconverter

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Illegal escape character in json grammar for character that is in the grammar [message #1073136 is a reply to message #1073128] Wed, 24 July 2013 07:48 Go to previous messageGo to next message
Claudio Heeg is currently offline Claudio HeegFriend
Messages: 75
Registered: April 2013
Member
Hence my question, the default value converter wouldn't allow for such things?
That's what I was thinking, thanks for confirming that.
Re: Illegal escape character in json grammar for character that is in the grammar [message #1075371 is a reply to message #1073136] Mon, 29 July 2013 10:07 Go to previous messageGo to next message
Claudio Heeg is currently offline Claudio HeegFriend
Messages: 75
Registered: April 2013
Member
Hello,

sorry for reviving this thread again - I have now tried to work with the ValueConverter - Strings containing '\_' can be parsed now - however, Strings only containing a \, i.e. '\', still do not work and won't even lead to Illegal Escape Character Issues - the parsing will just break completely and there will be lots of syntactic errors as the String can't be read correctly.
However, shouldn't defining "'\\'" in my String terminal explicity allow such a String to exist?

Model:
	assign+=STRING;

terminal STRING	: 
		("\'\\\'") | ("'" ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|"'") )* "'")
		; 


Entering '\' leads to an error message - the error message states "1". Which is quite confusing.

--
Further edit:
Defining STRING as "\\", i.e. only the escape character, and then trying it out, leads to "String Index out of Range: -1" - seems to be a problem in the ValueConverter anyway, then?

---
More Editing:
Defining Strings as "\' \\ \'" also works as expected, with ' \ ' being parsed correctly as a String. Without the second space, it stops working - the \ seems to be seen as an escape for ' in any case, then.
  • Attachment: 1.png
    (Size: 31.52KB, Downloaded 202 times)

[Updated on: Mon, 29 July 2013 10:41]

Report message to a moderator

Re: Illegal escape character in json grammar for character that is in the grammar [message #1075400 is a reply to message #1075371] Mon, 29 July 2013 11:06 Go to previous message
Claudio Heeg is currently offline Claudio HeegFriend
Messages: 75
Registered: April 2013
Member
How i solved the issue:
First I explicitly disallowed the \' escaping.
Doing that led to the "1" error message, which still strikes me as a bug.
Following that I edited my ValueConverter (convertFromJavaString) so that it doesn't enter its further conversion on reading \ if the length of the read String is 1, i.e. adding the second clause in the following code
if (aChar == '\\' && len>1) 

Possibly not clean, but it seems to get the job done.

However, as mentioned, I still believe the "1" error message and the String Index out of Range message are bugs.

[Updated on: Mon, 29 July 2013 11:07]

Report message to a moderator

Previous Topic:Injection Error if grammer is inherited and referenced
Next Topic:Outline broken in Xtext 2.4.2
Goto Forum:
  


Current Time: Thu Mar 28 20:36:10 GMT 2024

Powered by FUDForum. Page generated in 0.02971 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top