Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Illegal escape character in json grammar for character that is in the grammar
Illegal escape character in json grammar for character that is in the grammar [message #1072331] Mon, 22 July 2013 10:37 Go to next message
Mr Manner is currently offline Mr Manner
Messages: 26
Registered: January 2013
Junior Member
Consider this json grammar where the STRING terminal allows for the backslash "\" to escape a number of character including the solidus "/".

grammar com.test.example.Json hidden(WS)
import "http://www.eclipse.org/emf/2002/Ecore" as ecore

generate json "http://www.test.com/example/Json"

RootObject returns JsonObject:
	(JsonObject | EmptyObject)?; 

JsonObject:
	{JsonObject} '{' (members+=Member) (',' members+=Member)* '}';

EmptyObject returns JsonObject:
	{JsonObject} '{' '}';

Member:
	key=STRING ':' name=Value;
	
Value:
	JsonObject | EmptyObject | StringValue | Array | EmptyArray | Boolean | Null | NumberValue;

NumberValue:
	name=NUMBER;

StringValue:
	name=STRING;
	
Array:
	{Array} '[' ( => values+=Value) (',' values+=Value)* ']';
 
 EmptyArray returns Array:
 	{Array} '[' ']';
 
Boolean:
	{Boolean} (name='true' | 'false');

Null:
	{Null} name='null';

terminal NUMBER:
	'-'? INT ('.' INT)? (('E' | 'e') '-'? INT)?;
	
terminal INT returns ecore::EInt:
	('0'..'9')+;

terminal STRING returns ecore::EString:
	'"' ( '\\' ('"'|'/'|'\\'|'b'|'f'|'n'|'r'|'t') | !('\\'|'"') )* '"';

terminal WS:
	(' '|'\t'|'\r'|'\n')+;


The editor for this grammar puts an error mark in eclipse when a string contains the "\/" sub-string, but not for other escapable characters like \n, \t, \\ or \".

Ex.
This Works
{ 	
  "test": "\t\"\n\b\f\\\r" 
}

This does not work
{ 	
  "test": "\t\"\n\b\f\\\r\/" 
}



I covered the basics like made sure that I generate the latest version of my grammar and cleaned my build and stuff.
Why, could this be a bug in xtext?
Re: Illegal escape character in json grammar for character that is in the grammar [message #1072385 is a reply to message #1072331] Mon, 22 July 2013 12:44 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 6144
Registered: July 2009
Senior Member
Did you have a look at the default String terminal and its value
converter. It does basically the same.

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de
Re: Illegal escape character in json grammar for character that is in the grammar [message #1072493 is a reply to message #1072385] Mon, 22 July 2013 18:15 Go to previous messageGo to next message
Mr Manner is currently offline Mr Manner
Messages: 26
Registered: January 2013
Junior Member
I based by String terminal on the default. What do you mean by "its value converter", the default String has the same behaviour as the json String.
I don't want this behaviour from my String, it should comply with the json standard. Why does "\/" not match the terminal rule?
Re: Illegal escape character in json grammar for character that is in the grammar [message #1072590 is a reply to message #1072493] Tue, 23 July 2013 01:21 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 6144
Registered: July 2009
Senior Member
???????
it matches the terminal rule!

the error message is "illegal escape character" this comes from the org.eclipse.xtext.conversion.impl.STRINGValueConverter
that delegates to org.eclipse.xtext.util.Strings.convertFromJavaString(String, boolean)
Re: Illegal escape character in json grammar for character that is in the grammar [message #1073118 is a reply to message #1072331] Wed, 24 July 2013 03:12 Go to previous messageGo to next message
Claudio Heeg is currently offline Claudio Heeg
Messages: 75
Registered: April 2013
Member
If I may hijack that topic as I have a...somewhat...similar question..
Is it at all possible to rewrite the String Terminal itself to allow either '\','\_' or a regular '...'-String, without using the ValueConverter?
My attempts at doing this directly via that rule ended up in getting an "Illegal Escape Character" error message, which strikes me as something coded behind the scenes, but I may be mistaken.

[Updated on: Wed, 24 July 2013 03:14]

Report message to a moderator

Re: Illegal escape character in json grammar for character that is in the grammar [message #1073128 is a reply to message #1073118] Wed, 24 July 2013 03:26 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 6144
Registered: July 2009
Senior Member
Hi just make sure you hook an appropriate valueconverter

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de
Re: Illegal escape character in json grammar for character that is in the grammar [message #1073136 is a reply to message #1073128] Wed, 24 July 2013 03:48 Go to previous messageGo to next message
Claudio Heeg is currently offline Claudio Heeg
Messages: 75
Registered: April 2013
Member
Hence my question, the default value converter wouldn't allow for such things?
That's what I was thinking, thanks for confirming that.
Re: Illegal escape character in json grammar for character that is in the grammar [message #1075371 is a reply to message #1073136] Mon, 29 July 2013 06:07 Go to previous messageGo to next message
Claudio Heeg is currently offline Claudio Heeg
Messages: 75
Registered: April 2013
Member
Hello,

sorry for reviving this thread again - I have now tried to work with the ValueConverter - Strings containing '\_' can be parsed now - however, Strings only containing a \, i.e. '\', still do not work and won't even lead to Illegal Escape Character Issues - the parsing will just break completely and there will be lots of syntactic errors as the String can't be read correctly.
However, shouldn't defining "'\\'" in my String terminal explicity allow such a String to exist?

Model:
	assign+=STRING;

terminal STRING	: 
		("\'\\\'") | ("'" ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|"'") )* "'")
		; 


Entering '\' leads to an error message - the error message states "1". Which is quite confusing.

--
Further edit:
Defining STRING as "\\", i.e. only the escape character, and then trying it out, leads to "String Index out of Range: -1" - seems to be a problem in the ValueConverter anyway, then?

---
More Editing:
Defining Strings as "\' \\ \'" also works as expected, with ' \ ' being parsed correctly as a String. Without the second space, it stops working - the \ seems to be seen as an escape for ' in any case, then.
  • Attachment: 1.png
    (Size: 31.52KB, Downloaded 23 times)

[Updated on: Mon, 29 July 2013 06:41]

Report message to a moderator

Re: Illegal escape character in json grammar for character that is in the grammar [message #1075400 is a reply to message #1075371] Mon, 29 July 2013 07:06 Go to previous message
Claudio Heeg is currently offline Claudio Heeg
Messages: 75
Registered: April 2013
Member
How i solved the issue:
First I explicitly disallowed the \' escaping.
Doing that led to the "1" error message, which still strikes me as a bug.
Following that I edited my ValueConverter (convertFromJavaString) so that it doesn't enter its further conversion on reading \ if the length of the read String is 1, i.e. adding the second clause in the following code
if (aChar == '\\' && len>1) 

Possibly not clean, but it seems to get the job done.

However, as mentioned, I still believe the "1" error message and the String Index out of Range message are bugs.

[Updated on: Mon, 29 July 2013 07:07]

Report message to a moderator

Previous Topic:Injection Error if grammer is inherited and referenced
Next Topic:Outline broken in Xtext 2.4.2
Goto Forum:
  


Current Time: Thu Jul 31 07:48:53 EDT 2014

Powered by FUDForum. Page generated in 0.07748 seconds