Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Dot '.' is not allowed inside double quote in Xtext(Query Regarding Handling of Dot in Xtext Terminal Rules)
Dot '.' is not allowed inside double quote in Xtext [message #1862984] Wed, 10 January 2024 06:02 Go to next message
Unbiased Coder is currently offline Unbiased CoderFriend
Messages: 3
Registered: January 2024
Junior Member
I have been actively working on configuring terminal rules within Xtext and encountered a puzzling inconsistency concerning the usage of dots within strings enclosed by different quotation marks.

In the existing terminal rule for STRING in Xtext, I've noticed that dots are restricted within strings encapsulated by double quotes, as defined by the grammar rules. However, what perplexes me is that dots seem to be permitted and operational within strings enclosed by single quotes.

For instance, consider the following snippet from the current terminal rule:

terminal STRING:
'"' ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|'"') )* '"' |
"'" ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|"'") )* "'"

Now, when utilizing this rule, I encountered the following behavior:


    "This is.dot" results in rejection or errors due to the dot within the double quotes.


    However, 'This.is.dot' is accepted and functions appropriately within the single quotes.



This discrepancy prompts my query regarding the reasoning behind disallowing dots within double quotes while permitting them within single quotes according to the existing terminal rule.

Could you kindly provide insights or propose an alteration to the terminal rule that would allow dots within strings enclosed by both double quotes and single quotes?

I kindly request insights or guidance on this matter. Additionally, if possible, could you provide clarification or elucidation with respect to this behavior?

Thank you very much for your time and consideration.
Re: Dot '.' is not allowed inside double quote in Xtext [message #1862996 is a reply to message #1862984] Wed, 10 January 2024 12:54 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14699
Registered: July 2009
Senior Member
frankly i cannot reproduce this.
can you give more context.
grammar + unit test?
do you have a custom value converter for STRING?


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Dot '.' is not allowed inside double quote in Xtext [message #1863018 is a reply to message #1862996] Thu, 11 January 2024 05:06 Go to previous messageGo to next message
Unbiased Coder is currently offline Unbiased CoderFriend
Messages: 3
Registered: January 2024
Junior Member
Hi,
The issue with example is explained below:

This is my existing xtext Grammar:

(isMyNameSet?='MyName' MyName= STRING)?

String here belongs to the Terminal Rule as below :

terminal STRING:
'"' ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|'"') )* '"' |
"'" ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|"'") )* "'"

Now in my output ide:

MyName "This is Name.with dot"---------> This input is showing error due to dot

but

MyName 'This is Name.with dot'----------> This has no errors
Re: Dot '.' is not allowed inside double quote in Xtext [message #1863019 is a reply to message #1863018] Thu, 11 January 2024 05:43 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14699
Registered: July 2009
Senior Member
please share a complete project with a unit test!
i tried it with a sample and it works.


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Dot '.' is not allowed inside double quote in Xtext [message #1863021 is a reply to message #1863019] Thu, 11 January 2024 05:47 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14699
Registered: July 2009
Senior Member
this test is green
@ExtendWith(InjectionExtension)
@InjectWith(MyDslInjectorProvider)
class MyDslParsingTest {
	@Inject
	ParseHelper<Model> parseHelper
	
	@Test
	def void loadModel() {
		val result = parseHelper.parse('''
			MyName "This is Name.with dot"
		''')
		Assertions.assertNotNull(result)
		val errors = result.eResource.errors
		Assertions.assertTrue(errors.isEmpty, '''Unexpected errors: «errors.join(", ")»''')
	}
}



Model:
	(isMyNameSet?='MyName' MyName= STRING)?
;


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Thu, 11 January 2024 05:47]

Report message to a moderator

Re: Dot '.' is not allowed inside double quote in Xtext [message #1863111 is a reply to message #1863021] Wed, 17 January 2024 03:59 Go to previous messageGo to next message
Unbiased Coder is currently offline Unbiased CoderFriend
Messages: 3
Registered: January 2024
Junior Member
Hi ,

I have three terminal rules in my grammar, each involving double quotes and potentially conflicting with each other. The rules are as follows:

1. terminal PATH: '"' (((ESCAPE)? CUSTOMID)+ '.' CUSTOMID) '"' ;
terminal CUSTOMID : '^'? ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '-' | '0'..'9' | WS)*;
terminal ESCAPE: ('\\' | '\\\\' | '/');


2. terminal VISUAL_STRING: '"%' INT '.' INT 'f"'?;

3. terminal STRING:
'"' ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|'"') )* '"' |
"'" ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|"'") )* "'"

The challenge arises when attempting to handle dot inside double quote for the STRING rule as dot inside double quote is possible for both PATH and VISUAL_STRING. This creates ambiguity and confusion within the grammar, particularly because dots are allowed in a similar pattern for both PATH and VISUAL_STRING. As a result, an error is occurring when a dot is used inside double quotes for a STRING due to this ambiguity.

Is there a way that all three rules can coexist without breaking any functionality?

[Updated on: Wed, 17 January 2024 04:08]

Report message to a moderator

Re: Dot '.' is not allowed inside double quote in Xtext [message #1863120 is a reply to message #1863111] Wed, 17 January 2024 06:59 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14699
Registered: July 2009
Senior Member
you might look into stateful lexing and other external lexers like jflex. but this is just starting points where you need to start digging yourself

additional question: why make not path a datatype rule

PATH: STRING;

and implement a value converter that makes sure it is a path?


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 17 January 2024 07:00]

Report message to a moderator

Previous Topic:Debugging Xtend2 generated Java
Next Topic:Language server seems to trigger build twice during initialization
Goto Forum:
  


Current Time: Sat Jul 20 19:07:47 GMT 2024

Powered by FUDForum. Page generated in 0.03446 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top