Eclipse Community Forums: TMF (Xtext) » Dot '.' is not allowed inside double quote in Xtext

Home » Modeling » TMF (Xtext) » Dot '.' is not allowed inside double quote in Xtext(Query Regarding Handling of Dot in Xtext Terminal Rules)

Show: Today's Messages :: Show Polls :: Message Navigator

Dot '.' is not allowed inside double quote in Xtext [message #1862984]

Wed, 10 January 2024 06:02

Unbiased Coder

Messages: 3
Registered: January 2024

Junior Member

I have been actively working on configuring terminal rules within Xtext and encountered a puzzling inconsistency concerning the usage of dots within strings enclosed by different quotation marks.

In the existing terminal rule for STRING in Xtext, I've noticed that dots are restricted within strings encapsulated by double quotes, as defined by the grammar rules. However, what perplexes me is that dots seem to be permitted and operational within strings enclosed by single quotes.

For instance, consider the following snippet from the current terminal rule:

terminal STRING:
'"' ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|'"') )* '"' |
"'" ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|"'") )* "'"

Now, when utilizing this rule, I encountered the following behavior:

"This is.dot"

'This.is.dot'

This discrepancy prompts my query regarding the reasoning behind disallowing dots within double quotes while permitting them within single quotes according to the existing terminal rule.

Could you kindly provide insights or propose an alteration to the terminal rule that would allow dots within strings enclosed by both double quotes and single quotes?

I kindly request insights or guidance on this matter. Additionally, if possible, could you provide clarification or elucidation with respect to this behavior?

Thank you very much for your time and consideration.

Report message to a moderator

Re: Dot '.' is not allowed inside double quote in Xtext [message #1862996 is a reply to message #1862984]

Wed, 10 January 2024 12:54

Christian Dietrich

Messages: 14735
Registered: July 2009

Senior Member

frankly i cannot reproduce this.
can you give more context.
grammar + unit test?
do you have a custom value converter for STRING?

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Day Job: https://www.everest-systems.com

Report message to a moderator

Re: Dot '.' is not allowed inside double quote in Xtext [message #1863018 is a reply to message #1862996]

Thu, 11 January 2024 05:06

Unbiased Coder

Messages: 3
Registered: January 2024

Junior Member

Hi,
The issue with example is explained below:

This is my existing xtext Grammar:

(isMyNameSet?='MyName' MyName= STRING)?

String here belongs to the Terminal Rule as below :

terminal STRING:
'"' ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|'"') )* '"' |
"'" ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|"'") )* "'"

Now in my output ide:

MyName "This is Name.with dot"---------> This input is showing error due to dot

but

MyName 'This is Name.with dot'----------> This has no errors

Report message to a moderator

Re: Dot '.' is not allowed inside double quote in Xtext [message #1863019 is a reply to message #1863018]

Thu, 11 January 2024 05:43

Christian Dietrich

Messages: 14735
Registered: July 2009

Senior Member

please share a complete project with a unit test!
i tried it with a sample and it works.

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Day Job: https://www.everest-systems.com

Report message to a moderator

Re: Dot '.' is not allowed inside double quote in Xtext [message #1863021 is a reply to message #1863019]

Thu, 11 January 2024 05:47

Christian Dietrich

Messages: 14735
Registered: July 2009

Senior Member

this test is green

@ExtendWith(InjectionExtension)
@InjectWith(MyDslInjectorProvider)
class MyDslParsingTest {
	@Inject
	ParseHelper<Model> parseHelper
	
	@Test
	def void loadModel() {
		val result = parseHelper.parse('''
			MyName "This is Name.with dot"
		''')
		Assertions.assertNotNull(result)
		val errors = result.eResource.errors
		Assertions.assertTrue(errors.isEmpty, '''Unexpected errors: «errors.join(", ")»''')
	}
}

Model:
	(isMyNameSet?='MyName' MyName= STRING)?
;

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Day Job: https://www.everest-systems.com

[Updated on: Thu, 11 January 2024 05:47]

Report message to a moderator

Re: Dot '.' is not allowed inside double quote in Xtext [message #1863111 is a reply to message #1863021]

Wed, 17 January 2024 03:59

Unbiased Coder

Messages: 3
Registered: January 2024

Junior Member

Hi ,

I have three terminal rules in my grammar, each involving double quotes and potentially conflicting with each other. The rules are as follows:

1. terminal PATH: '"' (((ESCAPE)? CUSTOMID)+ '.' CUSTOMID) '"' ;
terminal CUSTOMID : '^'? ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '-' | '0'..'9' | WS)*;
terminal ESCAPE: ('\\' | '\\\\' | '/');

2. terminal VISUAL_STRING: '"%' INT '.' INT 'f"'?;

3. terminal STRING:
'"' ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|'"') )* '"' |
"'" ( '\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\'|"'") )* "'"

The challenge arises when attempting to handle dot inside double quote for the STRING rule as dot inside double quote is possible for both PATH and VISUAL_STRING. This creates ambiguity and confusion within the grammar, particularly because dots are allowed in a similar pattern for both PATH and VISUAL_STRING. As a result, an error is occurring when a dot is used inside double quotes for a STRING due to this ambiguity.

Is there a way that all three rules can coexist without breaking any functionality?

[Updated on: Wed, 17 January 2024 04:08]

Report message to a moderator

Re: Dot '.' is not allowed inside double quote in Xtext [message #1863120 is a reply to message #1863111]

Wed, 17 January 2024 06:59

Christian Dietrich

Messages: 14735
Registered: July 2009

Senior Member

you might look into stateful lexing and other external lexers like jflex. but this is just starting points where you need to start digging yourself

additional question: why make not path a datatype rule

PATH: STRING;

and implement a value converter that makes sure it is a path?

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Day Job: https://www.everest-systems.com

[Updated on: Wed, 17 January 2024 07:00]

Report message to a moderator

Previous Topic:	Debugging Xtend2 generated Java
Next Topic:	Language server seems to trigger build twice during initialization

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Thu Dec 05 16:32:24 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter