Eclipse Community Forums: TMF (Xtext) » How to use EOL to grammar

Home » Modeling » TMF (Xtext) » How to use EOL to grammar(Without EOL my rules are overlapping)

How to use EOL to grammar [message #1770686]

Wed, 16 August 2017 05:12

Eclipse User

Hi,

I've some old legacy language with many problems... and it seems that only way (with ANTLR) is to use EndOfLine as part of the grammar and in general, make in withe space aware language.

But EOL solves one problem giving 2 back ;)

My problems:

1. Rule MyVarField is defined with list of modifiers (*)
2. When I try to define MyMethod after MyVarField modifiers from MyMethod are parsed as part of MyVarField...
3. With added EOL rule it works better but parser comes with errors like 'missing EOF at \n' or 'no viable alternative at input \n'.

So how bad is my situation? Any hope?

This is a short version of my grammar so far:

grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"

MyModel:
	myElements+=MyElements*;

MyElements:
	MyPackage | MyVarField | MyMethod ;

MyPackage:
	'$package' name=ID EOL+;

MyMethod:
	modifiers+=MyModifiers* name=ID '(' ')' EOL+
	myCommands+=MyCommands*;

MyCommands:
	MyReturn | MyVarField | MySet ;

MySet:
	'set' var=[MyVarField] '=' value=INT;

MyReturn:
	'return' var=[MyVarField];

MyVarField:
	type=('$var'|'$val') name=ID (isArray?='(' ')')? varFieldSettings+=MyFieldSettings* EOL+;

MyFieldSettings:
	MyType | MyModifiers;

MyModifiers:
	modifiers=MyModifiersEnum;

MyModifiersEnum:
	FINAL='final' |
	EXTERNAL='external' |
	INTERNAL='internal' |
	PACKAGE='package' |
	PUBLIC='public' |
	PRIVATE='private';

MyType:
	'type' '=' type=[MyModel];

@Override terminal WS:
	(' ' | '\t')+;
terminal EOL:
	'\r'? '\n';

Re: How to use EOL to grammar [message #1770688 is a reply to message #1770686]

Wed, 16 August 2017 05:21

Eclipse User

whats your actual problem/question
are
you asking about this ambiguity:

warning(200): ../org.xtext.example.mydsl3/src-gen/org/xtext/example/mydsl3/parser/antlr/internal/InternalMyDsl.g:276:3: Decision can match input such as "'$var' RULE_ID '(' ')' 'type' '=' RULE_ID RULE_EOL" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): ../org.xtext.example.mydsl3.ide/src-gen/org/xtext/example/mydsl3/ide/contentassist/antlr/internal/InternalMyDsl.g:741:41: Decision can match input such as "'$var' RULE_ID '(' ')' 'type' '=' RULE_ID RULE_EOL" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input

Re: How to use EOL to grammar [message #1770690 is a reply to message #1770688]

Wed, 16 August 2017 05:27

Eclipse User

My question is how to define rules MyMethod and MyVarField so they can be used in any order.

Re: How to use EOL to grammar [message #1770691 is a reply to message #1770688]

Wed, 16 August 2017 05:28

Eclipse User

thus the question:

MyMethod:
name=ID '(' ')' EOL+
myCommands+=MyCommands*;

if after the method there are two fields
go they into myCommands or the elements of the parent?

Re: How to use EOL to grammar [message #1770695 is a reply to message #1770691]

Wed, 16 August 2017 05:59

Eclipse User

If they are declared as part of myCommads in MyMehod then they are part of MyMethod till next MyMethod declaration, Just like in Python:

MyMethod:
	modifiers+=MyModifiers* name=ID '(' ')'
	BEGIN
		myCommands+=MyCommands*
	END;
terminal BEGIN: 'synthetic:BEGIN';
terminal END: 'synthetic:END';

Sos this should be correct:

$package test

$var testVar public external

// begining of method
public static methodName()
	$var intVal private internal
	return intVal
//end of method

// begining of method
private final method2 ()
	return testVar
// end of method

Re: How to use EOL to grammar [message #1770696 is a reply to message #1770695]

Wed, 16 August 2017 06:07

Eclipse User

yes but you dont have begin end thus
maybe

MyMethod:
name=ID '(' ')' EOL+
->myCommands+=MyCommands*;

is what you want

[Updated on: Wed, 16 August 2017 06:07] by Moderator

Re: How to use EOL to grammar [message #1770699 is a reply to message #1770696]

Wed, 16 August 2017 06:52

Eclipse User

This will nt solve this problem, as it is removing modifiers (public,static,...) that can be before method name. I can add

MyMethod:
	modifiers+=MyModifiers* name=ID '(' ')'
	BEGIN
		myCommands+=MyCommands*
	END;
terminal BEGIN: 'synthetic:BEGIN';
terminal END: 'synthetic:END';

to grammar... but anyway example file is not parsed properly.

Still "public internal" from

$var varName // nothing more
public static methodName ()  // public static will be part of varName not methodName()
    return varName

Re: How to use EOL to grammar [message #1770700 is a reply to message #1770699]

Wed, 16 August 2017 06:57

Eclipse User

please give a complete grammar and sample model.
you switch between using WS languages and not using them back and forth

Re: How to use EOL to grammar [message #1770701 is a reply to message #1770700]

Wed, 16 August 2017 07:01

Eclipse User

Changed grammar:

grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"

MyModel:
	myElements+=MyElements*;

MyElements:
	MyPackage | MyVarField | MyMethod;

MyPackage:
	'$package' name=ID;

MyVarField:
	'$var' name=ID (isArray?='(' ')')? varFieldSettings+=MyFieldSettings*;

MyMethod:
	modifiers+=MyModifiers*name=ID '(' ')'
	->myCommands+=MyCommands+;


MyCommands:
	MyReturn | MySet | MyType;

MySet:
	'set' var=ID '=' value=INT;

MyReturn:
	'return' var=ID;
MyFieldSettings:
	MyType | MyModifiers;

MyModifiers:
	modifiers=MyModifiersEnum;

MyModifiersEnum:
	FINAL='final' |
	EXTERNAL='external' |
	INTERNAL='internal' |
	PACKAGE='package' |
	PUBLIC='public' |
	PRIVATE='private';

MyType:
	'type' '=' type=ID;

Sample file:

$package vars

$var varName public external

private final method()
	type = ONE
	return TWO

With this grammar I don't have any parser errors but result is not correct as is parsed as:

$package vars

$var varName public external private final // All in one line

// just that
method()
	type = ONE
	return TWO

Re: How to use EOL to grammar [message #1770703 is a reply to message #1770701]

Wed, 16 August 2017 07:12

Eclipse User

i am still get tons of ambiguity errors in the log.

and still the question

where does the field end and where does the method start?

maybe you can try something like

grammar org.xtext.example.mydsl5.MyDsl with org.eclipse.xtext.common.Terminals hidden(SL_COMMENT, ML_COMMENT, WS, EOL)

MyVarField hidden(WS):
'$var' name=ID (isArray?='(' ')')? varFieldSettings+=MyFieldSettings*;

@Override terminal WS:
(' ' | '\t')+;
terminal EOL:
'\r'? '\n';

this grammar now has no EOL nor WHITESPACE at all

maybe something like this works for you

Re: How to use EOL to grammar [message #1770716 is a reply to message #1770703]

Wed, 16 August 2017 08:56

Eclipse User

Hi,

Now is much better :)

I'm able to correctly parse this model:

$package vars

$var varName public external //test
$var var2Arr() private internal

public final method1()
	type = String
	set var = 123

private final method2()
	return Var

Without any errors in Error log or parser errors.

Just on top of your changes I changed this:

MyVarField hidden(WS):
	'$var' name=ID (isArray?='(' ')')? varFieldSettings+=MyFieldSettings* (EOL | SL_COMMENT)+;

to add some kind of separator as varFieldSettings was still mixed with modifiers list

Thank you for help

Previous Topic:	Multiple Sort Contributions (or Comparators) in Outline
Next Topic:	<MyDSL>TerminalsValidator class not being generated

Goto Forum:

-=] Back to Top [=-

Current Time: Thu Jul 24 22:27:42 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter