Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Formatting inserts spaces despite hidden WS(And general help with formatting and indenting)
Formatting inserts spaces despite hidden WS [message #1853743] Sat, 16 July 2022 11:01 Go to next message
Mykola Makhin is currently offline Mykola MakhinFriend
Messages: 50
Registered: July 2018
Member
For some reason I cannot prevent Xtext from putting spaces everywhere when formatting DSL code.

Here's an example grammar, where for every entity.property['key'][0] I've set hidden(WS), but still whitespace is automatically put in on formatting
grammar x.mvmn.permock.dsl.Dsl hidden(WS, ML_COMMENT, SL_COMMENT)

generate dsl "http://www.mvmn.x/permock/dsl/Dsl"
import "http://www.eclipse.org/emf/2002/Ecore" as ecore

Rule:
	'to' conditions=Condition
	('proxy' proxy=ProxyConf | 'respond' 'with' response=ResponseConf)?;

ProxyConf:
	'url' proxyUrl=STRING;

ResponseConf:
	('status' httpStatus?=INTEGER)? &
	('content' content?=STRING)? & ('headers' headers=Headers?)?;

Headers:
	headers+=Header (',' headers+=Header)*;

Header:
	headerName=STRING ':' headerValue=STRING;

Condition:
	OrCondition;

OrCondition returns Condition:
	AndCondition ({OrCondition.left=current} 'or' right=AndCondition)*;

AndCondition returns Condition:
	OptionalNegationCondition ({AndCondition.left=current} 'and' right=OptionalNegationCondition)*;

OptionalNegationCondition returns Condition:
	BracketedCondition | {Negation} 'not' negated=BracketedCondition;

BracketedCondition returns Condition:
	expression=Expression | '(' condition=Condition ')';

Expression:
	left=Operand (op=Operator right=Operand)?;

Operand:
	ref=Reference | const=Constant | listElementRef=ListElementReference;

ListFunction:
	'{' op=ListOperation alias=ListElementAlias ':' condition=Condition '}';

ListElementAlias:
	name=ID;

Reference hidden(WS):
	name=Entity (prop=PropertyRef)?;

ListElementReference hidden(WS):
	'$' name=[ListElementAlias|ID] (prop=PropertyRef)?;

PropertyRef hidden(WS):
	('.' name=ID | '[' (name=STRING | index=INTEGER) ']' | listFunc=ListFunction) (subPropery=PropertyRef)?;

enum Operator:
	EQ='=' | NEQ='!=' | GT='>' | GTEQ='>=' | LT='<' | LTEQ='<=' | REGEX='~=';

Constant:
	strVal=STRING | intVal=INTEGER | floatVal=FLOAT;

enum ListOperation:
	FILTER='where' | ALL='all' | ANY='any';

Entity:
	name=ID;

terminal FLOAT returns ecore::EDoubleObject:
	INTEGER '.' INTEGER;

terminal INTEGER returns ecore::EIntegerObject:
	('0'..'9')+;

terminal ID:
	'^'? ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*;

terminal STRING:
	'"' ('\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\' | '"'))* '"' |
	"'" ('\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\' | "'"))* "'";

terminal ML_COMMENT:
	'/*'->'*/';

terminal SL_COMMENT:
	'//' !('\n' | '\r')* ('\r'? '\n')?;

terminal WS:
	(' ' | '\t' | '\r' | '\n')+;

terminal ANY_OTHER:
	.;


Sample DSL code before formatting (the way I'd like it to be formatted):
to 
  name{where name: 
    $name[0]['test'].prop.subprop = 1 
  }
respond with 
  headers 
  	"headerName": "headerValue", 
	"headerName": "headerValue"


After formatting
to name { where name : $ name [ 0 ] [ 'test' ] . prop . subprop = 1 } respond with headers "headerName" : "headerValue" , "headerName" : "headerValue"


I haven't worked on indentation for formatting yet (any hits on this are appreciated BTW), but first and foremost I'm trying to understand how to get rid of spaces (and actually disallow spaces) for things like $listElement, entity.property.subProperty etc.

Can anyone give me a hint where to start? Thanks!
Re: Formatting inserts spaces despite hidden WS [message #1853744 is a reply to message #1853743] Sat, 16 July 2022 12:40 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14428
Registered: July 2009
Senior Member
i assume you need to implement the formatter yourself in this case

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Formatting inserts spaces despite hidden WS [message #1853746 is a reply to message #1853744] Sat, 16 July 2022 16:36 Go to previous messageGo to next message
Mykola Makhin is currently offline Mykola MakhinFriend
Messages: 50
Registered: July 2018
Member
No way to specify this in the grammar? Ok.
It's weird though that for FLOAT terminal rule it doesn't happen.

terminal FLOAT returns ecore::EDoubleObject:
	INTEGER '.' INTEGER;

Here the default formatter leaves 123.456 as is, and doesn't make it 123 . 456 (fortunately). What I don't understand is how the other rules differ.

As for the custom formatter, any examples of whitespace removal around?
Re: Formatting inserts spaces despite hidden WS [message #1853748 is a reply to message #1853746] Sat, 16 July 2022 17:04 Go to previous messageGo to next message
Mykola Makhin is currently offline Mykola MakhinFriend
Messages: 50
Registered: July 2018
Member
I've been scrambling for examples, and came up with this:
public class DslFormatter extends AbstractJavaFormatter {
	protected void format(PropertyRef propertyRef, IFormattableDocument doc) {
		System.out.println(propertyRef);
		doc.surround(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__NAME),
				i -> i.noSpace());
		doc.surround(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__INDEX),
				i -> i.noSpace());
	}

	protected void format(Rule rule, IFormattableDocument doc) {
		// TODO: format HiddenRegions around keywords, attributes, cross references,
		// etc.
		doc.format(rule.getConditions());
		doc.format(rule.getProxy());
		doc.format(rule.getResponse());
	}

	protected void format(ResponseConf responseConf, IFormattableDocument doc) {
		// TODO: format HiddenRegions around keywords, attributes, cross references,
		// etc.
		doc.format(responseConf.getHeaders());
	}
}


One would expect this to remove spaces around property name after dot, property name in brackets and index in brackets. But it doesn't do that.
For DSL like this:
to request . abc . xyz [ 'test' ] . prop [ 123 ] = dqw

the formatted code becomes
to request .abc. xyz [ 'test' ] . prop [ 123 ] = dqw

So clearly index and name in brackets are ignored completely. As for the property, some spaces are removed, while others - not.
It's hard to say what is happening here.

The only issue that is clear is the Reference rule:
Reference hidden(WS):
	name=Entity (prop=PropertyRef)?;

The dot is in PropertyRef, and the Entity may or may not have a property ref after it.
So also not clear how to say don't put the space in if it has a property ref, but keep spaces if it's followed by a different rule.

[Updated on: Sat, 16 July 2022 17:08]

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853750 is a reply to message #1853748] Sat, 16 July 2022 17:16 Go to previous messageGo to next message
Mykola Makhin is currently offline Mykola MakhinFriend
Messages: 50
Registered: July 2018
Member
P.S. Discovered that subproperties weren't formatted at all in the method call, so added this:
		if (propertyRef.getSubPropery() != null) {
			doc.format(propertyRef.getSubPropery());
		}


So now from
to request . abc . xyz [ 'test' ] . prop [ 123 ] . other = dqw

formatter gets me to
to request .abc.xyz['test'] .prop[123] .other= dqw

Still spaces after closing square bracket - for no apparent reason
Re: Formatting inserts spaces despite hidden WS [message #1853752 is a reply to message #1853750] Sat, 16 July 2022 18:08 Go to previous messageGo to next message
Mykola Makhin is currently offline Mykola MakhinFriend
Messages: 50
Registered: July 2018
Member
Ok, one workaround I've found is to assign names to the static literals in the grammar (which I didn't know I can do)
E.g.
PropertyRef hidden(WS):
	(dot='.' name=ID | openBr='[' (name=STRING | index=INTEGER) closeBr=']' | listFunc=ListFunction)
	(subPropery=PropertyRef)?;

Now '.' is referencable literal as DslPackage.Literals.PROPERTY_REF__DOT, same for openBr and closeBr
This allows me to achieve no spaces around/before those literals via this in AbstractJavaFormatter
	protected void format(PropertyRef propertyRef, IFormattableDocument doc) {
		System.out.println(propertyRef.getName());
		doc.surround(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__DOT),
				i -> i.noSpace());
		doc.prepend(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__NAME),
				i -> i.noSpace());
		doc.prepend(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__INDEX),
				i -> i.noSpace());
		doc.surround(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__OPEN_BR),
				i -> i.noSpace());
		doc.prepend(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__CLOSE_BR),
				i -> i.noSpace());
		if (propertyRef.getSubPropery() != null) {
			doc.format(propertyRef.getSubPropery());
		}
	}

[Updated on: Sat, 16 July 2022 18:08]

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853759 is a reply to message #1853752] Sun, 17 July 2022 07:51 Go to previous messageGo to next message
Mykola Makhin is currently offline Mykola MakhinFriend
Messages: 50
Registered: July 2018
Member
Now I've found XtextFormatter extends AbstractDeclarativeFormatter, and it's using a completely different approach.
But it seems auto-generated formatter that extends AbstractJavaFormatter does not extend AbstractDeclarativeFormatter
Re: Formatting inserts spaces despite hidden WS [message #1853764 is a reply to message #1853759] Sun, 17 July 2022 08:21 Go to previous messageGo to next message
Mykola Makhin is currently offline Mykola MakhinFriend
Messages: 50
Registered: July 2018
Member
Now I see that referencable literals are unnecessary, because one can do something like
doc.surround(textRegionExtensions.regionFor(propertyRef).keyword("["), i -> i.noSpace());
Re: Formatting inserts spaces despite hidden WS [message #1853772 is a reply to message #1853764] Sun, 17 July 2022 21:36 Go to previous message
Mykola Makhin is currently offline Mykola MakhinFriend
Messages: 50
Registered: July 2018
Member
Used this as a reference: https://labs.etsi.org/rep/top/ide/-/blob/master/plugins/org.etsi.mts.tdl.tx/src/org/etsi/mts/tdl/formatting2/TDLtxFormatter.java
May be helpful for others as well.
Previous Topic:Registering files needed by a dsl for Unit tests
Next Topic:Unability to serialize an Xtext grammar created with EMF
Goto Forum:
  


Current Time: Tue Jan 31 07:59:04 GMT 2023

Powered by FUDForum. Page generated in 0.04619 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top