Eclipse Community Forums: TMF (Xtext) » Formatting inserts spaces despite hidden WS

Home » Modeling » TMF (Xtext) » Formatting inserts spaces despite hidden WS(And general help with formatting and indenting)

Show: Today's Messages :: Show Polls :: Message Navigator

Formatting inserts spaces despite hidden WS [message #1853743]

Sat, 16 July 2022 11:01

Mykola Makhin

Messages: 50
Registered: July 2018

Member

For some reason I cannot prevent Xtext from putting spaces everywhere when formatting DSL code.

Here's an example grammar, where for every entity.property['key'][0] I've set hidden(WS), but still whitespace is automatically put in on formatting

grammar x.mvmn.permock.dsl.Dsl hidden(WS, ML_COMMENT, SL_COMMENT)

generate dsl "http://www.mvmn.x/permock/dsl/Dsl"
import "http://www.eclipse.org/emf/2002/Ecore" as ecore

Rule:
	'to' conditions=Condition
	('proxy' proxy=ProxyConf | 'respond' 'with' response=ResponseConf)?;

ProxyConf:
	'url' proxyUrl=STRING;

ResponseConf:
	('status' httpStatus?=INTEGER)? &
	('content' content?=STRING)? & ('headers' headers=Headers?)?;

Headers:
	headers+=Header (',' headers+=Header)*;

Header:
	headerName=STRING ':' headerValue=STRING;

Condition:
	OrCondition;

OrCondition returns Condition:
	AndCondition ({OrCondition.left=current} 'or' right=AndCondition)*;

AndCondition returns Condition:
	OptionalNegationCondition ({AndCondition.left=current} 'and' right=OptionalNegationCondition)*;

OptionalNegationCondition returns Condition:
	BracketedCondition | {Negation} 'not' negated=BracketedCondition;

BracketedCondition returns Condition:
	expression=Expression | '(' condition=Condition ')';

Expression:
	left=Operand (op=Operator right=Operand)?;

Operand:
	ref=Reference | const=Constant | listElementRef=ListElementReference;

ListFunction:
	'{' op=ListOperation alias=ListElementAlias ':' condition=Condition '}';

ListElementAlias:
	name=ID;

Reference hidden(WS):
	name=Entity (prop=PropertyRef)?;

ListElementReference hidden(WS):
	'$' name=[ListElementAlias|ID] (prop=PropertyRef)?;

PropertyRef hidden(WS):
	('.' name=ID | '[' (name=STRING | index=INTEGER) ']' | listFunc=ListFunction) (subPropery=PropertyRef)?;

enum Operator:
	EQ='=' | NEQ='!=' | GT='>' | GTEQ='>=' | LT='<' | LTEQ='<=' | REGEX='~=';

Constant:
	strVal=STRING | intVal=INTEGER | floatVal=FLOAT;

enum ListOperation:
	FILTER='where' | ALL='all' | ANY='any';

Entity:
	name=ID;

terminal FLOAT returns ecore::EDoubleObject:
	INTEGER '.' INTEGER;

terminal INTEGER returns ecore::EIntegerObject:
	('0'..'9')+;

terminal ID:
	'^'? ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*;

terminal STRING:
	'"' ('\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\' | '"'))* '"' |
	"'" ('\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\' | "'"))* "'";

terminal ML_COMMENT:
	'/*'->'*/';

terminal SL_COMMENT:
	'//' !('\n' | '\r')* ('\r'? '\n')?;

terminal WS:
	(' ' | '\t' | '\r' | '\n')+;

terminal ANY_OTHER:
	.;

Sample DSL code before formatting (the way I'd like it to be formatted):

to 
  name{where name: 
    $name[0]['test'].prop.subprop = 1 
  }
respond with 
  headers 
  	"headerName": "headerValue", 
	"headerName": "headerValue"

After formatting

to name { where name : $ name [ 0 ] [ 'test' ] . prop . subprop = 1 } respond with headers "headerName" : "headerValue" , "headerName" : "headerValue"

I haven't worked on indentation for formatting yet (any hits on this are appreciated BTW), but first and foremost I'm trying to understand how to get rid of spaces (and actually disallow spaces) for things like $listElement, entity.property.subProperty etc.

Can anyone give me a hint where to start? Thanks!

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853744 is a reply to message #1853743]

Sat, 16 July 2022 12:40

Christian Dietrich

Messages: 14699
Registered: July 2009

Senior Member

i assume you need to implement the formatter yourself in this case

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853746 is a reply to message #1853744]

Sat, 16 July 2022 16:36

Mykola Makhin

Messages: 50
Registered: July 2018

Member

No way to specify this in the grammar? Ok.
It's weird though that for FLOAT terminal rule it doesn't happen.

terminal FLOAT returns ecore::EDoubleObject:
	INTEGER '.' INTEGER;

Here the default formatter leaves 123.456 as is, and doesn't make it 123 . 456 (fortunately). What I don't understand is how the other rules differ.

As for the custom formatter, any examples of whitespace removal around?

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853748 is a reply to message #1853746]

Sat, 16 July 2022 17:04

Mykola Makhin

Messages: 50
Registered: July 2018

Member

I've been scrambling for examples, and came up with this:

public class DslFormatter extends AbstractJavaFormatter {
	protected void format(PropertyRef propertyRef, IFormattableDocument doc) {
		System.out.println(propertyRef);
		doc.surround(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__NAME),
				i -> i.noSpace());
		doc.surround(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__INDEX),
				i -> i.noSpace());
	}

	protected void format(Rule rule, IFormattableDocument doc) {
		// TODO: format HiddenRegions around keywords, attributes, cross references,
		// etc.
		doc.format(rule.getConditions());
		doc.format(rule.getProxy());
		doc.format(rule.getResponse());
	}

	protected void format(ResponseConf responseConf, IFormattableDocument doc) {
		// TODO: format HiddenRegions around keywords, attributes, cross references,
		// etc.
		doc.format(responseConf.getHeaders());
	}
}

One would expect this to remove spaces around property name after dot, property name in brackets and index in brackets. But it doesn't do that.
For DSL like this:

to request . abc . xyz [ 'test' ] . prop [ 123 ] = dqw

the formatted code becomes

to request .abc. xyz [ 'test' ] . prop [ 123 ] = dqw

So clearly index and name in brackets are ignored completely. As for the property, some spaces are removed, while others - not.
It's hard to say what is happening here.

The only issue that is clear is the Reference rule:

Reference hidden(WS):
	name=Entity (prop=PropertyRef)?;

The dot is in PropertyRef, and the Entity may or may not have a property ref after it.
So also not clear how to say don't put the space in if it has a property ref, but keep spaces if it's followed by a different rule.

[Updated on: Sat, 16 July 2022 17:08]

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853750 is a reply to message #1853748]

Sat, 16 July 2022 17:16

Mykola Makhin

Messages: 50
Registered: July 2018

Member

P.S. Discovered that subproperties weren't formatted at all in the method call, so added this:

		if (propertyRef.getSubPropery() != null) {
			doc.format(propertyRef.getSubPropery());
		}

So now from

to request . abc . xyz [ 'test' ] . prop [ 123 ] . other = dqw

formatter gets me to

to request .abc.xyz['test'] .prop[123] .other= dqw

Still spaces after closing square bracket - for no apparent reason

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853752 is a reply to message #1853750]

Sat, 16 July 2022 18:08

Mykola Makhin

Messages: 50
Registered: July 2018

Member

Ok, one workaround I've found is to assign names to the static literals in the grammar (which I didn't know I can do)
E.g.

PropertyRef hidden(WS):
	(dot='.' name=ID | openBr='[' (name=STRING | index=INTEGER) closeBr=']' | listFunc=ListFunction)
	(subPropery=PropertyRef)?;

Now '.' is referencable literal as DslPackage.Literals.PROPERTY_REF__DOT, same for openBr and closeBr
This allows me to achieve no spaces around/before those literals via this in AbstractJavaFormatter

	protected void format(PropertyRef propertyRef, IFormattableDocument doc) {
		System.out.println(propertyRef.getName());
		doc.surround(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__DOT),
				i -> i.noSpace());
		doc.prepend(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__NAME),
				i -> i.noSpace());
		doc.prepend(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__INDEX),
				i -> i.noSpace());
		doc.surround(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__OPEN_BR),
				i -> i.noSpace());
		doc.prepend(textRegionExtensions.regionFor(propertyRef).feature(DslPackage.Literals.PROPERTY_REF__CLOSE_BR),
				i -> i.noSpace());
		if (propertyRef.getSubPropery() != null) {
			doc.format(propertyRef.getSubPropery());
		}
	}

[Updated on: Sat, 16 July 2022 18:08]

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853759 is a reply to message #1853752]

Sun, 17 July 2022 07:51

Mykola Makhin

Messages: 50
Registered: July 2018

Member

Now I've found XtextFormatter extends AbstractDeclarativeFormatter, and it's using a completely different approach.
But it seems auto-generated formatter that extends AbstractJavaFormatter does not extend AbstractDeclarativeFormatter

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853764 is a reply to message #1853759]

Sun, 17 July 2022 08:21

Mykola Makhin

Messages: 50
Registered: July 2018

Member

Now I see that referencable literals are unnecessary, because one can do something like

doc.surround(textRegionExtensions.regionFor(propertyRef).keyword("["), i -> i.noSpace());

Report message to a moderator

Re: Formatting inserts spaces despite hidden WS [message #1853772 is a reply to message #1853764]

Sun, 17 July 2022 21:36

Mykola Makhin

Messages: 50
Registered: July 2018

Member

Used this as a reference: https://labs.etsi.org/rep/top/ide/-/blob/master/plugins/org.etsi.mts.tdl.tx/src/org/etsi/mts/tdl/formatting2/TDLtxFormatter.java
May be helpful for others as well.

Report message to a moderator

Previous Topic:	Registering files needed by a dsl for Unit tests
Next Topic:	Unability to serialize an Xtext grammar created with EMF

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sat Jul 27 14:59:56 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter