Eclipse Community Forums: TMF (Xtext) » Backtrack and Formatting bug?

Help

Home

Home » Modeling » TMF (Xtext) » Backtrack and Formatting bug?(Backtrack breaks Formatting)

Show: Today's Messages :: Show Polls :: Message Navigator

Backtrack and Formatting bug? [message #924043]

Wed, 26 September 2012 12:47

Sandra Shklyaeva

Messages: 16
Registered: July 2012

Junior Member

During the developing, I found the following strange behavior that looks like a bug.

When I turn on the backtracking for MyDsl, formatting fails in the places where backtrack is needed.
When I turn off the backtrack, formatting works as expected.

I attached ready-to-use MyDsl projects with the grammar and formatting.
I also added the project with MyDsl file which reproduce the behavior.

For example, when the backtrack is not needed, formatting is:

for X in Z do
	for X in Z do
		for X in Z do
                        //backtrack is not needed
			X + Y
		end ;
	end ;
end ;

When the backtrack is needed, formatting fails:

for X in Z do
	for X in Z do
		for X in Z do
                        //back track is needed
			X => Y end
				;
			end
				;
			end ;

The grammar is:

grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "MyDsl"

Model:
	(statemets+=Statement ';')*;

Block:
	statemets+=Statement (';' statemets+=Statement?)*;
	
Statement:
	For |
	Import |
	AssignExpr;
	
Sequence:
	expressions+=Expression (',' expressions+=Expression)*;
	
Import:
  'import' imports+=ImportItem (',' imports+=ImportItem)*
      =>'in' block=Block? 'end';

ImportItem:
	name=[Declaration] ('=' value=Expression)?;

Name:
	name=[Declaration];

Parameter returns Declaration:
	{Parameter} name=ID (=>'=' value=Expression)?;
Var returns Declaration:
	{Local} 'var' name=ID (=>'=' value=Expression)?;

Declaration:
	Var | Parameter | Rule;
	
Rule returns Declaration:
	{Rule} (
	'(' (parameters+=Parameter (',' parameters+=Parameter)*)? ')' name=ID? |
		parameters+=Parameter
	) '=>' body = EqualityExpression;

AssignExpr returns Expression:
	Sequence ({Assign.left = current} 'assing' right=Sequence)?;

Expression:
	EqualityExpression ({Plus.left = current} '+' right=EqualityExpression)?;
	
EqualityExpression returns Expression:
	FactorialExpression (
		(
			{In.left = current} 'in' |
			{Equal.left = current} =>'=='
		) 
		right=FactorialExpression
	)?;

FactorialExpression returns Expression:
	PrimaryExpression ({Factorial.expression = current} '!' )?;

PrimaryExpression returns Expression:
	Primary ({Function.scope = current} '(' args = Sequence? ')')*;

Primary returns Expression:
	Var |
	Rule |
	Name |
	Enclosed |
	Literal;

Literal:
	{Literal} (STRING | INT);
	
Enclosed:
	{Enclosed} '(' args = Sequence? ')';

For:
	'for' name=Name 'in' inExpr=Sequence 'do' body=Block? 'end';

The formatting is:

/*
 * generated by Xtext
 */
package org.xtext.example.mydsl.formatting;

import org.eclipse.xtext.Keyword;
import org.eclipse.xtext.ParserRule;
import org.eclipse.xtext.formatting.impl.AbstractDeclarativeFormatter;
import org.eclipse.xtext.formatting.impl.FormattingConfig;
import org.eclipse.xtext.util.Pair;
import org.xtext.example.mydsl.services.MyDslGrammarAccess;

/**
 * This class contains custom formatting description.
 * 
 * see : http://www.eclipse.org/Xtext/documentation/latest/xtext.html#formatting
 * on how and when to use it
 * 
 * Also see {@link org.eclipse.xtext.xtext.XtextFormattingTokenSerializer} as an
 * example
 */
public class MyDslFormatter extends AbstractDeclarativeFormatter {

	@Override
	protected void configureFormatting(FormattingConfig c) {
		MyDslGrammarAccess f = (MyDslGrammarAccess) getGrammarAccess();
		// formatting for Comments
		c.setLinewrap(0, 1, 2).before(f.getSL_COMMENTRule());
		c.setLinewrap(0, 1, 2).before(f.getML_COMMENTRule());
		c.setLinewrap(0, 1, 1).after(f.getML_COMMENTRule());
		// find common keywords an specify formatting for them
		for (Pair<Keyword, Keyword> pair : f.findKeywordPairs("(", ")")) {
			c.setNoSpace().after(pair.getFirst());
			c.setNoSpace().before(pair.getSecond());
		}
		for (Keyword keyword : f.findKeywords(",")) {
			c.setNoSpace().before(keyword);
		}
		// spaces between statements in compilation unit
		c.setLinewrap(2).after(f.getModelAccess().getSemicolonKeyword_1());
		c.setNoSpace().before(
				f.getPrimaryExpressionAccess().getLeftParenthesisKeyword_1_1());
		c.setNoSpace().before(
				f.getPrimaryExpressionAccess().getLeftParenthesisKeyword_1_1());

		// block formating
		ParserRule blockRule = f.getBlockRule();
		c.setLinewrap().before(blockRule);
		c.setIndentationIncrement().before(blockRule);
		c.setLinewrap().after(blockRule);
		c.setIndentationDecrement().after(blockRule);
		c.setLinewrap().after(f.getBlockAccess().getSemicolonKeyword_1_0());
	}
}

Attachment: MyDsl.zip
(Size: 385.37KB, Downloaded 182 times)

Report message to a moderator

Re: Backtrack and Formatting bug? [message #925962 is a reply to message #924043]

Fri, 28 September 2012 07:18

Sandra Shklyaeva

Messages: 16
Registered: July 2012

Junior Member

Nobody knows?
If this is a bug, probably there is a workaround?

Report message to a moderator

Re: Backtrack and Formatting bug? [message #927062 is a reply to message #924043]

Sat, 29 September 2012 08:06

Sandra Shklyaeva

Messages: 16
Registered: July 2012

Junior Member

I also tried to put some spikes for formatting in the place where backtracking is expected. Unfortunately it doesn't help.

Report message to a moderator

Re: Backtrack and Formatting bug? [message #927109 is a reply to message #927062]

Sat, 29 September 2012 08:56

Henrik Lindberg

Messages: 2509
Registered: July 2009

Senior Member

On 2012-29-09 1:06, Sandra Shklyaeva wrote:
> I also tried to put some spikes for formatting in the place where
> backtracking is expected. Unfortunately it doesn't help.
>
Sometimes it is very hard to figure out how to specifiy rules - ie. to
be able to pinpoint the nodes that identify a particular position in the
source text given only the grammar and a parsed node model.

One way to understand what is going on, is to dump the node model (there
is a method in the NodeModelUtils for that) that can be used when
debugging. I typically use a plugin that adds debug commands to the
editor's context menu, so I can easily print out a Node model dump,
and/or other interesting information from within a text editor. This to
understand what the parsed node model looks like.

Sorry for not being able to give you more concrete advice. Hope it helps
in some way.

Regards
- henrik

Report message to a moderator

Re: Backtrack and Formatting bug? [message #927176 is a reply to message #924043]

Sat, 29 September 2012 10:39

Sandra Shklyaeva

Messages: 16
Registered: July 2012

Junior Member

Dear Henrik,

Thank you very much for your reply.
I tried to use NodeModelUtils.compactDump as you advised.
Unfortunately, the model is correct in both cases (with backtrack and without backtrack).

I even tried the following more simple code for testing.

Formatting works:

var X = 1 ;

for X in X do
	X + X
end ;

Model:

Model: {
  Statement {
    AssignExpr {
      Sequence {
        Expression {
          EqualityExpression {
            FactorialExpression {
              PrimaryExpression {
                Primary {
                  {Local} {
                    Var {
                    }
                    'var' => 'var'
                    ID => 'X'
                    '=' => '='
                    Expression {
                      EqualityExpression {
                        FactorialExpression {
                          PrimaryExpression {
                            Primary {
                              {Literal} {
                                Literal {
                                }
                                INT => '1'
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
  ';' => ';'
  Statement {
    For {
      'for' => 'for'
      Name {
        [Declaration] => 'X'
      }
      'in' => 'in'
      Sequence {
        Expression {
          EqualityExpression {
            FactorialExpression {
              PrimaryExpression {
                Primary {
                  Name {
                    [Declaration] => 'X'
                  }
                }
              }
            }
          }
        }
      }
      'do' => 'do'
      Block {
        Statement {
          AssignExpr {
            Sequence {
              {Plus.left=} {
                Expression {
                  EqualityExpression {
                    FactorialExpression {
                      PrimaryExpression {
                        Primary {
                          Name {
                            [Declaration] => 'X'
                          }
                        }
                      }
                    }
                  }
                }
                '+' => '+'
                EqualityExpression {
                  FactorialExpression {
                    PrimaryExpression {
                      Primary {
                        Name {
                          [Declaration] => 'X'
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
      'end' => 'end'
    }
  }
  ';' => ';'
}

Formatting fails:

var X = 1 ;

for X in X do
	Y => Y end ;

Model:

Model: {
  Statement {
    AssignExpr {
      Sequence {
        Expression {
          EqualityExpression {
            FactorialExpression {
              PrimaryExpression {
                Primary {
                  {Local} {
                    Var {
                    }
                    'var' => 'var'
                    ID => 'X'
                    '=' => '='
                    Expression {
                      EqualityExpression {
                        FactorialExpression {
                          PrimaryExpression {
                            Primary {
                              {Literal} {
                                Literal {
                                }
                                INT => '1'
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
  ';' => ';'
  Statement {
    For {
      'for' => 'for'
      Name {
        [Declaration] => 'X'
      }
      'in' => 'in'
      Sequence {
        Expression {
          EqualityExpression {
            FactorialExpression {
              PrimaryExpression {
                Primary {
                  Name {
                    [Declaration] => 'X'
                  }
                }
              }
            }
          }
        }
      }
      'do' => 'do'
      Block {
        Statement {
          AssignExpr {
            Sequence {
              Expression {
                EqualityExpression {
                  FactorialExpression {
                    PrimaryExpression {
                      Primary {
                        {Rule} {
                          Rule {
                          }
                          {Parameter} {
                            Parameter {
                            }
                            ID => 'Y'
                          }
                          '=>' => '=>'
                          EqualityExpression {
                            FactorialExpression {
                              PrimaryExpression {
                                Primary {
                                  Name {
                                    [Declaration] => 'Y'
                                  }
                                }
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
      'end' => 'end'
    }
  }
  ';' => ';'
}

Report message to a moderator

Re: Backtrack and Formatting bug? [message #936554 is a reply to message #924043]

Mon, 08 October 2012 05:20

Sandra Shklyaeva

Messages: 16
Registered: July 2012

Junior Member

After all I found the solution. Unfortunately the solution is dirty hack, but it works!

In the class org.eclipse.xtext.formatting.impl.ElementMatcherProvider I commented out the following if in the findTransitionPath method.

With this hack, backtracking doesn't break formatting and all works as expected.
What is the purpose of this if? I still cannot understand why it's needed.

protected Pair<List<MatcherTransition>, List<MatcherState>> findTransitionPath(MatcherState from, AbstractElement to, boolean returning, boolean canReturn, Set<Pair<Boolean, MatcherState>> visited) {
//    if (!visited.add(Tuples.create(returning, from)))
//        return null;
    if (from != null) { ...

OR even

protected Pair<List<MatcherTransition>, List<MatcherState>> findTransitionPath(MatcherState from, AbstractElement to, boolean returning, boolean canReturn, Set<Pair<Boolean, MatcherState>> visited) {
    if (!visited.add(Tuples.create(returning, from))) {
//        return null;
    }
    if (from != null) { ...

[Updated on: Mon, 08 October 2012 05:34]

Report message to a moderator

Previous Topic:	How can I autoload default imports from resources
Next Topic:	Formatting nested if expression in switch expression fails

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 19 21:58:13 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter