Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Proposals crash when within quotes
Proposals crash when within quotes [message #1154170] Fri, 25 October 2013 03:32 Go to next message
Chris Ainsley is currently offline Chris AinsleyFriend
Messages: 78
Registered: March 2010
Location: UK
Member
Hello,

Proposals are crashing the whole IDE (full hang that requires a kill of the process) when I try to activate a code completion within quoted text and there is one or more letters in the quoted text.

I have debugged it to line 253 in the org.eclipse.xtext.ui.editor.contentassist.antlr.ParserBasedContentAssistContextFactory.createContextsForLastCompleteNode(EObject previousModel, boolean strict) method:

	Token token = lexer.nextToken();


Strangely, if I execute the completion without any text, all the correct completions are proposed, or even if I activate the completion after an existing quoted text (but not within) it works. It just crashes Eclipse if I try to activate the completion within the quoted text itself.

I'm not using a value converter for quoted text so I don't quite know why this is happening. This is not a new problem, I've never been able to get this to work.

I attach a link to the video of the symptoms here : https://drive.google.com/file/d/0B1pmqE1cXkSDeTFoRmV5T0o5Z2M/edit?usp=sharing


This is a long lasting bug, I've avoided it for about 5 months but sometimes I accidentally press control and space inside quotes and I lose all my unsaved work.

If any XText committers wish to step debug this on my machine via Skype then I'm available via the id associated with my profile. I simply don't have the knowledge of the completion architecture to know why the lexer is breaking only in this single context.

** NOTE :: I'm running the most recent 2.4.3 version of XText.
** NOTE 2 :: In addition to the demonstration in the video, if trying to perform code completion with the cursor between two double quotes then this works fine. The only case that crashes Eclipse is when the cursor is between the quotes but after one or more alphanumerics (corresponding to the STRING terminal)

[Updated on: Fri, 25 October 2013 05:59]

Report message to a moderator

Re: Proposals crash when within quotes [message #1154377 is a reply to message #1154170] Fri, 25 October 2013 06:46 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13488
Registered: July 2009
Senior Member
Hmmm strange,

this does not happen with a simple grammar like this one

grammar org.xtext.example.mydsl2.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "http://www.xtext.org/example/mydsl2/MyDsl"

Model:
	greetings+=Greeting*;
	
Greeting:
	'Hello' name=STRING '!';


// generates a more lightweight Antlr parser and lexer tailored for content assist
    		fragment = parser.antlr.XtextAntlrUiGeneratorFragment auto-inject {}
    


so there must be something strange about yours


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Proposals crash when within quotes [message #1155164 is a reply to message #1154377] Fri, 25 October 2013 18:43 Go to previous messageGo to next message
Chris Ainsley is currently offline Chris AinsleyFriend
Messages: 78
Registered: March 2010
Location: UK
Member
OK, I've dug down into the lexing code more and I think I know why XText/Eclipse/Antlr is crashing.

I'm using a custom lexer which is working perfectly in all but this context. Now the problem that I seem to be aware of is that the custom lexer uses some state flags that are initialized from the context of the beginning of a file.

Upon digging deep in the xtext content assist code, I can see that the lexer is being invoked at any given location when the content assist is invoked without lexing the portion of the document before the current token. The initial state of lexer is such that certain tokens are not recognised (such as STRING).

Here is a snippet from my custom lexer

@members{
  private boolean isArray = false;
  private boolean isShorthandLine = false;
  private boolean isPropertiesValueContext = false;
  private boolean isAtBeginningOfLine = true;
}

...

RULE_STRING : { !isPropertiesValueContext }?=> '"' ('\\' ('b'|'t'|'n'|'f'|'r'|'"'|'\''|'\\')|~(('\\'|'"')))* '"' {isAtBeginningOfLine = false; };

RULE_PROPERTY_VALUE : { isPropertiesValueContext }?=> ~(('\n'|'\r'))+ {isPropertiesValueContext=false; isAtBeginningOfLine = false;};

...

RULE_ANY_OTHER : .;



Now obviously, because isAtBeginningOfLine=true, RULE_STRING tokens are not able to be lexed, and instead RULE_PROPERTY_VALUE are able to be lexed. I'm not sure why exactly this crashes (the crash seems to occur in the nextToken() method within ANTLR, during a while(true) { .... } block, which never exits.

So, I guess my question is now very specific. How do I make my custom lexer compatible with code completion? Should the code completion lexer be completely without state? Or is there a flag to change the way that the code completion works so that it invokes the lexer on the whole document up to the current carat in the document, not just the partial token prior to the carat?

[Updated on: Fri, 25 October 2013 18:44]

Report message to a moderator

Re: Proposals crash when within quotes [message #1155780 is a reply to message #1155164] Sat, 26 October 2013 04:31 Go to previous message
Chris Ainsley is currently offline Chris AinsleyFriend
Messages: 78
Registered: March 2010
Location: UK
Member
OK, so I haven't solved the issue, but I've isolated it into a project set which I attach here. The custom lexer is contained here : org.xtext.example.mydsl/src/org/xtext/example/mydsl/MyDslLexer.g




  1. Everything should build fine with MWE2 script.
  2. Launch a debug instance of eclipse and copy across test.mydsl
  3. Go to the quoted string and after "an ... press ctrl+space (but before y").
  4. Eclipse will lock, put a breakpoint on org.eclipse.xtext.ui.editor.contentassist.antlr.ParserBasedContentAssistContextFactory.createContextsForLastCompleteNode(EObject previousModel, boolean strict), line 253.


I hope this helps someone far smarter than myself to find the issue with my lexer. It may of course be a configuration problem or it may be an issue with a code/fragment generator, I just don't know. It seems that others have had a variation on this issue in the past so it would be useful to understand the triggers and workarounds.

** Again, for the record, I'm using XText 2.4.3.

I do hope someone can figure this out.

Chris

Previous Topic:Customizing "Source and Libraries" of "Java Build Path" in Eclipse Platform laun
Next Topic:JvmModelInferrer and JvmModelGenerator: generating error-less elements
Goto Forum:
  


Current Time: Mon Jul 06 02:53:13 GMT 2020

Powered by FUDForum. Page generated in 0.01317 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top