| Antlr failure on complicated grammar [message #524723] |
Fri, 02 April 2010 03:17  |
 Thom Boyer
Messages: 7
Registered: April 2010 |
Junior Member |
|
|
I am trying to write a grammar for a Java-like language using xtext. Things have been going quite well, but I seem to have reached a threshold of complexity where Antlr fails. The interesting thing is that as I modify my grammar, simplifying and complexifying it, I'm finding that Antlr's results are not deterministic: at a certain point of grammar complexity, Antlr manages to generate a lexer and parser for the grammar but only if I'm not otherwise tasking my CPU. If I play music while Antlr is running, then Antlr fails:
warning(205): ../com.sample.xtext.sky.ui/src-gen/com/sample/ui/contentassi st/antlr/internal/InternalSky.g:1:8: ANTLR could not analyze this decision in rule Tokens; often this is because of recursive rule references visible from the left edge of alternatives. ANTLR will re-analyze the decision with a fixed lookahead of k=1. Consider using "options {k=1;}" for that decision and possibly adding a syntactic predicate.
warning(209): ../com.sample.xtext.sky.ui/src-gen/com/sample/ui/contentassi st/antlr/internal/InternalSky.g:10594:1: Multiple token rules can match input such as "'s'": T23, T89, RULE_ID, RULE_ANY_OTHER
As a result, tokens(s) T89,RULE_ID,RULE_ANY_OTHER were disabled for that input
The second and third line ("Multiple token rules can match input such as..." and "As a result, tokens(s) ... were disabled for that input") are repeated many times, for different tokens.
It appears (from looking at http://www.antlr.org/pipermail/antlr-interest/2009-May/03456 3.html) that Antlr actually uses a timer to decide when to quit its attempt to generate a DFA from an NFA. And there is a commandline option for changing the timer duration, -Xconversiontimeout.
It appears that I need to get xtext to pass something like "-Xconverstiontimeout 10000" to antlr when I run my GenerateLang.mwe2 as "MWE2 Workflow". Can anybody tell me how to do that?
Or perhaps I'm completely wrong about what's going on here. Any tips would be appreciated.
=thom
|
|
|
| Re: Antlr failure on complicated grammar [message #524732 is a reply to message #524723] |
Fri, 02 April 2010 05:43  |
Sven Efftinge
Messages: 1823
Registered: July 2009 |
Senior Member |
|
|
Thom Boyer schrieb:
> I am trying to write a grammar for a Java-like language using xtext.
> Things have been going quite well, but I seem to have reached a
> threshold of complexity where Antlr fails. The interesting thing is that
> as I modify my grammar, simplifying and complexifying it, I'm finding
> that Antlr's results are not deterministic: at a certain point of
> grammar complexity, Antlr manages to generate a lexer and parser for the
> grammar but only if I'm not otherwise tasking my CPU. If I play music
> while Antlr is running, then Antlr fails:
What kind of music? ... (just kidding, hihi)
> It appears that I need to get xtext to pass something like
> "-Xconverstiontimeout 10000" to antlr when I run my GenerateLang.mwe2 as
> "MWE2 Workflow". Can anybody tell me how to do that?
You need to create a subclass of
de.itemis.xtext.antlr.XtextAntlrGeneratorFragment and override generate.
Just copy the five lines from the original body and modify the call to
AntlrToolRunner,run() to use
de.itemis.xtext.antlr.AntlrToolRunner.runWithParams(String, String...)
and pass the needed arguments.
You also need to override to
@Override
protected String getTemplate() {
return XtextAntlrGeneratorFragment.class.getName().replaceAll("\\. ",
"::");
}
>
> Or perhaps I'm completely wrong about what's going on here. Any tips
> would be appreciated.
Sounds strange indeed and you're the first with this effect, but it
makes sense from your description.
Sven
--
Need professional support for Xtext and EMF?
Go to: http://xtext.itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de
|
|
|
|
| Re: Antlr failure on complicated grammar [message #524790 is a reply to message #524745] |
Fri, 02 April 2010 14:20 |
Thom Boyer
Messages: 7
Registered: April 2010 |
Junior Member |
|
|
Increasing the heap allocated to the MWE2 process was sufficient to solve the problem.
I put "-Xmx2000m" into the "VM arguments" field of the "Arguments" tab of the "Run Configurations..." context menu of "GenerateMyLang.mwe2", and the full grammar processed without errors.
Thanks so much!
=thom
=thom
|
|
|
Powered by
FUDForum. Page generated in 0.01942 seconds
] 
Search
Help
Register
Login
Home
