Home » Archived » IMP » Using JavaCC with IMP?(using javacc with imp to create custom plugin for a language)
| | |
| Re: (no subject) [message #697970 is a reply to message #695552] |
Mon, 18 July 2011 13:48   |
 Robert M. Fuhrer
Messages: 294
Registered: July 2009 |
Senior Member |
|
|
On 7/11/11 9:13 PM, forums-noreply@eclipse.org wrote:
> Hi,
>
> Is it possible for you to summarize what you have done here? I have tinkered with some of the options that are in the
> IMP perspective but couldn't figure out how to embed the JavaCC into the project -though I'm guessing that the .gi and
> .g files will be replaced with the output files the JavaCC prints, am I right about this?-
JavaCC generates a complete parser (a set of .java source files) from its grammar;
there's no runtime. This is a good thing, in a sense (no extra componentns), but
it also results in a bunch of duplicated code in each parser, and there are no
base types/classes that can be used across JavaCC parsers, which means you can't
easily write language-independent code for JavaCC.
Also, you have to be careful to have a lexer specification that covers every possible
input. There's no generic mechanism to produce error tokens for parts of the input
stream that JavaCC can't understand. In other words, there has to be a token rule to
cover any lexically invalid characters in the input stream.
Another annoyance: JavaCC-generated parsers throw exceptions for syntax errors that
often don't carry very much information. So, AFAICT, your error messages may end up
a bit less useful than you'd like. There may be a way to customize this to get more
info into the exceptions, but I'm not sure yet. Also this means that your
IParseController has to catch any exceptions from the parser, extract the info from
them, and pass messages on to the IMessageHandler.
Here's what my draft IParseController implementation looks like now:
public class JavaccParseController extends ParseControllerBase {
public JavaccParseController() {
super(Activator.getInstance().kLanguageID);
}
private JavaCCParser fParser;
private IMessageHandler fMsgHandler;
private Token fFirstToken;
public void initialize(IPath filePath, ISourceProject project, IMessageHandler handler) {
super.initialize(filePath, project, handler);
IPath fullFilePath= project.getRawProject().getLocation().append(filePath);
createLexerAndParser(fullFilePath);
fMsgHandler= handler;
}
public ISourcePositionLocator getSourcePositionLocator() {
return new JavaccSourcePositionLocator();
}
public ILanguageSyntaxProperties getSyntaxProperties() {
return null;
}
private void createLexerAndParser(IPath filePath) {
fParser= new JavaCCParser((Reader) null);
}
public Object parse(String contents, IProgressMonitor monitor) {
StringReader reader= new StringReader(contents);
JavaCCParser.ReInit(reader);
fFirstToken= fParser.token; // Nab the first token before it gets consumed
if (monitor.isCanceled())
return fCurrentAst;
try {
JavaCCParser.javacc_input();
fCurrentAst= fParser.getASTRootNode();
} catch (ParseException e) {
Token errToken = e.currentToken.next;
StringBuffer sb = new StringBuffer();
sb.append("Parse error at token '");
sb.append(errToken);
sb.append("': expected ");
for(int[] expTokens: e.expectedTokenSequences) {
sb.append(e.tokenImage[expTokens[0]]);
sb.append(",");
}
getHandler().handleSimpleMessage(sb.toString(), errToken.offset, errToken.offset +
errToken.image.length()-1, errToken.beginColumn, errToken.endColumn, errToken.beginLine, errToken.endLine);
} catch (TokenMgrError e) {
int startOffset= 0;
try {
startOffset= this.fDocument.getLineOffset(e.errorLine-1) + e.errorColumn;
} catch (BadLocationException e1) {
}
getHandler().handleSimpleMessage(e.getMessage(), startOffset, startOffset, e.errorColumn, e.errorColumn,
e.errorLine-1, e.errorLine-1);
}
return fCurrentAst;
}
public IAnnotationTypeInfo getAnnotationTypeInfo() {
// TODO Auto-generated method stub
return null;
}
public Iterator getTokenIterator(IRegion region) {
return new Iterator() {
Token curToken= fFirstToken.next;
public boolean hasNext() {
return curToken != null;
}
public Object next() {
Token result= curToken;
curToken= curToken.next;
return result;
}
public void remove() {
throw new UnsupportedOperationException();
}
};
}
}
--
Cheers,
-- Bob
--------------------------------
Robert M. Fuhrer
Research Staff Member
Programming Technologies Dept.
IBM T.J. Watson Research Center
IDE Meta-tooling Platform Project Lead (http://www.eclipse.org/imp)
X10: Productive High-Performance Parallel Programming (http://x10-lang.org)
|
|
|
| Re: (no subject) [message #697986 is a reply to message #697970] |
Mon, 18 July 2011 14:45 |
Robert M. Fuhrer
Messages: 294
Registered: July 2009 |
Senior Member |
|
|
Another thing I should have mentioned: you may need to specify a few options to
JavaCC to get it to retain certain kinds of info that are important to an IDE,
like tokens (CACHE_TOKENS) and token positions (KEEP_LINE_COLUMN).
On 7/18/11 9:48 AM, Robert M. Fuhrer wrote:
> On 7/11/11 9:13 PM, forums-noreply@eclipse.org wrote:
>> Hi,
>>
>> Is it possible for you to summarize what you have done here? I have tinkered with some of the options that are in the
>> IMP perspective but couldn't figure out how to embed the JavaCC into the project -though I'm guessing that the .gi and
>> .g files will be replaced with the output files the JavaCC prints, am I right about this?-
>
> JavaCC generates a complete parser (a set of .java source files) from its grammar;
> there's no runtime. This is a good thing, in a sense (no extra componentns), but
> it also results in a bunch of duplicated code in each parser, and there are no
> base types/classes that can be used across JavaCC parsers, which means you can't
> easily write language-independent code for JavaCC.
>
> Also, you have to be careful to have a lexer specification that covers every possible
> input. There's no generic mechanism to produce error tokens for parts of the input
> stream that JavaCC can't understand. In other words, there has to be a token rule to
> cover any lexically invalid characters in the input stream.
>
> Another annoyance: JavaCC-generated parsers throw exceptions for syntax errors that
> often don't carry very much information. So, AFAICT, your error messages may end up
> a bit less useful than you'd like. There may be a way to customize this to get more
> info into the exceptions, but I'm not sure yet. Also this means that your
> IParseController has to catch any exceptions from the parser, extract the info from
> them, and pass messages on to the IMessageHandler.
--
Cheers,
-- Bob
--------------------------------
Robert M. Fuhrer
Research Staff Member
Programming Technologies Dept.
IBM T.J. Watson Research Center
IDE Meta-tooling Platform Project Lead (http://www.eclipse.org/imp)
X10: Productive High-Performance Parallel Programming (http://x10-lang.org)
|
|
| |
Goto Forum:
Current Time: Fri Apr 26 20:06:34 GMT 2024
Powered by FUDForum. Page generated in 0.04810 seconds |
] 
Search
Help
Register
Login
Home
