Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Archived » IMP » Using JavaCC with IMP?(using javacc with imp to create custom plugin for a language)
Using JavaCC with IMP? [message #695046] Sun, 10 July 2011 17:07 Go to next message
sultanikincikopru is currently offline sultanikincikopru
Messages: 5
Registered: July 2011
Junior Member
Hello everyone, I am building a language plugin and I wanted to know if I could use JavaCC for IMP. The documentation section for using JavaCC with IMP says "coming soon", are there any examples/pointers that I can use to learn using JavaCC with IMP? Thanks.
(no subject) [message #695546 is a reply to message #695046] Mon, 11 July 2011 20:39 Go to previous messageGo to next message
Robert M. Fuhrer is currently offline Robert M. Fuhrer
Messages: 294
Registered: July 2009
Senior Member
On 7/10/11 5:07 PM, forums-noreply@eclipse.org wrote:
> Hello everyone, I am building a language plugin and I wanted to know if I could use JavaCC for IMP. The documentation
> section for using JavaCC with IMP says "coming soon", are there any examples/pointers that I can use to learn using
> JavaCC with IMP? Thanks.

Yes, you should be able to use JavaCC with IMP. I've got some projects in my
workspace for JavaCC support that are nearing completion. It will probably
still be a couple of weeks yet before I commit them, though.

--
Cheers,
-- Bob

--------------------------------
Robert M. Fuhrer
Research Staff Member
Programming Technologies Dept.
IBM T.J. Watson Research Center

IDE Meta-tooling Platform Project Lead (http://www.eclipse.org/imp)
X10: Productive High-Performance Parallel Programming (http://x10-lang.org)
Re: (no subject) [message #695552 is a reply to message #695546] Mon, 11 July 2011 21:13 Go to previous messageGo to next message
sultanikincikopru is currently offline sultanikincikopru
Messages: 5
Registered: July 2011
Junior Member
Hi,

Is it possible for you to summarize what you have done here? I have tinkered with some of the options that are in the IMP perspective but couldn't figure out how to embed the JavaCC into the project -though I'm guessing that the .gi and .g files will be replaced with the output files the JavaCC prints, am I right about this?-

Thanks.
Re: (no subject) [message #697970 is a reply to message #695552] Mon, 18 July 2011 09:48 Go to previous messageGo to next message
Robert M. Fuhrer is currently offline Robert M. Fuhrer
Messages: 294
Registered: July 2009
Senior Member
On 7/11/11 9:13 PM, forums-noreply@eclipse.org wrote:
> Hi,
>
> Is it possible for you to summarize what you have done here? I have tinkered with some of the options that are in the
> IMP perspective but couldn't figure out how to embed the JavaCC into the project -though I'm guessing that the .gi and
> .g files will be replaced with the output files the JavaCC prints, am I right about this?-

JavaCC generates a complete parser (a set of .java source files) from its grammar;
there's no runtime. This is a good thing, in a sense (no extra componentns), but
it also results in a bunch of duplicated code in each parser, and there are no
base types/classes that can be used across JavaCC parsers, which means you can't
easily write language-independent code for JavaCC.

Also, you have to be careful to have a lexer specification that covers every possible
input. There's no generic mechanism to produce error tokens for parts of the input
stream that JavaCC can't understand. In other words, there has to be a token rule to
cover any lexically invalid characters in the input stream.

Another annoyance: JavaCC-generated parsers throw exceptions for syntax errors that
often don't carry very much information. So, AFAICT, your error messages may end up
a bit less useful than you'd like. There may be a way to customize this to get more
info into the exceptions, but I'm not sure yet. Also this means that your
IParseController has to catch any exceptions from the parser, extract the info from
them, and pass messages on to the IMessageHandler.

Here's what my draft IParseController implementation looks like now:

public class JavaccParseController extends ParseControllerBase {
public JavaccParseController() {
super(Activator.getInstance().kLanguageID);
}

private JavaCCParser fParser;
private IMessageHandler fMsgHandler;
private Token fFirstToken;

public void initialize(IPath filePath, ISourceProject project, IMessageHandler handler) {
super.initialize(filePath, project, handler);
IPath fullFilePath= project.getRawProject().getLocation().append(filePath);
createLexerAndParser(fullFilePath);

fMsgHandler= handler;
}

public ISourcePositionLocator getSourcePositionLocator() {
return new JavaccSourcePositionLocator();
}

public ILanguageSyntaxProperties getSyntaxProperties() {
return null;
}

private void createLexerAndParser(IPath filePath) {
fParser= new JavaCCParser((Reader) null);
}

public Object parse(String contents, IProgressMonitor monitor) {
StringReader reader= new StringReader(contents);

JavaCCParser.ReInit(reader);

fFirstToken= fParser.token; // Nab the first token before it gets consumed

if (monitor.isCanceled())
return fCurrentAst;

try {
JavaCCParser.javacc_input();

fCurrentAst= fParser.getASTRootNode();
} catch (ParseException e) {
Token errToken = e.currentToken.next;
StringBuffer sb = new StringBuffer();
sb.append("Parse error at token '");
sb.append(errToken);
sb.append("': expected ");
for(int[] expTokens: e.expectedTokenSequences) {
sb.append(e.tokenImage[expTokens[0]]);
sb.append(",");
}
getHandler().handleSimpleMessage(sb.toString(), errToken.offset, errToken.offset +
errToken.image.length()-1, errToken.beginColumn, errToken.endColumn, errToken.beginLine, errToken.endLine);
} catch (TokenMgrError e) {
int startOffset= 0;

try {
startOffset= this.fDocument.getLineOffset(e.errorLine-1) + e.errorColumn;
} catch (BadLocationException e1) {
}
getHandler().handleSimpleMessage(e.getMessage(), startOffset, startOffset, e.errorColumn, e.errorColumn,
e.errorLine-1, e.errorLine-1);
}

return fCurrentAst;
}

public IAnnotationTypeInfo getAnnotationTypeInfo() {
// TODO Auto-generated method stub
return null;
}

public Iterator getTokenIterator(IRegion region) {
return new Iterator() {
Token curToken= fFirstToken.next;
public boolean hasNext() {
return curToken != null;
}
public Object next() {
Token result= curToken;
curToken= curToken.next;
return result;
}
public void remove() {
throw new UnsupportedOperationException();
}
};
}
}


--
Cheers,
-- Bob

--------------------------------
Robert M. Fuhrer
Research Staff Member
Programming Technologies Dept.
IBM T.J. Watson Research Center

IDE Meta-tooling Platform Project Lead (http://www.eclipse.org/imp)
X10: Productive High-Performance Parallel Programming (http://x10-lang.org)
Re: (no subject) [message #697986 is a reply to message #697970] Mon, 18 July 2011 10:45 Go to previous messageGo to next message
Robert M. Fuhrer is currently offline Robert M. Fuhrer
Messages: 294
Registered: July 2009
Senior Member
Another thing I should have mentioned: you may need to specify a few options to
JavaCC to get it to retain certain kinds of info that are important to an IDE,
like tokens (CACHE_TOKENS) and token positions (KEEP_LINE_COLUMN).

On 7/18/11 9:48 AM, Robert M. Fuhrer wrote:
> On 7/11/11 9:13 PM, forums-noreply@eclipse.org wrote:
>> Hi,
>>
>> Is it possible for you to summarize what you have done here? I have tinkered with some of the options that are in the
>> IMP perspective but couldn't figure out how to embed the JavaCC into the project -though I'm guessing that the .gi and
>> .g files will be replaced with the output files the JavaCC prints, am I right about this?-
>
> JavaCC generates a complete parser (a set of .java source files) from its grammar;
> there's no runtime. This is a good thing, in a sense (no extra componentns), but
> it also results in a bunch of duplicated code in each parser, and there are no
> base types/classes that can be used across JavaCC parsers, which means you can't
> easily write language-independent code for JavaCC.
>
> Also, you have to be careful to have a lexer specification that covers every possible
> input. There's no generic mechanism to produce error tokens for parts of the input
> stream that JavaCC can't understand. In other words, there has to be a token rule to
> cover any lexically invalid characters in the input stream.
>
> Another annoyance: JavaCC-generated parsers throw exceptions for syntax errors that
> often don't carry very much information. So, AFAICT, your error messages may end up
> a bit less useful than you'd like. There may be a way to customize this to get more
> info into the exceptions, but I'm not sure yet. Also this means that your
> IParseController has to catch any exceptions from the parser, extract the info from
> them, and pass messages on to the IMessageHandler.


--
Cheers,
-- Bob

--------------------------------
Robert M. Fuhrer
Research Staff Member
Programming Technologies Dept.
IBM T.J. Watson Research Center

IDE Meta-tooling Platform Project Lead (http://www.eclipse.org/imp)
X10: Productive High-Performance Parallel Programming (http://x10-lang.org)
Re: (no subject) [message #698049 is a reply to message #697986] Mon, 18 July 2011 13:25 Go to previous message
sultanikincikopru is currently offline sultanikincikopru
Messages: 5
Registered: July 2011
Junior Member
Thanks a lot for the information, I will look into these shortly.
Previous Topic:LPG and building IMP from source
Next Topic:Problem running the LEG example?
Goto Forum:
  


Current Time: Fri Apr 18 16:46:05 EDT 2014

Powered by FUDForum. Page generated in 0.06122 seconds