| Re: Eclipse Java Parser (JikesPG) vs. ANTLR [message #192236] |
Tue, 11 January 2005 00:52  |
Eclipse User |
|
|
|
Originally posted by: tonychen.ca.ibm.com
Hi JDT folks,
I'm doing some research about the Java Parser used in Eclipse, and was
redirected to this newsgroup. The reason I'm interested in the Java
Parser used in Eclipse is we found ANTLR a bit slow in our project. Can
anyone here show me some basics of how to use Jikes Parser Generator
(the tool used to generate Eclipse Java Parser as per John Arthorne).
Some pros and cons comparing JikesPG and ANTLR would also be highly
appreciated. Thanks in advance!
Tony
### John Arthorne wrote: ###
> The Eclipse Java parser is generated with the Jikes parser generator.
> More information can be found at:
>
>
http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/jdt- core-home/howto/generate%20parser/generateParser.html
>
>
> If you have further questions, I suggest asking on the eclipse.tools.jdt
> newsgroup, where people with more knowledge of the parser may be able to
> give you more details.
> --
>
> Tony Chen wrote:
>
>> Hi Everyone,
>>
>> Happy New Year! We used Antlr, http://www.antlr.org/ , to parse Java
>> source code, and found the performance to be not acceptable. So, I'm
>> wondering what parser Eclipse is using to parse the Java source code.
>> Is there a way to reuse the parser in our own project? Thanks,
>>
>> Regards,
>>
>> Tony
>>
>
|
|
|
| Re: Eclipse Java Parser (JikesPG) vs. ANTLR [message #192343 is a reply to message #192236] |
Tue, 11 January 2005 13:50  |
| Eclipse User |
|
|
|
Originally posted by: olivier_thomannNOSPAM.ca.ibm.com
Tony Chen a écrit :
> I'm doing some research about the Java Parser used in Eclipse, and was
> redirected to this newsgroup. The reason I'm interested in the Java
> Parser used in Eclipse is we found ANTLR a bit slow in our project. Can
> anyone here show me some basics of how to use Jikes Parser Generator
> (the tool used to generate Eclipse Java Parser as per John Arthorne).
> Some pros and cons comparing JikesPG and ANTLR would also be highly
> appreciated. Thanks in advance!
This is a tough question.
If you want to use our grammar, then it is not too difficult. You don't
need to generate a new parser. You can simply change the existing parser
implementation.
If you want to change the grammar, this is much trickier.
I will assume that you want to keep the existing grammar. You need to
change the existing semantic actions for each rule to create your nodes.
Look at the consumexxxx methods in the class
org.eclipse.jdt.internal.compiler.parser.Parser. There are special
methods that are really important:
1) consumeRule(int) is the method that says what action should be done
when a specific rule is reduced.
2) consumeToken(int) is the method that defines what action is doing the
parser when a certain token is read from the source file.
3) For a full list of token, look at
org.eclipse.jdt.internal.compiler.parser.TerminalTokens.
4) We use a trick to have multiple goals in the grammar. See
getFirstToken() and the grammar file in the grammars folder in the
org.eclipse.jdt.core projects.
5) You need to
org.eclipse.jdt.internal.compiler.parser.ParserBasicInformat ion
interface. This is used by the parsing loop.
If you don't change the grammar, then you don't need to change the
interface org.eclipse.jdt.internal.compiler.parser.TerminalTokens. It
contains the right token types.
The parser has different stacks that are used to store information
(identifiers, positions, reduced nodes, etc.). You can change this.
You also need all the resources files that are used to initialize the
parser tables.
Note that all this code is in an internal package. So you should copy it
for your project. It can be changed without any warning since it is not
supposed to be called by external plugins.
Good luck,
Olivier
|
|
|
| SableCC, was Re: Eclipse Java Parser (JikesPG) vs. ANTLR [message #192469 is a reply to message #192236] |
Wed, 12 January 2005 11:33 |
| Eclipse User |
|
|
|
Originally posted by: joerg.von.frantzius.artnology.nospam.com
Hi Tony,
if you are looking for a good compiler compiler, I suggest taking a look
at SableCC. I used it a while ago, and I thought it was brilliantly
engineered. I'd be surprised if its author would unnecessarily
compromise performance anywhere.
With SableCC, your grammar description is separated from any Java code,
and it generates Visitor classes that you can subclass to do whatever
you need to to when traversing the AST. That way, any generated code
stays separated from your hand-written code.
A lot of grammars exist for SableCC, also some for the Java syntax.
Tony Chen schrieb:
> Hi JDT folks,
>
> I'm doing some research about the Java Parser used in Eclipse, and was
> redirected to this newsgroup. The reason I'm interested in the Java
> Parser used in Eclipse is we found ANTLR a bit slow in our project.
> Can anyone here show me some basics of how to use Jikes Parser
> Generator (the tool used to generate Eclipse Java Parser as per John
> Arthorne). Some pros and cons comparing JikesPG and ANTLR would also
> be highly appreciated. Thanks in advance!
>
> Tony
>
>
> ### John Arthorne wrote: ###
>
> > The Eclipse Java parser is generated with the Jikes parser generator.
> > More information can be found at:
> >
> >
> http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/jdt- core-home/howto/generate%20parser/generateParser.html
>
> >
> >
> > If you have further questions, I suggest asking on the
> eclipse.tools.jdt
> > newsgroup, where people with more knowledge of the parser may be
> able to
> > give you more details.
> > --
> >
> > Tony Chen wrote:
> >
> >> Hi Everyone,
> >>
> >> Happy New Year! We used Antlr, http://www.antlr.org/ , to parse Java
> >> source code, and found the performance to be not acceptable. So, I'm
> >> wondering what parser Eclipse is using to parse the Java source code.
> >> Is there a way to reuse the parser in our own project? Thanks,
> >>
> >> Regards,
> >>
> >> Tony
> >>
> >
>
|
|
|
Powered by
FUDForum. Page generated in 0.04470 seconds
Home