Skip to main content



      Home
Home » Language IDEs » Java Development Tools (JDT) » There is a way to use ASTParser outside Eclipse?
There is a way to use ASTParser outside Eclipse? [message #258486] Tue, 03 February 2009 22:55 Go to next message
Eclipse UserFriend
Hello, all. In advance, my apologies for the poor English.

Searching on the web I found a thread from this group (06/05/2008) about
to use the ASTParser functionalities on a standalone application. The main
answers in the thread are these:

http://dev.eclipse.org/newslists/news.eclipse.tools.jdt/msg2 2658.html
http://dev.eclipse.org/newslists/news.eclipse.tools.jdt/msg2 2687.html
http://dev.eclipse.org/newslists/news.eclipse.tools.jdt/msg2 2691.html

In the end of discussion, no objective answer was given. My question is
exactly the same of Marco, who started the thread. There is a way (not
necessarily a beautiful way :) ) to use the Eclipse AST capabilities (with
the right libraries on runtime, as he explained) on a standalone context?

I'll be very grateful for any answer about this.

Thanks,

Paulo R.
Re: There is a way to use ASTParser outside Eclipse? [message #514896 is a reply to message #258486] Tue, 16 February 2010 23:27 Go to previous messageGo to next message
Eclipse UserFriend
I'm facing the same problem. In luck in getting full AST navigation and binding resolving working in a non-plugin, standalone application?

Cheers,
Erik
Re: There is a way to use ASTParser outside Eclipse? [message #514952 is a reply to message #514896] Wed, 17 February 2010 00:22 Go to previous messageGo to next message
Eclipse UserFriend
Erik van Zijst wrote:

> I'm facing the same problem. In luck in getting full AST navigation and
> binding resolving working in a non-plugin, standalone application?
>
> Cheers,
> Erik

Well the same structures are used in ECJ, which is outside Eclipse. But
binding resolution seems not even to work properly inside Eclipse. I
reported recently a problem with null Scope values being passes into
ASTVisitor methods - and no one seems to be able to see why.

You might want to look at the Lombok project, which is trying to address
these issues in a platform independent way, with a common model to handle
both Javac and Eclipse. The new Lombok way of doing things has a fully
navigable AST, you move in all directions from a given node as it is fully
linked. I have written a resolution (finding local/argument/field
definitions, qualifying names from package and imports, and binding to
binary objects from the classpath and finding things like method return
types. In the long run this new Lombok model will have a resolution
mechanism (whether it includes any of my code remains to be seen).

David
Re: There is a way to use ASTParser outside Eclipse? [message #515113 is a reply to message #514896] Wed, 17 February 2010 13:07 Go to previous messageGo to next message
Eclipse UserFriend
Le 2010-02-16 23:28, Erik van Zijst a écrit :
> I'm facing the same problem. In luck in getting full AST navigation and
> binding resolving working in a non-plugin, standalone application?
Please add yourself to the cc list of:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=206391

I plan to fix it for 3.6 if I get the proper API in place.
--
Olivier
Re: There is a way to use ASTParser outside Eclipse? [message #515114 is a reply to message #514952] Wed, 17 February 2010 13:11 Go to previous messageGo to next message
Eclipse UserFriend
Le 2010-02-17 05:09, David Goodenough a écrit :
> Well the same structures are used in ECJ, which is outside Eclipse. But
> binding resolution seems not even to work properly inside Eclipse. I
> reported recently a problem with null Scope values being passes into
> ASTVisitor methods - and no one seems to be able to see why.
Did you open a bug report for this? I cannot find it.
--
Olivier
Re: There is a way to use ASTParser outside Eclipse? [message #515116 is a reply to message #515114] Wed, 17 February 2010 13:36 Go to previous messageGo to next message
Eclipse UserFriend
Olivier Thomann wrote:

> Le 2010-02-17 05:09, David Goodenough a écrit :
>> Well the same structures are used in ECJ, which is outside Eclipse. But
>> binding resolution seems not even to work properly inside Eclipse. I
>> reported recently a problem with null Scope values being passes into
>> ASTVisitor methods - and no one seems to be able to see why.
> Did you open a bug report for this? I cannot find it.
> --
> Olivier

No because I did not know whether it was a bug. So I asked a question on
this forum, and no-one answer it. I would not presume that it was a
bug in Eclipse (although it might be) as I am doing this inside Lombok
and that does horrible things to Eclipse that you (the authors) might
not wish to be be associated with.

David
Re: There is a way to use ASTParser outside Eclipse? [message #515160 is a reply to message #515116] Wed, 17 February 2010 17:26 Go to previous messageGo to next message
Eclipse UserFriend
Le 2010-02-17 13:36, David Goodenough a écrit :
> No because I did not know whether it was a bug. So I asked a question on
> this forum, and no-one answer it. I would not presume that it was a
> bug in Eclipse (although it might be) as I am doing this inside Lombok
> and that does horrible things to Eclipse that you (the authors) might
> not wish to be be associated with.
I am trying to follow this newsgroup as much as I can, but sometimes I
am missing messages.
So if you have a reproducable test case, pleas open a bug report and
I'll have a look at it.
Thanks,
--
Olivier
Re: There is a way to use ASTParser outside Eclipse? [message #515269 is a reply to message #515160] Thu, 18 February 2010 05:21 Go to previous messageGo to next message
Eclipse UserFriend
Olivier Thomann wrote:

> Le 2010-02-17 13:36, David Goodenough a écrit :
>> No because I did not know whether it was a bug. So I asked a question on
>> this forum, and no-one answer it. I would not presume that it was a
>> bug in Eclipse (although it might be) as I am doing this inside Lombok
>> and that does horrible things to Eclipse that you (the authors) might
>> not wish to be be associated with.
> I am trying to follow this newsgroup as much as I can, but sometimes I
> am missing messages.
> So if you have a reproducable test case, pleas open a bug report and
> I'll have a look at it.
> Thanks,
> --
> Olivier

The basic question was as to why the scope might be null, and what I
was supposed to do when it happened. I have come across comments that
this might happen when serious errors have occurred earlier in the
process, but in this particular case there were no other errors (that
I could see). I can try to put together a test case (although it will
involve the basic Lombok code as well as my handler) but I would like
to understand if there are circumstances when I am supposed to get null
and action should be taken by an ASTVisitor if this happens.

David
Re: There is a way to use ASTParser outside Eclipse? [message #515332 is a reply to message #515269] Thu, 18 February 2010 04:08 Go to previous messageGo to next message
Eclipse UserFriend
Le 2010-02-18 05:21, David Goodenough a écrit :
> The basic question was as to why the scope might be null, and what I
> was supposed to do when it happened. I have come across comments that
> this might happen when serious errors have occurred earlier in the
> process, but in this particular case there were no other errors (that
> I could see). I can try to put together a test case (although it will
> involve the basic Lombok code as well as my handler) but I would like
> to understand if there are circumstances when I am supposed to get null
> and action should be taken by an ASTVisitor if this happens.
I would need to know when you are calling the ASTVisitor.
After cleanup is called, most of the scopes are null.

If you give me an example, I can have a look.
--
Olivier
Re: There is a way to use ASTParser outside Eclipse? [message #515395 is a reply to message #515332] Thu, 18 February 2010 11:18 Go to previous messageGo to next message
Eclipse UserFriend
Olivier Thomann wrote:

> Le 2010-02-18 05:21, David Goodenough a écrit :
>> The basic question was as to why the scope might be null, and what I
>> was supposed to do when it happened. I have come across comments that
>> this might happen when serious errors have occurred earlier in the
>> process, but in this particular case there were no other errors (that
>> I could see). I can try to put together a test case (although it will
>> involve the basic Lombok code as well as my handler) but I would like
>> to understand if there are circumstances when I am supposed to get null
>> and action should be taken by an ASTVisitor if this happens.
> I would need to know when you are calling the ASTVisitor.
> After cleanup is called, most of the scopes are null.
>
> If you give me an example, I can have a look.
> --
> Olivier

It is happening during Annotation processing.

I will put together an example, but it will require Lombok, and is
invoked through Lombok rather than directly from the eclipse compiler.
It may that this is a problem in Lombok, but they (the authors of Lombok)
seem to think not.

I am afraid that getting an example of this is not going to be simple,
which is why I was trying to find out in principle what having a null
scope might mean.

You can find the source for lombok on github at:-

git://github.com/rzwitserloot/lombok.git

My code can be found at:-

git://dga.co.uk/git/lombok-beans.git

There is some detail of Lombok at projectlombok.org and of my beans code
at dga.co.uk/lombokbeans

The problem occurs in the HandleFieldRef class. At the top of this is a
visitCompilationUnit method, to which is passed a CompilationUnit which
contains a Scope. The visitCompilationUnit code is called from within
Lombok for each object which has @ProviderFor(EclipseASTVisitor.class).
Exactly where the scope value comes from I am not sure, but my guess
is that it is whatever was passed to Lombok. Lombok hooks itself into
Eclipse in a rather odd way, it is (as I understand it) not entirely
an Annotation processor, but it is at about that time in the compilation.

That scope is sometimes null, and so when I call my
inner ASTVisitor it has no Scopes to do resolution with. If I add a
line in the visitCompilationUnit method which reads:-

if(unit.scope == null) top.addError("Null scope found");

then I will get errors displayed at the top of some of my source code.

This seems to happen most on modules with inner classes, and also
sometimes in constructors. I have yet to spot any other patterns
than these.

You will need to install lombok.jar (which you build using ant from
the directory that you download lombok.git into) and also beans.jar
(which you get from running ant on the directory used for lombok-beans.git).
If you run the lombok jar it will run up an installer, and tell you
where it is installing itself (in the same directory as eclipse.ini)
and beans.jar needs to be installed there as well.

I will dig out some samples of classes that cause the problem, but
the problem is that the ones I am using are part of a larger framework
and getting all of that (and its prereqs) to you would be difficult.

David
Re: There is a way to use ASTParser outside Eclipse? [message #515439 is a reply to message #515395] Thu, 18 February 2010 09:00 Go to previous messageGo to next message
Eclipse UserFriend
Le 2010-02-18 11:18, David Goodenough a écrit :
> That scope is sometimes null, and so when I call my
> inner ASTVisitor it has no Scopes to do resolution with. If I add a
> line in the visitCompilationUnit method which reads:-
>
> if(unit.scope == null) top.addError("Null scope found");
Ok, I'll try to reproduce. I might need a few days to install all the
required pieces.
--
Olivier
Re: There is a way to use ASTParser outside Eclipse? [message #515470 is a reply to message #515439] Thu, 18 February 2010 15:26 Go to previous messageGo to next message
Eclipse UserFriend
Olivier Thomann wrote:

> Le 2010-02-18 11:18, David Goodenough a écrit :
>> That scope is sometimes null, and so when I call my
>> inner ASTVisitor it has no Scopes to do resolution with. If I add a
>> line in the visitCompilationUnit method which reads:-
>>
>> if(unit.scope == null) top.addError("Null scope found");
> Ok, I'll try to reproduce. I might need a few days to install all the
> required pieces.
> --
> Olivier

Rather than setting the error, I threw an exception. And this is what
I got in the log. As you will see Lombok inserts itself rather
unnaturally into the Parser. I have not put the whole of the stack trace
in here, but here are the top 20 or so:-

at
lombok.eclipse.handlers.HandleFieldRef.visitCompilationUnit( HandleFieldRef.java:31)
at lombok.eclipse.EclipseNode.traverse(EclipseNode.java:55)
at lombok.eclipse.EclipseAST.traverse(EclipseAST.java:86)
at
lombok.eclipse.HandlerLibrary.callASTVisitors(HandlerLibrary .java:178)
at
lombok.eclipse.TransformEclipseAST.go(TransformEclipseAST.ja va:161)
at
lombok.eclipse.TransformEclipseAST.transform(TransformEclips eAST.java:109)
at
lombok.eclipse.TransformEclipseAST.transform_swapped(Transfo rmEclipseAST.java:84)
at
org.eclipse.jdt.internal.compiler.parser.Parser.endParse(Par ser.java:8351)
at
org.eclipse.jdt.internal.core.util.CommentRecorderParser.end Parse(CommentRecorderParser.java:116)
at
org.eclipse.jdt.internal.compiler.SourceElementParser.endPar se(SourceElementParser.java:714)
at
org.eclipse.jdt.internal.compiler.parser.Parser.parse(Parser .java:9360)
at
org.eclipse.jdt.internal.compiler.parser.Parser.parse(Parser .java:9670)
at
org.eclipse.jdt.internal.compiler.ast.MethodDeclaration.pars eStatements(MethodDeclaration.java:119)
at
org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.parseM ethods(TypeDeclaration.java:830)
at
org.eclipse.jdt.internal.compiler.parser.Parser.getMethodBod ies(Parser.java:8542)

From the 8th entry (the Parser line) I guess it will be familiar
to you, and you will probably be quite surprised to find endParse calling
Lombok, but as I said it hooks itself in quite oddly.

I am running 3.5.1.

David
Re: There is a way to use ASTParser outside Eclipse? [message #515617 is a reply to message #515470] Fri, 19 February 2010 09:26 Go to previous messageGo to next message
Eclipse UserFriend
Le 2010-02-18 15:26, David Goodenough a écrit :
> at
> org.eclipse.jdt.internal.compiler.parser.Parser.endParse(Par ser.java:8351)
> at
> org.eclipse.jdt.internal.core.util.CommentRecorderParser.end Parse(CommentRecorderParser.java:116)
> at
> org.eclipse.jdt.internal.compiler.SourceElementParser.endPar se(SourceElementParser.java:714)
> at
> org.eclipse.jdt.internal.compiler.parser.Parser.parse(Parser .java:9360)
> at
> org.eclipse.jdt.internal.compiler.parser.Parser.parse(Parser .java:9670)
> at
> org.eclipse.jdt.internal.compiler.ast.MethodDeclaration.pars eStatements(MethodDeclaration.java:119)
> at
> org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.parseM ethods(TypeDeclaration.java:830)
> at
> org.eclipse.jdt.internal.compiler.parser.Parser.getMethodBod ies(Parser.java:8542)
That explains it. The scopes are not created yet.
They are created in the buildTypeBindings(..) method which is called later.
So now I can tell you for sure that at that time all scopes are null. I
am in fact surprised that you find some non-null scopes.
--
Olivier
Re: There is a way to use ASTParser outside Eclipse? [message #515635 is a reply to message #515617] Fri, 19 February 2010 10:16 Go to previous message
Eclipse UserFriend
Olivier,

Thank you for that. I will pass that on to the Lombok guys. They are
rewriting the whole thing at the moment, so I will make sure that they
get this right next time around.

David

Olivier Thomann wrote:

> Le 2010-02-18 15:26, David Goodenough a écrit :
>> at
>>
org.eclipse.jdt.internal.compiler.parser.Parser.endParse(Par ser.java:8351)
>> at
>>
org.eclipse.jdt.internal.core.util.CommentRecorderParser.end Parse(CommentRecorderParser.java:116)
>> at
>>
org.eclipse.jdt.internal.compiler.SourceElementParser.endPar se(SourceElementParser.java:714)
>> at
>> org.eclipse.jdt.internal.compiler.parser.Parser.parse(Parser .java:9360)
>> at
>> org.eclipse.jdt.internal.compiler.parser.Parser.parse(Parser .java:9670)
>> at
>>
org.eclipse.jdt.internal.compiler.ast.MethodDeclaration.pars eStatements(MethodDeclaration.java:119)
>> at
>>
org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.parseM ethods(TypeDeclaration.java:830)
>> at
>>
org.eclipse.jdt.internal.compiler.parser.Parser.getMethodBod ies(Parser.java:8542)
> That explains it. The scopes are not created yet.
> They are created in the buildTypeBindings(..) method which is called
> later. So now I can tell you for sure that at that time all scopes are
> null. I am in fact surprised that you find some non-null scopes.
> --
> Olivier
Previous Topic:Viewing MANIFEST files for JARs in my build path
Next Topic:File locking error while using Ant
Goto Forum:
  


Current Time: Sat Mar 15 04:50:42 EDT 2025

Powered by FUDForum. Page generated in 0.04276 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top