Home » Language IDEs » Java Development Tools (JDT) » ASTParser not generating correct CompilationUnit
| ASTParser not generating correct CompilationUnit [message #248666] |
Thu, 11 October 2007 01:49  |
Eclipse User |
|
|
|
Originally posted by: zinon.z.gmail.com
Hi all,
I have an ASTParser which is not generating the correct CompilationUnit
for the source file I am giving it. To be more precise, it is
systematically omitting some methods, nested classes, etc.
As a trivial example, if I send it this source:
public class exampleClass {
public void method1(String name) {
//
}
public void method2(String name1) {
//
}
}
The ASTParser generates a CompilationUnit whose contents are:
public class exampleClass {
public void method1(String name) {
//
}
}
Why would ASTParser do something like this?
|
|
| |
| Re: ASTParser not generating correct CompilationUnit [message #248697 is a reply to message #248681] |
Thu, 11 October 2007 13:19  |
| Eclipse User |
|
|
|
Originally posted by: zinon.z.gmail.com
Olivier Thomann wrote:
> ZZ a e'crit :
>> Why would ASTParser do something like this?
> Could you please provide the code snippet that you are using?
> It looks like a bug, but without further details it is difficult to say
> for sure.
> So I would need:
> 1) build id
> 2) code snippet
> 3) test case
> 4) errors in the .log file if any
>
> Thanks,
> --
> Olivier
I get an IFile which is called 'file', then I run the following:
StringBuilder input = new StringBuilder();
InputStream istream = file.getContents();
Scanner scanner = new Scanner(istream);
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
input.append(line);
}
try {
char[] source = input.toString().toCharArray();
ASTParser parser = ASTParser.newParser(AST.JLS3);
parser.setSource(source);
parser.setResolveBindings(true);
CompilationUnit root = (CompilationUnit) parser.createAST(null);
...
These are the details of the Eclipse I am using:
Eclipse SDK
Version: 3.3.0
Build id: I20070621-1340
The only entries in the log file are ones like the following (which
appear to be unrelated):
!ENTRY org.eclipse.osgi 2 0 2007-10-11 20:01:27.859
!MESSAGE While loading class
"org.eclipse.mylyn.tasks.ui.TaskListManager", thread
"Thread[main,6,main]" timed out waiting (5000ms) for thread
"Thread[Worker-2,5,main]" to finish starting bundle
"update@plugins/org.eclipse.mylyn.tasks.ui_2.0.0.v20070627-1400.jar
[141]". To avoid deadlock, thread "Thread[main,6,main]" is proceeding
but "org.eclipse.mylyn.tasks.ui.TaskListManager" may not be fully
initialized.
!STACK 0
org.osgi.framework.BundleException: State change in progress for bundle
"update@plugins/org.eclipse.mylyn.tasks.ui_2.0.0.v20070627-1400.jar" by
thread "Worker-2".
at
org.eclipse.osgi.framework.internal.core.AbstractBundle.begi nStateChange(AbstractBundle.java:1141)
at
org.eclipse.osgi.framework.internal.core.AbstractBundle.star t(AbstractBundle.java:258)
at
org.eclipse.osgi.framework.util.SecureAction.start(SecureAct ion.java:400)
at
org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter .postFindLocalClass(EclipseLazyStarter.java:111)
at
org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLoc alClass(ClasspathManager.java:417)
at
org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.fin dLocalClass(DefaultClassLoader.java:189)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findLo calClass(BundleLoader.java:340)
at
org.eclipse.osgi.framework.internal.core.SingleSourcePackage .loadClass(SingleSourcePackage.java:37)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findCl assInternal(BundleLoader.java:405)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findCl ass(BundleLoader.java:369)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findCl ass(BundleLoader.java:357)
at
org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loa dClass(DefaultClassLoader.java:83)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)
at
org.eclipse.mylyn.context.ui.ContextUiPlugin$5.run(ContextUi Plugin.java:259)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:3 5)
at
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchr onizer.java:123)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.jav a:3659)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java :3296)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.jav a:2389)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2353)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:22 19)
at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:466)
at
org.eclipse.core.databinding.observable.Realm.runWithDefault (Realm.java:289)
at
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Work bench.java:461)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.j ava:149)
at
org.eclipse.ui.internal.ide.application.IDEApplication.start (IDEApplication.java:106)
at
org.eclipse.equinox.internal.app.EclipseAppHandle.run(Eclips eAppHandle.java:153)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .runApplication(EclipseAppLauncher.java:106)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .start(EclipseAppLauncher.java:76)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:363)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:176)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java: 504)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:443)
at org.eclipse.equinox.launcher.Main.run(Main.java:1169)
at org.eclipse.equinox.launcher.Main.main(Main.java:1144)
Caused by:
org.eclipse.osgi.framework.internal.core.AbstractBundle$Bund leStatusException
... 40 more
Root exception:
org.eclipse.osgi.framework.internal.core.AbstractBundle$Bund leStatusException
at
org.eclipse.osgi.framework.internal.core.AbstractBundle.begi nStateChange(AbstractBundle.java:1141)
at
org.eclipse.osgi.framework.internal.core.AbstractBundle.star t(AbstractBundle.java:258)
at
org.eclipse.osgi.framework.util.SecureAction.start(SecureAct ion.java:400)
at
org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter .postFindLocalClass(EclipseLazyStarter.java:111)
at
org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLoc alClass(ClasspathManager.java:417)
at
org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.fin dLocalClass(DefaultClassLoader.java:189)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findLo calClass(BundleLoader.java:340)
at
org.eclipse.osgi.framework.internal.core.SingleSourcePackage .loadClass(SingleSourcePackage.java:37)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findCl assInternal(BundleLoader.java:405)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findCl ass(BundleLoader.java:369)
at
org.eclipse.osgi.framework.internal.core.BundleLoader.findCl ass(BundleLoader.java:357)
at
org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loa dClass(DefaultClassLoader.java:83)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)
at
org.eclipse.mylyn.context.ui.ContextUiPlugin$5.run(ContextUi Plugin.java:259)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:3 5)
at
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchr onizer.java:123)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.jav a:3659)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java :3296)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.jav a:2389)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2353)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:22 19)
at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:466)
at
org.eclipse.core.databinding.observable.Realm.runWithDefault (Realm.java:289)
at
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Work bench.java:461)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.j ava:149)
at
org.eclipse.ui.internal.ide.application.IDEApplication.start (IDEApplication.java:106)
at
org.eclipse.equinox.internal.app.EclipseAppHandle.run(Eclips eAppHandle.java:153)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .runApplication(EclipseAppLauncher.java:106)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .start(EclipseAppLauncher.java:76)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:363)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:176)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java: 504)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:443)
at org.eclipse.equinox.launcher.Main.run(Main.java:1169)
at org.eclipse.equinox.launcher.Main.main(Main.java:1144)
|
|
| |
| Re: ASTParser not generating correct CompilationUnit [message #248722 is a reply to message #248702] |
Thu, 11 October 2007 21:19 |
| Eclipse User |
|
|
|
String line = scanner.nextLine();
input.append(line);
scanner.nextLine() isn't returning the newline, so the source code you
get in the StringBuilder have the star of the exampleClass declaration,
the star of the method1 declaration, and then a line comment that
comments everything else. :-)
Since Java do parsing recovery, it returns you the CompilationUnit, I
guess with syntax errors.
Try doing this instead:
String line = scanner.nextLine();
input.append(line);
input.append("\n");
(or not using Scanner at all)
Olivier Thomann escribio':
> ZZ a e'crit :
>> I get an IFile which is called 'file', then I run the following:
>> StringBuilder input = new StringBuilder();
>> InputStream istream = file.getContents();
>> Scanner scanner = new Scanner(istream);
>> while (scanner.hasNextLine()) {
>> String line = scanner.nextLine();
>> input.append(line);
>> }
> I guess you forgot to mention that you are closing the stream.
> So at this point you have the contents of the cu inside the input object.
> In this case:
> public class exampleClass {
> public void method1(String name) {
> //
> }
> public void method2(String name1) {
> //
> }
> }
>
>> try {
>> char[] source = input.toString().toCharArray();
>> ASTParser parser = ASTParser.newParser(AST.JLS3);
>> parser.setSource(source);
>> parser.setResolveBindings(true);
> This is useless if you don't set up a java project.
>
>> CompilationUnit root = (CompilationUnit) parser.createAST(null);
> At this point, you should get two methods in the resulting compilation
> units.
> Is the source that I mentionned above the exact one?
> One thing that can cause you some trouble would be if your code is using
> 1.4 or 1.5 constructs. Since you don't set up the options that the
> ASTParser should use, you get the JavaCore default options.
>
> If you could send me a reproducable test case, I would do further
> investigations.
>
> Thanks,
> --
> Olivier
|
|
|
| Re: ASTParser not generating correct CompilationUnit [message #248727 is a reply to message #248722] |
Fri, 12 October 2007 12:51 |
| Eclipse User |
|
|
|
Originally posted by: zinon.z.gmail.com
Ary Manzana wrote:
> String line = scanner.nextLine();
> input.append(line);
>
> scanner.nextLine() isn't returning the newline, so the source code you
> get in the StringBuilder have the star of the exampleClass declaration,
> the star of the method1 declaration, and then a line comment that
> comments everything else. :-)
>
> Since Java do parsing recovery, it returns you the CompilationUnit, I
> guess with syntax errors.
There were no syntax errors, that's what was so strange about it.
>
> Try doing this instead:
>
> String line = scanner.nextLine();
> input.append(line);
> input.append("\n");
Added the newline and it parsed correctly!
Don't you just love it when you realise you've spent many hours over a
problem when the answer is as simple as that?!
Thanks for your replies guys.
ZZ
>
> (or not using Scanner at all)
>
> Olivier Thomann escribio':
>> ZZ a e'crit :
>>> I get an IFile which is called 'file', then I run the following:
>>> StringBuilder input = new StringBuilder();
>>> InputStream istream = file.getContents();
>>> Scanner scanner = new Scanner(istream);
>>> while (scanner.hasNextLine()) {
>>> String line = scanner.nextLine();
>>> input.append(line);
>>> }
>> I guess you forgot to mention that you are closing the stream.
>> So at this point you have the contents of the cu inside the input object.
>> In this case:
>> public class exampleClass {
>> public void method1(String name) {
>> //
>> }
>> public void method2(String name1) {
>> //
>> }
>> }
>>
>>> try {
>>> char[] source = input.toString().toCharArray();
>>> ASTParser parser = ASTParser.newParser(AST.JLS3);
>>> parser.setSource(source);
>>> parser.setResolveBindings(true);
>> This is useless if you don't set up a java project.
>>
>>> CompilationUnit root = (CompilationUnit) parser.createAST(null);
>> At this point, you should get two methods in the resulting compilation
>> units.
>> Is the source that I mentionned above the exact one?
>> One thing that can cause you some trouble would be if your code is
>> using 1.4 or 1.5 constructs. Since you don't set up the options that
>> the ASTParser should use, you get the JavaCore default options.
>>
>> If you could send me a reproducable test case, I would do further
>> investigations.
>>
>> Thanks,
>> --
>> Olivier
|
|
| |
| Re: ASTParser not generating correct CompilationUnit [message #248742 is a reply to message #248732] |
Sat, 13 October 2007 09:18 |
| Eclipse User |
|
|
|
Originally posted by: zinon.z.gmail.com
Olivier Thomann wrote:
> ZZ a e'crit :
>> There were no syntax errors, that's what was so strange about it.
> If I use this code:
>
> public class exampleClass {
> public void method1(String name) {// } public void
> method2(String name1) { // }}
>
> Everything on one line for the second line, then I do get a compilation
> unit that contains a recovered type declarations and the compilation
> unit problems list contains two problems:
> Syntax error, insert "}" to complete ClassBody
> Syntax error, insert "}" to complete MethodBody
>
> Also the only MethodDeclaration node in the TypeDeclaration is tagged as
> malformed.
>
> Do you also get this?
> --
> Olivier
Yes I get them too.
So, therefore, ASTParser will add/modify the source it receives in order
to always return a syntactically correct compilation unit. I can see why
this would be a good thing in that interrogating the compilation unit
would not throw unexpected exceptions, but can we be sure that all
elements of the source have been transferred to the compilation unit?
Would we need to check the problems list in code before proceeding?
And what changes, other than adding braces as we've seen here, will
ASTParser likely make to the source to make it 'correct'?
|
|
| |
| Re: ASTParser not generating correct CompilationUnit [message #248757 is a reply to message #248742] |
Sat, 13 October 2007 12:18 |
| Eclipse User |
|
|
|
You probably used CompilationUnit.toString() method. You have a "broken"
compilation unit with a class declaration, which has a method
declaration. When you issue CompilationUnit.toString(), that method
doesn't take into account that the compilation unit is broken or not: it
just opens a class declaration and closes it with curlies. Same goes for
methods.
In short, you shouldn't rely on the toString() method (it's just for
debugging purposes). Instead, as Olivier pointed, you should check the
ASTNode flags, and also the problems of the CompilationUnit.
ZZ escribió:
> Olivier Thomann wrote:
>> ZZ a e'crit :
>>> There were no syntax errors, that's what was so strange about it.
>> If I use this code:
>>
>> public class exampleClass {
>> public void method1(String name) {// } public void
>> method2(String name1) { // }}
>>
>> Everything on one line for the second line, then I do get a
>> compilation unit that contains a recovered type declarations and the
>> compilation unit problems list contains two problems:
>> Syntax error, insert "}" to complete ClassBody
>> Syntax error, insert "}" to complete MethodBody
>>
>> Also the only MethodDeclaration node in the TypeDeclaration is tagged
>> as malformed.
>>
>> Do you also get this?
>> --
>> Olivier
>
> Yes I get them too.
>
> So, therefore, ASTParser will add/modify the source it receives in order
> to always return a syntactically correct compilation unit. I can see why
> this would be a good thing in that interrogating the compilation unit
> would not throw unexpected exceptions, but can we be sure that all
> elements of the source have been transferred to the compilation unit?
> Would we need to check the problems list in code before proceeding?
>
> And what changes, other than adding braces as we've seen here, will
> ASTParser likely make to the source to make it 'correct'?
|
|
|
Goto Forum:
Current Time: Sat Sep 06 12:26:26 EDT 2025
Powered by FUDForum. Page generated in 0.05368 seconds |
Home