Home » Language IDEs » Java Development Tools (JDT) » [apt] Preserving result between processor runs
| [apt] Preserving result between processor runs [message #258229] |
Fri, 23 January 2009 09:34  |
Eclipse User |
|
|
|
Hi,
Since the annotation processors are invoked with incremental results in
Eclipse, I'd like to preserve some data between runs.
My use case is the following:
I'm trying to build a testng suite file out of @Test-annotated methods and
classes. When a new annotated method or class is added, I only receive
that element.
I am writing the suite file when the run ends with
processingEnv.getFiler().createResource(StandardLocation.SOU RCE_OUTPUT,
"", "testng-all.xml");
and trying to read it with
processingEnv.getFiler().getResource(StandardLocation.SOURCE _OUTPUT, "",
"testng-all.xml");
Since the file is not found, I assume that the file is cleared eagerly
between processor runs? If so, how can I store previous state between runs?
Thank you,
Robert
|
|
|
| Re: [apt] Preserving result between processor runs [message #258233 is a reply to message #258229] |
Sat, 24 January 2009 20:11  |
| Eclipse User |
|
|
|
"Robert Munteanu" <robert.munteanu@gmail.com> wrote in message
news:45dc8f885008f3ef7c30a296c90b1c57$1@www.eclipse.org...
> Since the annotation processors are invoked with incremental results in
> Eclipse, I'd like to preserve some data between runs.
A very reasonable use case and one which naturally stems from running
annotation processors in an IDE, as you observe. A similar case is trying
to build an EJB config file from annotations on individual beans.
Unfortunately in the JSR269 spec process concerns related to incremental
compilation did not get a lot of traction, and we are left with an API that
fails to support some important use cases.
From the Javadoc for Filer: "During each run of an annotation processing
tool, a file with a given pathname may be created only once. If that file
already exists before the first attempt to create it, the old contents will
be deleted."
So, how can you maintain a database of annotations, which will be
incrementally updated as individual files are processed? I don't have a
good answer. One option is to skip the Filer interface and use java.io
methods instead; you can't get away with that for source files, because they
wouldn't get recompiled, but for something like this the compiler will be
none the wiser. Another is to create a bunch of individual files, and use a
separate process to assemble them all, driving that process from an Eclipse
builder and/or from ant in a command-line build. A third possibility is to
manually do a clean build, to force the annotation processor to see all the
files in one run.
That said, I do seem to recall that we came up with some clever way of
making a composite file like you're talking about - I don't remember the
details now, or even whether it was still supported in the final version of
the JSR269 API, but I'll try to dig it up.
-walter harley
JDT APT lead
|
|
|
| Re: [apt] Preserving result between processor runs [message #258249 is a reply to message #258229] |
Sun, 25 January 2009 22:44 |
| Eclipse User |
|
|
|
"Robert Munteanu" <robert.munteanu@gmail.com> wrote in message
news:45dc8f885008f3ef7c30a296c90b1c57$1@www.eclipse.org...
> I am writing the suite file when the run ends with
>
> processingEnv.getFiler().createResource(StandardLocation.SOU RCE_OUTPUT,
> "", "testng-all.xml");
>
> and trying to read it with
>
> processingEnv.getFiler().getResource(StandardLocation.SOURCE _OUTPUT, "",
> "testng-all.xml");
>
> Since the file is not found, I assume that the file is cleared eagerly
> between processor runs? If so, how can I store previous state between
> runs?
Actually, in my other post I wasn't paying close enough attention to what
you said. The caveats I stated were true, but I don't think they apply
here. Your problem may have something to do with the order in which you
read and write, I'm not sure.
I just did some testing, and I am able to read and write resource files as
you suggest you want to. The relevant part of my hacked-together processor
code looks something like this:
CharSequence existing = "";
FileObject fin = _filer.getResource(StandardLocation.SOURCE_OUTPUT, pkg,
"composite.txt");
try {
existing = fin.getCharContent(true);
} catch (IOException ioe) {
// file doesn't exist yet
}
FileObject fo = _filer.createResource(StandardLocation.SOURCE_OUTPUT,
pkg, "composite.txt");
w = fo.openWriter();
pw = new PrintWriter(w);
pw.println(existing);
pw.println("Processed " + clazz + " and " + method);
pw.close();
w.close();
For me, this "accumulates" files as they are compiled, just as I'd expect.
If you are getting something different, could you please enter a Bugzilla
against JDT/APT, including your processor code so I can reproduce it?
By the way, notice that the createResource() method has an additional
parameter. I've ignored it above, to echo your code, but it's actually
important in the IDE. It is "originatingElements", and it lets the IDE
build a map of dependencies, so that it know what files to regenerate and/or
delete when source files change. Without this, we have no way of, for
instance, deleting a generated file if you remove the annotation that caused
it to be generated.
|
|
|
| Re: [apt] Preserving result between processor runs [message #258253 is a reply to message #258249] |
Mon, 26 January 2009 03:28 |
| Eclipse User |
|
|
|
Thank you for your reply.
Before trying to isolate the processor, there's only one thing I can think
which may cause problems:
My main process method tries to write the file after the processing is
over:
@Override
public boolean process(Set<? extends TypeElement> annotations,
RoundEnvironment env) {
for (TypeElement type : annotations)
processClasses(env, type);
if (env.processingOver())
writeTestNGSuiteFile();
return true;
}
and inside the writeTestNGSuiteFile() the error occurs.
Does that look right to you? If it does, I'll create a sample project and
file a bug.
And as for the additional parameter, I've noticed that and I am using it
when creating ( for instance ) resource related to specific classes.On the
other hand I'm not doing that here since I'm worried exactly about the
incremental case:
- first run, resource is related to all @Test-annotated classes
- second run, a new test case, resource is re-created but related only to
the new test case, which is not what I want.
Thank you,
Robert
|
|
|
| Re: [apt] Preserving result between processor runs [message #258271 is a reply to message #258253] |
Mon, 26 January 2009 15:49 |
| Eclipse User |
|
|
|
"Robert Munteanu" <robert.munteanu@gmail.com> wrote in message
news:6950040945da3d19a45bdb799b18a796$1@www.eclipse.org...
> My main process method tries to write the file after the processing is
> over: [...]
> Does that look right to you? If it does, I'll create a sample project and
> file a bug.
Offhand, it looks like exactly the right thing; but I didn't try it in my
own test project, so let me give that a shot. If you can create a sample
project and file a bug, that would be great, it'll help me keep track.
> And as for the additional parameter, I've noticed that and I am using it
> when creating ( for instance ) resource related to specific classes.On the
> other hand I'm not doing that here since I'm worried exactly about the
> incremental case:
>
> - first run, resource is related to all @Test-annotated classes
> - second run, a new test case, resource is re-created but related only to
> the new test case, which is not what I want.
Well, the hope would be that we can make it work so that your code would
first read the file to get the existing results, then generate a new file
based on the updated information as well as any non-updated info from other
sources.
The dependency is many-to-many, so for instance if you have A, B, and C all
of which contribute to ABCGen, the file should be regenerated if any of A B
C are changed, but will still be removed if A B C are all touched in a way
that causes none of them to generate the file. It sounds like in your case
you really have no dependencies at all (the file should exist even if there
are no annotated source files)? In that case you're doing the right thing
by not specifying an originatingElement.
|
|
|
| Re: [apt] Preserving result between processor runs [message #258351 is a reply to message #258271] |
Wed, 28 January 2009 04:10 |
| Eclipse User |
|
|
|
On Mon, 26 Jan 2009 12:49:34 -0800 "Walter Harley"
<eclipse@cafewalter.com> wrote:
> "Robert Munteanu" <robert.munteanu@gmail.com> wrote in message
> news:6950040945da3d19a45bdb799b18a796$1@www.eclipse.org...
>> My main process method tries to write the file after the processing is
>> over: [...]
>> Does that look right to you? If it does, I'll create a sample project
>> and file a bug.
>
> Offhand, it looks like exactly the right thing; but I didn't try it in
> my own test project, so let me give that a shot. If you can create a
> sample project and file a bug, that would be great, it'll help me keep
> track.
>
The sample project helped me keep track of what was going on:
1. On an incremental build, the file is always found.
2. On a clean build, the file is found but then ( on a subsequent run ? )
it is not found.
The notes from the processor are similar to:
Info | 10:54:18.483 |Failed to open file . Assuming it does not exist :
org.eclipse.core.internal.resources.ResourceException: Resource
'/.../.apt_generated/testng-all.xml' does not exist.
Info | 10:54:18.482 | Opened file.
I'm not sure on why this happens but then again it does not interfere
with my generation process since the class names are accumulated in an
instance field.
3. My XML reading was wrongly done, and was not reading the old class
names properly .
This is definitely not a bug, but the exception threw me off the first
time.
Thank you for your help,
Robert
|
|
|
| Re: [apt] Preserving result between processor runs [message #258358 is a reply to message #258351] |
Wed, 28 January 2009 11:35 |
| Eclipse User |
|
|
|
"Robert Munteanu" <robert.munteanu@gmail.com> wrote in message
news:pan.2009.01.28.09.11.15@gmail.com...
> The sample project helped me keep track of what was going on:
>
> 1. On an incremental build, the file is always found.
> 2. On a clean build, the file is found but then ( on a subsequent run ? )
> it is not found.
>
> The notes from the processor are similar to:
>
> Info | 10:54:18.483 |Failed to open file . Assuming it does not exist :
> org.eclipse.core.internal.resources.ResourceException: Resource
> '/.../.apt_generated/testng-all.xml' does not exist.
> Info | 10:54:18.482 | Opened file.
>
> I'm not sure on why this happens but then again it does not interfere
> with my generation process since the class names are accumulated in an
> instance field.
>
> 3. My XML reading was wrongly done, and was not reading the old class
> names properly .
>
> This is definitely not a bug, but the exception threw me off the first
> time.
Good, sounds like you're getting some traction again.
It sounds to me like a bug report would be a good idea, if only to capture
all this information in a place we can look it up next time someone wonders.
Heaven knows the behavior is complex enough to need explaining; I'm not sure
I understand it myself. If you've got time, please file a bug report
including the sample processor.
Are those log entries being generated by your processor code, or are those
from the APT code?
|
|
|
Goto Forum:
Current Time: Mon Apr 21 02:11:23 EDT 2025
Powered by FUDForum. Page generated in 0.09842 seconds |
Home