Home » Eclipse Projects » Plugin Development Environment (PDE) » Plugins Out of Sync
| Plugins Out of Sync [message #5843] |
Fri, 06 June 2008 10:51  |
Eclipse User |
|
|
|
Originally posted by: bruce.skingle.org
I am seeing a problem with 3.2.2 where some of my plugin projects show
as "(Out of Sync)" in the Plug-ins view and in the depdendencies panel
of other plugins which depend on the out of sync plugins.
This is nothing to do with the file system being out of sync with the
work space, and no amount of refresh, clean, or rebuild seems to make
the problem go away. Sometimes closing and reopening the project helps,
and I think always, exiting Eclipse and restarting sorts it out.
This has been reported before (see
http://www.eclipsepowered.org/eclipse/forums/t102020.html) but it seems
that no one has been able to isolate a reproducable test case.
I have also been unable to reliably reproduce the error, but I have a
hunch and wondered if anyone on the PDE team could consider this as a
possible cause:
The projects which this happens on are the result of a code generator I
am developing, the whole project is generated including MANIFEST.MF and
plugin.xml. All file writes are done via the Eclipse API and I do not
see any issues with files out of sync on the file system. An alternative
mode where the generated code is dropped into a folder in a normal (i.e
not derived object) project works without any problems.
So, could it be that the problem is related to updating MANIFEST.MF? The
PDE obviously maintains some other state about plugins and plugin
dependencies, because at one point I noticed that calling
IProject.getReferencingProjects() did not return everything it should
have, and anyway, without some kind of state cache that method would
have to re-parse every MANIFEST.MF in the workspace which would probably
take too long, but this is pure guesswork on my part.
Obviously if I can isolate a test case I'll raise a bug, but I just
wondered if this possible scenario struck a chord with anyone?
Bruce.
|
|
|
| Re: Plugins Out of Sync [message #5860 is a reply to message #5843] |
Mon, 09 June 2008 11:07  |
| Eclipse User |
|
|
|
Originally posted by: caniszczyk.gmail.com
Bruce Skingle wrote:
> So, could it be that the problem is related to updating MANIFEST.MF? The
> PDE obviously maintains some other state about plugins and plugin
> dependencies, because at one point I noticed that calling
> IProject.getReferencingProjects() did not return everything it should
> have, and anyway, without some kind of state cache that method would
> have to re-parse every MANIFEST.MF in the workspace which would probably
> take too long, but this is pure guesswork on my part.
>
> Obviously if I can isolate a test case I'll raise a bug, but I just
> wondered if this possible scenario struck a chord with anyone?
So, PDE has become a bit more resilient with this problem in 3.4, but
this can still happen. PDE keeps a state on disk and this state is
encoded via a hash. The hash is generated by xoring a ton of things
included the timestamp of the MANIFEST.MF. What is happening here is
that you're probably modifying the MANIFEST.MF and PDE's cache has a
different value because your modification changes the timestamp.
Can you be more specific about how you do things?
Are these plug-ins that you modify exist outside the workspace?
We're really dying for a reproducible test case for this problem so
anything that can help towards that would be great. We have only been
able to reproduce this problem on plug-ins in the target platform, not
ones in the workspace.
Cheers,
~ Chris
|
|
|
| Re: Plugins Out of Sync [message #8252 is a reply to message #5860] |
Mon, 23 June 2008 11:37 |
| Eclipse User |
|
|
|
Originally posted by: bruce.skingle.org
Chris Aniszczyk wrote:
> Bruce Skingle wrote:
>> So, could it be that the problem is related to updating MANIFEST.MF?
>> The PDE obviously maintains some other state about plugins and plugin
>> dependencies, because at one point I noticed that calling
>> IProject.getReferencingProjects() did not return everything it should
>> have, and anyway, without some kind of state cache that method would
>> have to re-parse every MANIFEST.MF in the workspace which would
>> probably take too long, but this is pure guesswork on my part.
>>
>> Obviously if I can isolate a test case I'll raise a bug, but I just
>> wondered if this possible scenario struck a chord with anyone?
>
> So, PDE has become a bit more resilient with this problem in 3.4, but
> this can still happen. PDE keeps a state on disk and this state is
> encoded via a hash. The hash is generated by xoring a ton of things
> included the timestamp of the MANIFEST.MF. What is happening here is
> that you're probably modifying the MANIFEST.MF and PDE's cache has a
> different value because your modification changes the timestamp.
>
> Can you be more specific about how you do things?
>
> Are these plug-ins that you modify exist outside the workspace?
>
> We're really dying for a reproducible test case for this problem so
> anything that can help towards that would be great. We have only been
> able to reproduce this problem on plug-ins in the target platform, not
> ones in the workspace.
>
> Cheers,
>
> ~ Chris
This happens to me quite often but I can't write a reproducable test
case despite spending quite a few hours trying, it's obviously an
elusive problem.
This is quite complex but you asked for specifics, so here is what I'm
doing:
I have written a code generation plugin called coda, which reads a model
stored in a file in the workspace and generates Java code from it, into
the workspace. I'm using this as a tool to develop my actual product.
Coda consists of a number of development time plugins which generate
Java code, plus a number of runtime plugins, which contain super classes
for the generated code and other run time stuff. These runtime plugins
need to be deployed along with the final product, the development time
plugins are only needed during development.
Generally I run 3 Eclipse instances:
1. Coda development workspace, with Eclipse 3.3.2 plus subversion. The
workspace includes all the coda sources. From this I launch a debug mode
eclipse:
2. Coda Test workspace, with Eclipse 3.3.2 plus subversion plus coda,
and coda rutime. This workspace contains my product source code, and
here is where I edit the coda application model and the generation takes
place. (This is a bit of a tangent, but for reasons I don't understand,
and it would be good to know, I have to import all the run time plugins
into this development workspace. The generated code will not link to the
runtime plugins which are installed and visible, unless they are in the
workspace. If I deploy the coda plugins (see below) then this is not
necessary).
This workspace is made up of three sets of projects, Coda runtime
plugins imported from workspace 1, application plugins imported from my
development workspace (see below), and generated plugins actually stored
in this workspace.
From this I launch another debug mode eclipse:
3. Application test workbench, with Eclipse 3.3.2, coda runtime and
application plugins. This is the test environment for my actual product.
Aside from the wrinkle mentioned above all this works just fine (most of
the time), and I can change the coda generation code in workspace 1,
regenerate in instance 2 and test in instance 3 without restarting
anything, all the hot code replacement works just fine, and life is *good*.
Sometimes it's convenient to edit the coda runtime plugins in instance
2, which also works fine, these plugin projects are just imported from
the instance 1 workspace, so after going back to instance 1 I just have
to do a refresh on the changed plugins and all the changes show up in
both instances.
Sometimes I run an alternative configuration, where I actually deploy
the Coda development and runtime plugins into a separate Eclipse
installation and just run the development workspace directly. This is
the same workspace from which the application projects are imported into
the Coda Test workspace above, so a quick refresh and any work done in
the other configuration appears. From this I launch the application test
workbench as before. As I said above, in this configuration the Coda
runtime plugins do not need to be in the development workspace, the
generated code links against the installed plugins as I expect.
Mostly I stick with the 3 instance configuration because the convenience
of being able to edit everything on the fly outweighs any disadvantages.
The "plugins Out of Sync" problem happens in both scenarios and is not
an artifact of my 3 instance configuration, although it's possible (but
unproven) that the 3 instance config makes it happen more often, I can't
really tell.
So within instance 2, where I see the plugins out of sync problem,
what's happening?
If we assume that my product is called "foo", then I have the following
projects:
foo Main application plugin, all hand written code
this is a plugin project
foo.coda Coda model project, contains only coda model
this is a general project with the Coda nature
foo.coda.core Generated files plus hand written code
foo.coda.admin Generated code
foo.coda.ui.admin Generated code
these 3 are plugin projects
foo.coda.core
foo.coda.admin
foo.coda.ui.admin all depend on foo.coda
foo.coda.ui.admin depends on foo.coda.admin
foo.coda.admin depends on foo.coda.core
foo depends on foo.coda.core
[Actually there are 3 sets of these in the workspace and foo depends on
other plugins in the workspace, but all of that seems to work fine and I
do not believe the Out of Sync problem is affected by any of that]
So, foo.coda.core is the main body of generated code, needed by the app
at runtime. foo.coda.admin is admin related stuff like the DDL code to
create the database for the application, in production this would not be
deployed, and foo.coda.ui.admin is some GUI code to access the admin
functions which also would not be deployed in production.
The admin plugins are entirely generated and do not exist in subversion.
The core plugin also contains hand written code so is partially checked
in. All of the hand written code sub-classes generated code, the hand
written code and the generated code go into separate source folders to
prevent the generator ever over writing hand written code. I would
prefer to have the hand written code in a separate project altogether,
but there is a mutual dependency between the generated code and the hand
written sub-classes so that would cause a circular dependency problem.
The MANIFEST.MF and plugin.xml are always generated and overwritten, it
is possible to put certain things into the coda model for example to add
a plugin dependency to the generated MANIFEST.MF.
All of the generated files have the derived bit set, and are written via
a ByteArrayOutputStream with the following method:
private IFile file_;
private boolean derived_;
public void close(IProgressMonitor monitor) throws IOException,
CoreException
{
super.close();
if(file_ == null)
return;
if(file_.exists())
{
if(derived_)
file_.setContents(new ByteArrayInputStream(toByteArray()),
IFile.FORCE|IFile.DERIVED , monitor);
}
else
{
file_.create(new ByteArrayInputStream(toByteArray()),
(derived_ ? IFile.FORCE|IFile.DERIVED : IFile.FORCE),
monitor);
}
}
The member derived_ is always true, that was something I was
experimeting with a while ago.
Coda has an IncrementalProjectBuilder so as soon as the coda model file
is saved the generation takes place, the resource management system sees
the generated java files change and kicks off the java compiler and
everything gets built pretty much flawlessly in the steady state, the
problems arise, I think, from the initial state.
The Coda builder implements the clean method and knows where all the
generated code goes so more or less cleans up everything if you do a
clean with autobuild turned off. The one exception is that the .project
and .classpath files and the project folder itself do not carry the
derived bit (I don't create the files myself (I call IProject.create())
so I don't get the chance to assert the derived bit. I can see why it
never occurred to anyone that a project might be derived, but it would
be nice if this could be allowed.) and I cannot bring myself to
automatically delete files not marked as derived, so after a clean you
are left with the empty generated projects.
In the initialization scenario you start with a fresh checkout from
subversion which gives you foo and foo.coda, plus part of foo.coda.core,
in particular you get
foo.coda.core
+ .classpath
+ .project
+ src
+ hand written Java source files.....
and that is all.
When the builder for foo.coda kicks off the following files get created:
foo.coda.core
+ META-INF
+ MANIFEST.MF
+ build.properties
+ plugin.xml
+ generatedSrc
+ generated Java source files
plus the whole of the folders for foo.coda.admin and foo.coda.ui.admin.
The generated plugins (foo.coda.core, foo.coda.admin and
foo.coda.ui.admin) are the ones which show up with the out of sync
problem, and when it happens it always happens to all three, it never
happens to the regular projects in the workspace.
Sometimes the build process gets in a mess and the generated projects
need to be cleaned, to get rid of an error saying that exported packages
do not exist. I have played around with the order of creation of
MANIFEST.MF and java sources to try to eliminate this and it works
correctly almost all of the time. I experimented with creating a dummy
class with nothing in first, then the manifest, then the real code,
which seems to help with this.
The one thing which does not work properly is clean all projects, which
of course is the one thing which ought to be a safe and reliable fix
everything action. This is entirely (I think) because of the out of sync
problem. I previously had a lot of dependency order problems, which is
why I have ended up with so many plugin projects.
Hopefully something in this mass of data will be of help......
It would be good if there was some way to tell PDE that the set of
plugins in the workspace have changed and that plugin dependencies
should be re-checked.
Regards,
Bruce.
|
|
|
| Re: Plugins Out of Sync [message #568389 is a reply to message #5843] |
Mon, 09 June 2008 11:07 |
| Eclipse User |
|
|
|
Bruce Skingle wrote:
> So, could it be that the problem is related to updating MANIFEST.MF? The
> PDE obviously maintains some other state about plugins and plugin
> dependencies, because at one point I noticed that calling
> IProject.getReferencingProjects() did not return everything it should
> have, and anyway, without some kind of state cache that method would
> have to re-parse every MANIFEST.MF in the workspace which would probably
> take too long, but this is pure guesswork on my part.
>
> Obviously if I can isolate a test case I'll raise a bug, but I just
> wondered if this possible scenario struck a chord with anyone?
So, PDE has become a bit more resilient with this problem in 3.4, but
this can still happen. PDE keeps a state on disk and this state is
encoded via a hash. The hash is generated by xoring a ton of things
included the timestamp of the MANIFEST.MF. What is happening here is
that you're probably modifying the MANIFEST.MF and PDE's cache has a
different value because your modification changes the timestamp.
Can you be more specific about how you do things?
Are these plug-ins that you modify exist outside the workspace?
We're really dying for a reproducible test case for this problem so
anything that can help towards that would be great. We have only been
able to reproduce this problem on plug-ins in the target platform, not
ones in the workspace.
Cheers,
~ Chris
|
|
|
| Re: Plugins Out of Sync [message #569859 is a reply to message #5860] |
Mon, 23 June 2008 11:37 |
| Eclipse User |
|
|
|
Chris Aniszczyk wrote:
> Bruce Skingle wrote:
>> So, could it be that the problem is related to updating MANIFEST.MF?
>> The PDE obviously maintains some other state about plugins and plugin
>> dependencies, because at one point I noticed that calling
>> IProject.getReferencingProjects() did not return everything it should
>> have, and anyway, without some kind of state cache that method would
>> have to re-parse every MANIFEST.MF in the workspace which would
>> probably take too long, but this is pure guesswork on my part.
>>
>> Obviously if I can isolate a test case I'll raise a bug, but I just
>> wondered if this possible scenario struck a chord with anyone?
>
> So, PDE has become a bit more resilient with this problem in 3.4, but
> this can still happen. PDE keeps a state on disk and this state is
> encoded via a hash. The hash is generated by xoring a ton of things
> included the timestamp of the MANIFEST.MF. What is happening here is
> that you're probably modifying the MANIFEST.MF and PDE's cache has a
> different value because your modification changes the timestamp.
>
> Can you be more specific about how you do things?
>
> Are these plug-ins that you modify exist outside the workspace?
>
> We're really dying for a reproducible test case for this problem so
> anything that can help towards that would be great. We have only been
> able to reproduce this problem on plug-ins in the target platform, not
> ones in the workspace.
>
> Cheers,
>
> ~ Chris
This happens to me quite often but I can't write a reproducable test
case despite spending quite a few hours trying, it's obviously an
elusive problem.
This is quite complex but you asked for specifics, so here is what I'm
doing:
I have written a code generation plugin called coda, which reads a model
stored in a file in the workspace and generates Java code from it, into
the workspace. I'm using this as a tool to develop my actual product.
Coda consists of a number of development time plugins which generate
Java code, plus a number of runtime plugins, which contain super classes
for the generated code and other run time stuff. These runtime plugins
need to be deployed along with the final product, the development time
plugins are only needed during development.
Generally I run 3 Eclipse instances:
1. Coda development workspace, with Eclipse 3.3.2 plus subversion. The
workspace includes all the coda sources. From this I launch a debug mode
eclipse:
2. Coda Test workspace, with Eclipse 3.3.2 plus subversion plus coda,
and coda rutime. This workspace contains my product source code, and
here is where I edit the coda application model and the generation takes
place. (This is a bit of a tangent, but for reasons I don't understand,
and it would be good to know, I have to import all the run time plugins
into this development workspace. The generated code will not link to the
runtime plugins which are installed and visible, unless they are in the
workspace. If I deploy the coda plugins (see below) then this is not
necessary).
This workspace is made up of three sets of projects, Coda runtime
plugins imported from workspace 1, application plugins imported from my
development workspace (see below), and generated plugins actually stored
in this workspace.
From this I launch another debug mode eclipse:
3. Application test workbench, with Eclipse 3.3.2, coda runtime and
application plugins. This is the test environment for my actual product.
Aside from the wrinkle mentioned above all this works just fine (most of
the time), and I can change the coda generation code in workspace 1,
regenerate in instance 2 and test in instance 3 without restarting
anything, all the hot code replacement works just fine, and life is *good*.
Sometimes it's convenient to edit the coda runtime plugins in instance
2, which also works fine, these plugin projects are just imported from
the instance 1 workspace, so after going back to instance 1 I just have
to do a refresh on the changed plugins and all the changes show up in
both instances.
Sometimes I run an alternative configuration, where I actually deploy
the Coda development and runtime plugins into a separate Eclipse
installation and just run the development workspace directly. This is
the same workspace from which the application projects are imported into
the Coda Test workspace above, so a quick refresh and any work done in
the other configuration appears. From this I launch the application test
workbench as before. As I said above, in this configuration the Coda
runtime plugins do not need to be in the development workspace, the
generated code links against the installed plugins as I expect.
Mostly I stick with the 3 instance configuration because the convenience
of being able to edit everything on the fly outweighs any disadvantages.
The "plugins Out of Sync" problem happens in both scenarios and is not
an artifact of my 3 instance configuration, although it's possible (but
unproven) that the 3 instance config makes it happen more often, I can't
really tell.
So within instance 2, where I see the plugins out of sync problem,
what's happening?
If we assume that my product is called "foo", then I have the following
projects:
foo Main application plugin, all hand written code
this is a plugin project
foo.coda Coda model project, contains only coda model
this is a general project with the Coda nature
foo.coda.core Generated files plus hand written code
foo.coda.admin Generated code
foo.coda.ui.admin Generated code
these 3 are plugin projects
foo.coda.core
foo.coda.admin
foo.coda.ui.admin all depend on foo.coda
foo.coda.ui.admin depends on foo.coda.admin
foo.coda.admin depends on foo.coda.core
foo depends on foo.coda.core
[Actually there are 3 sets of these in the workspace and foo depends on
other plugins in the workspace, but all of that seems to work fine and I
do not believe the Out of Sync problem is affected by any of that]
So, foo.coda.core is the main body of generated code, needed by the app
at runtime. foo.coda.admin is admin related stuff like the DDL code to
create the database for the application, in production this would not be
deployed, and foo.coda.ui.admin is some GUI code to access the admin
functions which also would not be deployed in production.
The admin plugins are entirely generated and do not exist in subversion.
The core plugin also contains hand written code so is partially checked
in. All of the hand written code sub-classes generated code, the hand
written code and the generated code go into separate source folders to
prevent the generator ever over writing hand written code. I would
prefer to have the hand written code in a separate project altogether,
but there is a mutual dependency between the generated code and the hand
written sub-classes so that would cause a circular dependency problem.
The MANIFEST.MF and plugin.xml are always generated and overwritten, it
is possible to put certain things into the coda model for example to add
a plugin dependency to the generated MANIFEST.MF.
All of the generated files have the derived bit set, and are written via
a ByteArrayOutputStream with the following method:
private IFile file_;
private boolean derived_;
public void close(IProgressMonitor monitor) throws IOException,
CoreException
{
super.close();
if(file_ == null)
return;
if(file_.exists())
{
if(derived_)
file_.setContents(new ByteArrayInputStream(toByteArray()),
IFile.FORCE|IFile.DERIVED , monitor);
}
else
{
file_.create(new ByteArrayInputStream(toByteArray()),
(derived_ ? IFile.FORCE|IFile.DERIVED : IFile.FORCE),
monitor);
}
}
The member derived_ is always true, that was something I was
experimeting with a while ago.
Coda has an IncrementalProjectBuilder so as soon as the coda model file
is saved the generation takes place, the resource management system sees
the generated java files change and kicks off the java compiler and
everything gets built pretty much flawlessly in the steady state, the
problems arise, I think, from the initial state.
The Coda builder implements the clean method and knows where all the
generated code goes so more or less cleans up everything if you do a
clean with autobuild turned off. The one exception is that the .project
and .classpath files and the project folder itself do not carry the
derived bit (I don't create the files myself (I call IProject.create())
so I don't get the chance to assert the derived bit. I can see why it
never occurred to anyone that a project might be derived, but it would
be nice if this could be allowed.) and I cannot bring myself to
automatically delete files not marked as derived, so after a clean you
are left with the empty generated projects.
In the initialization scenario you start with a fresh checkout from
subversion which gives you foo and foo.coda, plus part of foo.coda.core,
in particular you get
foo.coda.core
+ .classpath
+ .project
+ src
+ hand written Java source files.....
and that is all.
When the builder for foo.coda kicks off the following files get created:
foo.coda.core
+ META-INF
+ MANIFEST.MF
+ build.properties
+ plugin.xml
+ generatedSrc
+ generated Java source files
plus the whole of the folders for foo.coda.admin and foo.coda.ui.admin.
The generated plugins (foo.coda.core, foo.coda.admin and
foo.coda.ui.admin) are the ones which show up with the out of sync
problem, and when it happens it always happens to all three, it never
happens to the regular projects in the workspace.
Sometimes the build process gets in a mess and the generated projects
need to be cleaned, to get rid of an error saying that exported packages
do not exist. I have played around with the order of creation of
MANIFEST.MF and java sources to try to eliminate this and it works
correctly almost all of the time. I experimented with creating a dummy
class with nothing in first, then the manifest, then the real code,
which seems to help with this.
The one thing which does not work properly is clean all projects, which
of course is the one thing which ought to be a safe and reliable fix
everything action. This is entirely (I think) because of the out of sync
problem. I previously had a lot of dependency order problems, which is
why I have ended up with so many plugin projects.
Hopefully something in this mass of data will be of help......
It would be good if there was some way to tell PDE that the set of
plugins in the workspace have changed and that plugin dependencies
should be re-checked.
Regards,
Bruce.
|
|
|
Goto Forum:
Current Time: Tue May 13 18:47:00 EDT 2025
Powered by FUDForum. Page generated in 0.03495 seconds |
Home