Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Web Tools Project (WTP) » Support for Tomcat AntiJar Locking...Needed for Publishing...
Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180083] Thu, 28 September 2006 02:41 Go to next message
Eclipse UserFriend
Originally posted by: peterdnight.yahoo.com

Did not see this in here anywhere. But from the Tomcat Windows FAQ
http://tomcat.apache.org/faq/windows.html#lock it is pretty clear that on
Windows, Tomcat MUST have a file in:
metadata\.plugins\org.eclipse.wst.server.core\tmp3\conf\cont ext.xml:
<Context
antiResourceLocking="true"
antiJARLocking="true"
>
WTP has to be extended to include the context.xml file inside of the server
config. I am hoping this gets added. I am getting "unable to publish" every
single time with 1 WebApp Project, with 1 Java Project referenced via the
J2EE Module dependencies. Both are HelloWorldish: a 3 line servlet invoking
a getString method on a 5 line class in the Java Project.

Works fine - until the Java Project is changed - then I get the "Unable to
publish on the jar as it is locked." WebApp changes are fine though - hot
deploy works beautifully!!

Unfortunately these flags cause Tomcat to deploy the TMP folder - so WTP
needs to be tweaked to get this to work....adding the context.xml file
manually will not work :(

Other then this - which will cause me alot of grief - great work.

....Peter
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180119 is a reply to message #180083] Thu, 28 September 2006 12:19 Go to previous messageGo to next message
Larry Isaacs is currently offline Larry IsaacsFriend
Messages: 1313
Registered: July 2009
Senior Member
Are you using a current WTP 1.5.1? The fix for Bug 125524[1] in WTP
1.5.1 should work around the jar locking problem by rewriting the jar
rather than replacing it.

Cheers,
Larry

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=125524

Peter wrote:
> Did not see this in here anywhere. But from the Tomcat Windows FAQ
> http://tomcat.apache.org/faq/windows.html#lock it is pretty clear that on
> Windows, Tomcat MUST have a file in:
> metadata\.plugins\org.eclipse.wst.server.core\tmp3\conf\cont ext.xml:
> <Context
> antiResourceLocking="true"
> antiJARLocking="true"
> WTP has to be extended to include the context.xml file inside of the server
> config. I am hoping this gets added. I am getting "unable to publish" every
> single time with 1 WebApp Project, with 1 Java Project referenced via the
> J2EE Module dependencies. Both are HelloWorldish: a 3 line servlet invoking
> a getString method on a 5 line class in the Java Project.
>
> Works fine - until the Java Project is changed - then I get the "Unable to
> publish on the jar as it is locked." WebApp changes are fine though - hot
> deploy works beautifully!!
>
> Unfortunately these flags cause Tomcat to deploy the TMP folder - so WTP
> needs to be tweaked to get this to work....adding the context.xml file
> manually will not work :(
>
> Other then this - which will cause me alot of grief - great work.
>
> ...Peter
>
>
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180151 is a reply to message #180119] Thu, 28 September 2006 15:09 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: peterdnight.yahoo.com

Hi Larry,

Thanks for the update. I noticed that 1.5.1 is not released yet - any ideas
when?

I tried to dump the 1.5.1 zip file into my eclipse folder, but got a bunch
of SWT errors. Then I noticed I would need to update eclipse itself, along
with gef, etc.

no easy way to do this I am guessing :(

Thanks,
Peter


eg.
a.. The Eclipse driver used in this build is
eclipse-SDK-M20060921-0945-linux-gtk.tar.gz. You can find a suitable driver
for your platform at here

a.. The EMF driver used in this build is emf-sdo-xsd-SDK-2.2.1RC2.zip

a.. The GEF driver used in this build is GEF-SDK-M20060920-1505.zip

a.. Java EMF Model Runtime driver used in this build is
JEM-SDK-M20060918.zip

"Larry Isaacs" <Larry.Isaacs@sas.com> wrote in message
news:efgejl$3un$1@utils.eclipse.org...
> Are you using a current WTP 1.5.1? The fix for Bug 125524[1] in WTP 1.5.1
> should work around the jar locking problem by rewriting the jar rather
> than replacing it.
>
> Cheers,
> Larry
>
> [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=125524
>
> Peter wrote:
>> Did not see this in here anywhere. But from the Tomcat Windows FAQ
>> http://tomcat.apache.org/faq/windows.html#lock it is pretty clear that on
>> Windows, Tomcat MUST have a file in:
>> metadata\.plugins\org.eclipse.wst.server.core\tmp3\conf\cont ext.xml:
>> <Context
>> antiResourceLocking="true"
>> antiJARLocking="true"
>> WTP has to be extended to include the context.xml file inside of the
>> server config. I am hoping this gets added. I am getting "unable to
>> publish" every single time with 1 WebApp Project, with 1 Java Project
>> referenced via the J2EE Module dependencies. Both are HelloWorldish: a 3
>> line servlet invoking a getString method on a 5 line class in the Java
>> Project.
>>
>> Works fine - until the Java Project is changed - then I get the "Unable
>> to publish on the jar as it is locked." WebApp changes are fine though -
>> hot deploy works beautifully!!
>>
>> Unfortunately these flags cause Tomcat to deploy the TMP folder - so WTP
>> needs to be tweaked to get this to work....adding the context.xml file
>> manually will not work :(
>>
>> Other then this - which will cause me alot of grief - great work.
>>
>> ...Peter
>>
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180185 is a reply to message #180151] Thu, 28 September 2006 16:23 Go to previous messageGo to next message
Larry Isaacs is currently offline Larry IsaacsFriend
Messages: 1313
Registered: July 2009
Senior Member
The target is tomorrow, 09/29. However, I would guess there are lots of
details involved in getting everything "ready". So a short delay is
possible due to issues that might crop up.

Cheers,
Larry

Peter wrote:
> Hi Larry,
>
> Thanks for the update. I noticed that 1.5.1 is not released yet - any ideas
> when?
>
> I tried to dump the 1.5.1 zip file into my eclipse folder, but got a bunch
> of SWT errors. Then I noticed I would need to update eclipse itself, along
> with gef, etc.
>
> no easy way to do this I am guessing :(
>
> Thanks,
> Peter
>
>
> eg.
> a.. The Eclipse driver used in this build is
> eclipse-SDK-M20060921-0945-linux-gtk.tar.gz. You can find a suitable driver
> for your platform at here
>
> a.. The EMF driver used in this build is emf-sdo-xsd-SDK-2.2.1RC2.zip
>
> a.. The GEF driver used in this build is GEF-SDK-M20060920-1505.zip
>
> a.. Java EMF Model Runtime driver used in this build is
> JEM-SDK-M20060918.zip
>
> "Larry Isaacs" <Larry.Isaacs@sas.com> wrote in message
> news:efgejl$3un$1@utils.eclipse.org...
>> Are you using a current WTP 1.5.1? The fix for Bug 125524[1] in WTP 1.5.1
>> should work around the jar locking problem by rewriting the jar rather
>> than replacing it.
>>
>> Cheers,
>> Larry
>>
>> [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=125524
>>
>> Peter wrote:
>>> Did not see this in here anywhere. But from the Tomcat Windows FAQ
>>> http://tomcat.apache.org/faq/windows.html#lock it is pretty clear that on
>>> Windows, Tomcat MUST have a file in:
>>> metadata\.plugins\org.eclipse.wst.server.core\tmp3\conf\cont ext.xml:
>>> <Context
>>> antiResourceLocking="true"
>>> antiJARLocking="true"
>>> WTP has to be extended to include the context.xml file inside of the
>>> server config. I am hoping this gets added. I am getting "unable to
>>> publish" every single time with 1 WebApp Project, with 1 Java Project
>>> referenced via the J2EE Module dependencies. Both are HelloWorldish: a 3
>>> line servlet invoking a getString method on a 5 line class in the Java
>>> Project.
>>>
>>> Works fine - until the Java Project is changed - then I get the "Unable
>>> to publish on the jar as it is locked." WebApp changes are fine though -
>>> hot deploy works beautifully!!
>>>
>>> Unfortunately these flags cause Tomcat to deploy the TMP folder - so WTP
>>> needs to be tweaked to get this to work....adding the context.xml file
>>> manually will not work :(
>>>
>>> Other then this - which will cause me alot of grief - great work.
>>>
>>> ...Peter
>>>
>
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180193 is a reply to message #180185] Thu, 28 September 2006 16:52 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: peterdnight.yahoo.com

Thanks for update Larry. I have really grown into the WTP model. I tried
MyEclipse, but found too many core issues that WTP addresses. They do have
one of the nicest JSP editors (with preview panes for both firefox and ie).

Not sure if this is cus I have used rad/wsad for the past 5 years, and I am
very used to the approach: servers, server projects, etc.

I did notice WTP shares one unpleasant commonality with RAD - when adding a
server config - you are NOT prompted for a server project. It is very nice
to have multiple server projects for the same reason it is nice to have
muliple Java projects ....

Any way - GREAT WORK WTP guys. Looking forward to the 1.5.1 update.....

....peter


"Larry Isaacs" <Larry.Isaacs@sas.com> wrote in message
news:efgsuf$qpm$1@utils.eclipse.org...
> The target is tomorrow, 09/29. However, I would guess there are lots of
> details involved in getting everything "ready". So a short delay is
> possible due to issues that might crop up.
>
> Cheers,
> Larry
>
> Peter wrote:
>> Hi Larry,
>>
>> Thanks for the update. I noticed that 1.5.1 is not released yet - any
>> ideas when?
>>
>> I tried to dump the 1.5.1 zip file into my eclipse folder, but got a
>> bunch of SWT errors. Then I noticed I would need to update eclipse
>> itself, along with gef, etc.
>>
>> no easy way to do this I am guessing :(
>>
>> Thanks,
>> Peter
>>
>>
>> eg.
>> a.. The Eclipse driver used in this build is
>> eclipse-SDK-M20060921-0945-linux-gtk.tar.gz. You can find a suitable
>> driver for your platform at here
>>
>> a.. The EMF driver used in this build is emf-sdo-xsd-SDK-2.2.1RC2.zip
>>
>> a.. The GEF driver used in this build is GEF-SDK-M20060920-1505.zip
>>
>> a.. Java EMF Model Runtime driver used in this build is
>> JEM-SDK-M20060918.zip
>>
>> "Larry Isaacs" <Larry.Isaacs@sas.com> wrote in message
>> news:efgejl$3un$1@utils.eclipse.org...
>>> Are you using a current WTP 1.5.1? The fix for Bug 125524[1] in WTP
>>> 1.5.1 should work around the jar locking problem by rewriting the jar
>>> rather than replacing it.
>>>
>>> Cheers,
>>> Larry
>>>
>>> [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=125524
>>>
>>> Peter wrote:
>>>> Did not see this in here anywhere. But from the Tomcat Windows FAQ
>>>> http://tomcat.apache.org/faq/windows.html#lock it is pretty clear that
>>>> on Windows, Tomcat MUST have a file in:
>>>> metadata\.plugins\org.eclipse.wst.server.core\tmp3\conf\cont ext.xml:
>>>> <Context
>>>> antiResourceLocking="true"
>>>> antiJARLocking="true"
>>>> WTP has to be extended to include the context.xml file inside of the
>>>> server config. I am hoping this gets added. I am getting "unable to
>>>> publish" every single time with 1 WebApp Project, with 1 Java Project
>>>> referenced via the J2EE Module dependencies. Both are HelloWorldish: a
>>>> 3 line servlet invoking a getString method on a 5 line class in the
>>>> Java Project.
>>>>
>>>> Works fine - until the Java Project is changed - then I get the "Unable
>>>> to publish on the jar as it is locked." WebApp changes are fine
>>>> though - hot deploy works beautifully!!
>>>>
>>>> Unfortunately these flags cause Tomcat to deploy the TMP folder - so
>>>> WTP needs to be tweaked to get this to work....adding the context.xml
>>>> file manually will not work :(
>>>>
>>>> Other then this - which will cause me alot of grief - great work.
>>>>
>>>> ...Peter
>>>>
>>
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180208 is a reply to message #180119] Thu, 28 September 2006 19:41 Go to previous messageGo to next message
Darryl Miles is currently offline Darryl MilesFriend
Messages: 123
Registered: July 2009
Senior Member
Larry Isaacs wrote:
> Are you using a current WTP 1.5.1? The fix for Bug 125524[1] in WTP
> 1.5.1 should work around the jar locking problem by rewriting the jar
> rather than replacing it.

Eeeeeeeeek! Don't re-write JARs, you'll have your TC JVM crashing out.

Around the WTP 0.7/1.0 era where the default way for a file to be
updated was to truncate it in place and then rewrite it. This was not
an atomic file update (which should be used for any cooperating processes).

The problem this caused was that the JVMs ZIP file handling functions
still had open handles, cached file offsets, cached zip header/directory
data so it did seek on the ZIP file it believed it had before it and
read what was there.

Unfortunately the native code which does the ZIP file handling is not
hardened to reading now corrupt data (because you re-wrote the contents
from what the JVM expected to see there). So the native code crashed
the JVM out. I was getting between 2 and 3 of these a day back then.


Maybe I don't care to much about this problem because not being able to
delete a file is a Win32'ism and I don't develop on windows. So in
effect because your patch shouldn't affect me I'm not going to complain
I can't turn it off. :)


Darryl
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180225 is a reply to message #180208] Thu, 28 September 2006 20:24 Go to previous messageGo to next message
Larry Isaacs is currently offline Larry IsaacsFriend
Messages: 1313
Registered: July 2009
Senior Member
Thanks Darryl. Publishing to a running Tomcat can have other problems
as well. I'll see if the chances of success are bad enough to make it
not worth doing. Unfortunately removing it guarantees we are back where
we were with the original post. :(

If users of the feature find this causes more problems that its worth,
please speak up.

Cheers,
Larry

Darryl L. Miles wrote:
> Larry Isaacs wrote:
>> Are you using a current WTP 1.5.1? The fix for Bug 125524[1] in WTP
>> 1.5.1 should work around the jar locking problem by rewriting the jar
>> rather than replacing it.
>
> Eeeeeeeeek! Don't re-write JARs, you'll have your TC JVM crashing out.
>
> Around the WTP 0.7/1.0 era where the default way for a file to be
> updated was to truncate it in place and then rewrite it. This was not
> an atomic file update (which should be used for any cooperating processes).
>
> The problem this caused was that the JVMs ZIP file handling functions
> still had open handles, cached file offsets, cached zip header/directory
> data so it did seek on the ZIP file it believed it had before it and
> read what was there.
>
> Unfortunately the native code which does the ZIP file handling is not
> hardened to reading now corrupt data (because you re-wrote the contents
> from what the JVM expected to see there). So the native code crashed
> the JVM out. I was getting between 2 and 3 of these a day back then.
>
>
> Maybe I don't care to much about this problem because not being able to
> delete a file is a Win32'ism and I don't develop on windows. So in
> effect because your patch shouldn't affect me I'm not going to complain
> I can't turn it off. :)
>
>
> Darryl
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180233 is a reply to message #180225] Thu, 28 September 2006 21:25 Go to previous messageGo to next message
Darryl Miles is currently offline Darryl MilesFriend
Messages: 123
Registered: July 2009
Senior Member
Larry Isaacs wrote:
> If users of the feature find this causes more problems that its worth,
> please speak up.


Just to put this into context, the chances of this code being executed
are now very remote.

1) You need to be modifying the code inside your JARs to cause your JARs
to need rewriting, like a contributing J2EE Utility Project.

2) It only truncates in place and re-writes _WHEN_ it has already
experienced a problem deleting/renaming the file for atomic updates.

3) If any great concern for this feature does occur, I sure a simple
checkbox to disable this feature would be enough to keep everyone happy.
With the default being whatever the majority dictates.


So this would need to be weighed up against some statistics to confirm
that the approach actually works and makes life better in the cases that
it does kick in. If the file is locked already, that usually means the
JVM has already got it open or memory mapped, thus pinning it down,
which opens up the case for the situation I detailed. Because of that
my gut feeling would be that this approach doesn't work anyway but I
have no proof to support that opinion.


I would favor having an option to export a J2EE Utility Project files as
class files into WEB-INF/classes via configuration setting in the J2EE
Module Dependencies page. The setting would allow export as JAR (like
now) or export at classes. My understanding is that .class files follow
conventional open, read, close with no open file handle or memory
mapping of the the file, simply because there are too many .class files
and they are too small an amount of data to cache in anyway.

I can't see anything wrong in the development environment with doing
that (providing precedence rules of the Build Path -> Order/Export are
followed and any resource name/location conflicts result in a warning).

Most JAR libraries don't have a MANIFEST configuration or any other
META-INF files but even some situations (like TLDs) can be handled
correctly and still get exported as classes.

Unrolling the JAR file into a directory is basically what Anti-Jar
locking does at the container (the way I understand it).


Darryl
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180256 is a reply to message #180233] Fri, 29 September 2006 12:34 Go to previous messageGo to next message
Larry Isaacs is currently offline Larry IsaacsFriend
Messages: 1313
Registered: July 2009
Senior Member
I did do testing to confirm that at least in a simple test case, all the
classes in a utility jar containing multiple classes were successfully
picked up after the jar rewrite. It does raise an important point not
yet mentioned, which is that the webapp has to reload for the jar to
have a chance of working correctly. My tests seem to suggest that
enough will get discarded for there to be hope that stale jar data won't
be a problem after the reload. Unfortunately, it's this auto-reload, or
the lack of one occurring, that the keeps the "anti-jarlocking" approach
from working.

I judged it too big a change for WTP 1.5.x, but I plan for WTP 2.0 to
hook the rewrite behavior to the auto-reload attribute of the context.
If the webapp isn't going to reload, then as you suggest, rewriting the
jar is guaranteed to be a bad idea. If there are general problems with
the jar rewriting, a option could be added as you suggest.


For general consumption:

It should be noted that this kind of "live" development can have
problems beyond what might be caused by jar rewriting. Updates to JSPs
shouldn't cause a problem, but changes to classes can, depending on the
nature of the changes. For example, by default Tomcat tries to save and
restore HTTP session data across a reload. If you change a class that
was stored in a session attribute, it may not restore successfully and
some undefined malfunction may be the result. If after such a reload,
something unusual happens which isn't obviously due to a mistake you
just made, it would be wise to restart Tomcat and see if the problem
still occurs before assuming it's due to a bug. It could be a side
effect of the live update of the webapp.

It should also be noted that getting the old webapp to fully garbage
collect can be quite a challenge. One of the objects most likely to
resist garbage collection is the old webapp's classloader, which can
represent a sizable memory leak. This leak can be due to issues in
various versions of Tomcat as well as mistakes in your webapp. It isn't
unusual to occasionally need to restart Tomcat just to recover the
leaked memory when attempting "live" development in this fashion.

Cheers,
Larry

P.S. For the idea of exporting utility projects as classes, I am still
looking at what might be done to integrate the approach used by the
experimental Tomcat plug-in found in:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=124914

Darryl L. Miles wrote:
> Larry Isaacs wrote:
>> If users of the feature find this causes more problems that its worth,
>> please speak up.
>
>
> Just to put this into context, the chances of this code being executed
> are now very remote.
>
> 1) You need to be modifying the code inside your JARs to cause your JARs
> to need rewriting, like a contributing J2EE Utility Project.
>
> 2) It only truncates in place and re-writes _WHEN_ it has already
> experienced a problem deleting/renaming the file for atomic updates.
>
> 3) If any great concern for this feature does occur, I sure a simple
> checkbox to disable this feature would be enough to keep everyone happy.
> With the default being whatever the majority dictates.
>
>
> So this would need to be weighed up against some statistics to confirm
> that the approach actually works and makes life better in the cases that
> it does kick in. If the file is locked already, that usually means the
> JVM has already got it open or memory mapped, thus pinning it down,
> which opens up the case for the situation I detailed. Because of that
> my gut feeling would be that this approach doesn't work anyway but I
> have no proof to support that opinion.
>
>
> I would favor having an option to export a J2EE Utility Project files as
> class files into WEB-INF/classes via configuration setting in the J2EE
> Module Dependencies page. The setting would allow export as JAR (like
> now) or export at classes. My understanding is that .class files follow
> conventional open, read, close with no open file handle or memory
> mapping of the the file, simply because there are too many .class files
> and they are too small an amount of data to cache in anyway.
>
> I can't see anything wrong in the development environment with doing
> that (providing precedence rules of the Build Path -> Order/Export are
> followed and any resource name/location conflicts result in a warning).
>
> Most JAR libraries don't have a MANIFEST configuration or any other
> META-INF files but even some situations (like TLDs) can be handled
> correctly and still get exported as classes.
>
> Unrolling the JAR file into a directory is basically what Anti-Jar
> locking does at the container (the way I understand it).
>
>
> Darryl
Re: Support for Tomcat AntiJar Locking...Needed for Publishing... [message #180280 is a reply to message #180256] Fri, 29 September 2006 15:34 Go to previous message
Darryl Miles is currently offline Darryl MilesFriend
Messages: 123
Registered: July 2009
Senior Member
> It does raise an important point not
> yet mentioned, which is that the webapp has to reload for the jar to
> have a chance of working correctly. My tests seem to suggest that
> enough will get discarded for there to be hope that stale jar data won't
> be a problem after the reload. Unfortunately, it's this auto-reload, or
> the lack of one occurring, that the keeps the "anti-jarlocking" approach
> from working.

Hmm... I'd taken it as a given that the point of rewriting the JAR file
was precisely because you don't want to redeploy more often than you
have to. This is why we're happy to jump through hoops to make this
possible with increasingly more complex solutions to achieve the goal of
a seamless development process.



If you are happy to deploy after each JAR file modification (because
thats the only way a JAR has a "chance of working correctly"), then the
route of least code, effort and complexity would be to batch up all JAR
modifications while the web-app is deployed. But commit those changes
to the file system as soon as the web-app completes un-deployment (by
way of undeploy, container shutdown, container crash, whatever... There
still can be issues where the container is not restarted and Igor's
plugging seeks to address that).

But this isn't a goal I seek myself, but would fix the windows case.



> I judged it too big a change for WTP 1.5.x, but I plan for WTP 2.0 to
> hook the rewrite behavior to the auto-reload attribute of the context.
> If the webapp isn't going to reload, then as you suggest, rewriting the
> jar is guaranteed to be a bad idea. If there are general problems with
> the jar rewriting, a option could be added as you suggest.

I find Eclipse most productive with manual reloads only. As things
stand now Eclipse reloads more often than it should and this alone is a
annoyance enough for me to not use that feature.

The classic, I edit a Servlet Java file which has XDoclet attributes
within it. When I save the file XDoclet runs but the resulting web.xml
is identical to the one currently running. But Eclipse updates it
anyway and so this would otherwise cause a web-app redeploy. Normally
hot-code replace did the trick as my servlet class was loaded and running.



> For example, by default Tomcat tries to save and
> restore HTTP session data across a reload. If you change a class that
> was stored in a session attribute, it may not restore successfully and
> some undefined malfunction may be the result.

And there is a mechanism exactly for this problem "serialVersionUID".
All we need to do ensure while we are developing and making
serialization affecting modifications to classes is teach eclipse how to
modify the value on the fly during the development process.


Darryl
Previous Topic:Increase Memory?
Next Topic:Exceptions in JSP editor
Goto Forum:
  


Current Time: Mon Dec 22 00:54:51 GMT 2014

Powered by FUDForum. Page generated in 0.03716 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software