I found out that there's an environment
variable to set the timeout in seconds: AS_START_TIMEOUT
If you set it, for example to 120 (default is
60), before running maven build, it should increase the
timeout. If that works for you, you could add it to where
you document this as a known issue.
Confirming that
Defender is the problem. I tested this out by repeatedly
adding and removing \GitHub from the scan exclusion
list. The results are basically consistent, though not
completely. There may be other factors slowing things
down beyond 60 seconds, but clearly Defender effects the
outcome.
For now, I’ll add a
known issues. Hopefully we can properly sort this soon.
Pretty sure a configurable timeout will do it.
Yes, that's clear for
7.0.23. We should catch the error and try to execute
GlassFish as before as a fallback behavior.
7.0.22 doesn't use this
script, I guess the problem is because it just takes
longer than 60 seconds to start.
We'll proceed with
creating an official GlassFish Arquillian plugin in the
Eclipse GlassFish project, which would initially wrap
OmniFish plugin and would also support Embedded
GlassFish. I was able to run the CargoTracker tests with
Embedded GlassFish, although there are a few catches
that I needed to address:
Embedded GlassFish is started on the same
classpath as Maven. In the case of CargoTracker,
there's a bit older Jersey dependency in the maven
project and it takes precedence over the Jersey
classes in Embedded GlassFish JAR. I fixed it by
surefire configuration to exclude the Jersey artifact
from the classpath.
GlassFish requires a few add-opens to run
on newer Java without issues. GlassFish server adds
them in domain.xml, but Embedded GlassFish needs them
as JVm arguments. Again, surefire configuration allows
adding them as additional JVM arguments
It would be even better
if we add support for running Embedded GlassFish as a
runnable JAR to the Arquillian plugin, that would
address both issues automatically and would work
similarly to running Payara Micro.
I have all working
locally. Once we release an official Arquillian plugin
via the Eclipse GlassFish project, I can contribute the
changes.
Mystery solved for 7.0.23 by
trying to run standalone. Log attached. My secure
machine won't let me do what 7.0.23 wants/needs. I
gave it an honest try.
On 3/26/2025 5:09 AM, Ondro
Mihályi wrote:
OK, now I found a
hint. It's this:
Time elapsed: 69.36 s
The default timeout
on start-domain command is 60 seconds. When I block
GlassFish execution, I get the same error as you
reported with the JBoss Arquillian connector.
It's very likely
that the Windows Defender slows down GlassFish
startup too much. I saw it before, when Windows
Defender would make Netbeans startup unbearably slow
to start.
It's not possible to
change the start-domain timeout in any of the
Arquillian containers. We could add support for
timeout in the OmniFish container, but it's not
supported now.
I’ll try as soon
as I get a moment. A new theory I have is that
this is Windows Defender causing problems by
locking newly created files for scanning. That’s
why it’s random. My Microsoft super duper secure
machine may not allow me to turn off Windows
Defender. The GlassFish version probably just
makes things marginally worse with Defender
somehow. My other Windows machine is personal and
doesn’t have the annoying and useless virus
scanner on it. Things are fine there.
Hantsy may also
be onto something with the timeout thing. I did
try all the timeout settings I know about, but
maybe there’s one I don’t know?
I checked the
code and we did a change in 7.0.23 to support
Windows SSH nodes: https://github.com/eclipse-ee4j/glassfish/commit/7d657198e4fde648331bd441abb94726d2ad8ff8.
On Windows, we now change the command line in the
prepareWindowsEnvironment to write the command
into a powershell script, and execute it via
powershell, instead of executing directly. Maybe
this is not stable enough, and you see even worse
behavior with 7.0.23. That doesn't explain the
issues with 7.0.22.
I tried
installing a Windows server in AWS, with
Temurin Temurin-17.0.8+7, and run mvnw install
-Pglassfish -Dglassfish.version=7.0.23 on the
current master without any issues, the tests pass.
I see that 7.0.23 prints the following into the
log: Executing: powershell.exe -noninteractive
-File ... followed by the path to the powershell
file. I ran all successfully also with 7.0.22,
which doesn't use powershell. I tried it several
times and always successfully.
Can you check the
server.log of GlassFish when you get an
unsuccessful run? The exception you attached
doesn't explain why the issue happened, only that
the start-domain was not successfully executed.
Running with
Embedded GlassFish would be an alternative,
however, there's no official Arquillian connector
for it. Only an OmniFish Arquillian connector. We
were discussing with Arjan, and because of license
issues (it contains files copyrighted by Oracla
and Payara, and license under CDDL and GPL), we're
not able to donate it to Eclipse GlassFish
project. But it should be possible to use it as a
binary dependency in GlassFish project, and create
new Arquillian container in the Eclipse GlassFish
project, that would include it as a dependency and
just wrap it.
This is simply bizarre. I
am out of time this morning but I will validate
this observation in the evening.
This entire problem started
with an attempted upgrade to 7.0.22 (looks like
by someone who did not test on Windows). It's
worse with 7.0.23. It seems to not be a problem
in 7.0.21.
I would still be very
keen on moving to GlassFish Embedded in a
way that's proper for an Eclipse Foundation
project if that's possible instead. This
problem can be someone else's headache to
solve.
I did notice that if
GlassFish was not shut down completely, I
get this:
Waiting finished
after 60,001 ms.
No response from the Domain Administration
Server (domain1) after 60 seconds.
The command is either taking too long to
complete or the server has failed.
Please see the server log files for
command status.
Please start with the --verbose option in
order to see early messages. No response
from the Domain Administration Server
(domain1) after 60 seconds.
The command is either taking too long to
complete or the server has failed.
Please see the server log files for
command status.
Please start with the --verbose option in
order to see early messages.
I'll try to pin down
under what exact conditions this happens.
It's not easy to reproduce. This is easier
to reproduce manually:
There is a process
already using the admin port 4,848 -- it
probably is another instance of a
GlassFish server.
On 3/24/2025 10:44
AM, Reza Rahman wrote:
This is what I did:
C:\GitHub\cargotracker>
cd "C:\Program Files\Eclipse
Adoptium\jdk-17.0.8.101-hotspot\bin\"
When I try
with the OmniFish plugin, I get a
slightly different error with the
same overall symptoms. The stack
trace is attached.
On
3/23/2025 6:36 PM, Reza Rahman
wrote:
Thanks
very much for taking a look. I
can try the OmniFish plugin as a
test.
However,
I cannot use it in the project
in relation to GlassFish. I must
use something in the GlassFish
or Arquillian domain instead.
I’ve already checked with the
EMO on this. My hope would be
that either the Arquillian
plugin is properly maintained or
a plugin is created within the
GlassFish project.
Alternatively, I would not mind
adding support for an OmniFish
branded runtime. That’s also
something that the EMO has
suggested.
I
haven't tried anything yet, but
from a cursory glance at master
the first thing that stands out
is that for GlassFish an alpha
version of the unmaintained
JBoss Arquillian connector is
used, that was only suited for
GlassFish 6:
The Cargo Tracker project (https://github.com/eclipse-ee4j/cargotracker)
has tried to support GlassFish
but once again I am running
into seeming
stability issues. While
running tests with Arquillian,
the attached
issue now suddenly crops up. I
have absolutely no idea what
is going on.
Honestly, this sort of thing
never happens with Payara or
Liberty, which
are the two other runtimes
Cargo Tracker supports.
Is there someone here that can
help? Otherwise I will remove
GlassFish
support from Cargo Tracker for
now.