Re: [glassfish-dev] GlassFish Fails with Arquillian

[Reza Rahman <reza_rahman@xxxxxxxx>]

I hate to ask this - are you sure this works? It does not seem to have an effect on my system?

 

---

From: Ondro Mihályi <mihalyi@xxxxxxxxxxx>
Sent: Wednesday, March 26, 2025 9:11 AM
To: Reza Rahman <reza_rahman@xxxxxxxx>
Cc: glassfish developer discussions <glassfish-dev@xxxxxxxxxxx>; David Matějček <david.matejcek@xxxxxxxxxxx>; cargotracker developer discussions <cargotracker-dev@xxxxxxxxxxx>
Subject: Re: [glassfish-dev] GlassFish Fails with Arquillian

 

Hi Reza,

I found out that there's an environment variable to set the timeout in seconds: AS_START_TIMEOUT

If you set it, for example to 120 (default is 60), before running maven build, it should increase the timeout. If that works for you, you could add it to where you document this as a known issue.

Ondro

On Wed, Mar 26, 2025 at 12:17 PM Reza Rahman <reza_rahman@xxxxxxxx> wrote:

Confirming that Defender is the problem. I tested this out by repeatedly adding and removing \GitHub from the scan exclusion list. The results are basically consistent, though not completely. There may be other factors slowing things down beyond 60 seconds, but clearly Defender effects the outcome.

For now, I’ll add a known issues. Hopefully we can properly sort this soon. Pretty sure a configurable timeout will do it.

---

From: Ondro Mihályi <mihalyi@xxxxxxxxxxx>
Sent: Wednesday, March 26, 2025 6:56 AM
To: Reza Rahman <reza_rahman@xxxxxxxx>
Cc: glassfish developer discussions <glassfish-dev@xxxxxxxxxxx>; David Matějček <david.matejcek@xxxxxxxxxxx>; cargotracker developer discussions <cargotracker-dev@xxxxxxxxxxx>
Subject: Re: [glassfish-dev] GlassFish Fails with Arquillian

 

Yes, that's clear for 7.0.23. We should catch the error and try to execute GlassFish as before as a fallback behavior.

7.0.22 doesn't use this script, I guess the problem is because it just takes longer than 60 seconds to start.

We'll proceed with creating an official GlassFish Arquillian plugin in the Eclipse GlassFish project, which would initially wrap OmniFish plugin and would also support Embedded GlassFish. I was able to run the CargoTracker tests with Embedded GlassFish, although there are a few catches that I needed to address:

• Embedded GlassFish is started on the same classpath as Maven. In the case of CargoTracker, there's a bit older Jersey dependency in the maven project and it takes precedence over the Jersey classes in Embedded GlassFish JAR. I fixed it by surefire configuration to exclude the Jersey artifact from the classpath.
• GlassFish requires a few add-opens to run on newer Java without issues. GlassFish server adds them in domain.xml, but Embedded GlassFish needs them as JVm arguments. Again, surefire configuration allows adding them as additional JVM arguments

It would be even better if we add support for running Embedded GlassFish as a runnable JAR to the Arquillian plugin, that would address both issues automatically and would work similarly to running Payara Micro.

I have all working locally. Once we release an official Arquillian plugin via the Eclipse GlassFish project, I can contribute the changes.

Ondro

On Wed, Mar 26, 2025 at 11:42 AM Reza Rahman <reza_rahman@xxxxxxxx> wrote:

Mystery solved for 7.0.23 by trying to run standalone. Log attached. My secure machine won't let me do what 7.0.23 wants/needs. I gave it an honest try.

On 3/26/2025 5:09 AM, Ondro Mihályi wrote:

OK, now I found a hint. It's this:

Time elapsed: 69.36 s

The default timeout on start-domain command is 60 seconds. When I block GlassFish execution, I get the same error as you reported with the JBoss Arquillian connector.

It's very likely that the Windows Defender slows down GlassFish startup too much. I saw it before, when Windows Defender would make Netbeans startup unbearably slow to start.

It's not possible to change the start-domain timeout in any of the Arquillian containers. We could add support for timeout in the OmniFish container, but it's not supported now.

Ondro

On Wed, Mar 26, 2025 at 10:04 AM Reza Rahman <reza_rahman@xxxxxxxx> wrote:

I’ll try as soon as I get a moment. A new theory I have is that this is Windows Defender causing problems by locking newly created files for scanning. That’s why it’s random. My Microsoft super duper secure machine may not allow me to turn off Windows Defender. The GlassFish version probably just makes things marginally worse with Defender somehow. My other Windows machine is personal and doesn’t have the annoying and useless virus scanner on it. Things are fine there.

Hantsy may also be onto something with the timeout thing. I did try all the timeout settings I know about, but maybe there’s one I don’t know?

---

From: Ondro Mihályi <mihalyi@xxxxxxxxxxx>
Sent: Wednesday, March 26, 2025 4:31 AM
To: glassfish developer discussions <glassfish-dev@xxxxxxxxxxx>
Cc: David Matějček <david.matejcek@xxxxxxxxxxx>; Reza Rahman <reza_rahman@xxxxxxxx>; cargotracker developer discussions <cargotracker-dev@xxxxxxxxxxx>
Subject: Re: [glassfish-dev] GlassFish Fails with Arquillian

 

I checked the code and we did a change in 7.0.23 to support Windows SSH nodes: https://github.com/eclipse-ee4j/glassfish/commit/7d657198e4fde648331bd441abb94726d2ad8ff8. On Windows, we now change the command line in the prepareWindowsEnvironment to write the command into a powershell script, and execute it via powershell, instead of executing directly. Maybe this is not stable enough, and you see even worse behavior with 7.0.23. That doesn't explain the issues with 7.0.22.

I tried installing a Windows server in AWS, with Temurin Temurin-17.0.8+7, and run mvnw install -Pglassfish -Dglassfish.version=7.0.23 on the current master without any issues, the tests pass. I see that 7.0.23 prints the following into the log: Executing: powershell.exe -noninteractive -File ... followed by the path to the powershell file. I ran all successfully also with 7.0.22, which doesn't use powershell. I tried it several times and always successfully.

Can you check the server.log of GlassFish when you get an unsuccessful run? The exception you attached doesn't explain why the issue happened, only that the start-domain was not successfully executed.

Running with Embedded GlassFish would be an alternative, however, there's no official Arquillian connector for it. Only an OmniFish Arquillian connector. We were discussing with Arjan, and because of license issues (it contains files copyrighted by Oracla and Payara, and license under CDDL and GPL), we're not able to donate it to Eclipse GlassFish project. But it should be possible to use it as a binary dependency in GlassFish project, and create new Arquillian container in the Eclipse GlassFish project, that would include it as a dependency and just wrap it.

Ondro

On Tue, Mar 25, 2025 at 2:38 PM Reza Rahman via glassfish-dev <glassfish-dev@xxxxxxxxxxx> wrote:

This is simply bizarre. I am out of time this morning but I will validate this observation in the evening.

This entire problem started with an attempted upgrade to 7.0.22 (looks like by someone who did not test on Windows). It's worse with 7.0.23. It seems to not be a problem in 7.0.21.

On 3/25/2025 7:27 AM, Reza Rahman wrote:

This is 7.0.22. I can try an upgrade today.

 

---

From: David Matějček <david.matejcek@xxxxxxxxxxx>
Sent: Tuesday, March 25, 2025 3:20 AM
To: glassfish developer discussions <glassfish-dev@xxxxxxxxxxx>
Cc: Reza Rahman <reza_rahman@xxxxxxxx>; cargotracker developer discussions <cargotracker-dev@xxxxxxxxxxx>
Subject: Re: [glassfish-dev] GlassFish Fails with Arquillian

 

Quick question, which gf version do you use?

We fixed many these issues for 7.0.23.

Dne út 25. 3. 2025 0:20 uživatel Reza Rahman via glassfish-dev <glassfish-dev@xxxxxxxxxxx> napsal:

So by sheer amount of random trial and error I was able to reduce the instances of this happening: https://github.com/eclipse-ee4j/cargotracker/commit/733884ef435290113a70958fd7922d34a2b83362. It is still happening intermittently. It looks like some kind of low level file locking and/or permissions issue. I'll keep trying but honestly I could use some real help.

I would still be very keen on moving to GlassFish Embedded in a way that's proper for an Eclipse Foundation project if that's possible instead. This problem can be someone else's headache to solve.

On 3/24/2025 11:08 AM, Reza Rahman wrote:

Also tried the following, it works fine:

& "C:\Program Files\Eclipse Adoptium\jdk-17.0.8.101-hotspot\bin\java.exe" -jar "C:\GitHub\cargotracker\target\glassfish7\glassfish\modules\admin-cli.jar" start-domain -t

I did notice that if GlassFish was not shut down completely, I get this:

Waiting finished after 60,001 ms.
No response from the Domain Administration Server (domain1) after 60 seconds.
The command is either taking too long to complete or the server has failed.
Please see the server log files for command status.
Please start with the --verbose option in order to see early messages. No response from the Domain Administration Server (domain1) after 60 seconds.
The command is either taking too long to complete or the server has failed.
Please see the server log files for command status.
Please start with the --verbose option in order to see early messages.

I'll try to pin down under what exact conditions this happens. It's not easy to reproduce. This is easier to reproduce manually:

There is a process already using the admin port 4,848 -- it probably is another instance of a GlassFish server.

On 3/24/2025 10:44 AM, Reza Rahman wrote:

This is what I did:

C:\GitHub\cargotracker> cd "C:\Program Files\Eclipse Adoptium\jdk-17.0.8.101-hotspot\bin\"

C:\Program Files\Eclipse Adoptium\jdk-17.0.8.101-hotspot\bin> java.exe -jar "C:\GitHub\cargotracker\target\glassfish7\glassfish\modules\admin-cli.jar" start-domain -t

This is the output I get:

Waiting finished after 48,817 ms.
Successfully started the domain : domain1
domain  Location: C:\GitHub\cargotracker\target\glassfish7\glassfish\domains\domain1
Log File: C:\GitHub\cargotracker\target\glassfish7\glassfish\domains\domain1\logs\server.log
Admin Port: 4,848

This still could be an issue, but at least the above works.

On 3/24/2025 6:17 AM, Arjan Tijms via glassfish-dev wrote:

Hi,

It looks like a rather basic thing:

"org.jboss.arquillian.container.spi.client.container.LifecycleException: Unable to execute [C:\Program Files\Eclipse Adoptium\jdk-17.0.8.101-hotspot\bin\java.exe, -jar, C:\GitHub\cargotracker\target\glassfish7\glassfish\modules\admin-cli.jar, start-domain, -t]"

Can you double check if the JDK is indeed at that location?

Kind regards,

Arjan Tijms

On Mon, 24 Mar 2025 at 09:22, Ondro Mihályi via glassfish-dev <glassfish-dev@xxxxxxxxxxx> wrote:

Can you try if it also happens on Linux/Mac? When I run the BookingServiceTest test with "maven test -Pglassfish" on master branch on Linux, it works.

I suspect that the problem is only on Windows, can you confirm it, Reza?

Ondro

On Mon, Mar 24, 2025 at 12:05 AM Reza Rahman via glassfish-dev <glassfish-dev@xxxxxxxxxxx> wrote:

When I try with the OmniFish plugin, I get a slightly different error with the same overall symptoms. The stack trace is attached.

On 3/23/2025 6:36 PM, Reza Rahman wrote:

Thanks very much for taking a look. I can try the OmniFish plugin as a test.

However, I cannot use it in the project in relation to GlassFish. I must use something in the GlassFish or Arquillian domain instead. I’ve already checked with the EMO on this. My hope would be that either the Arquillian plugin is properly maintained or a plugin is created within the GlassFish project. Alternatively, I would not mind adding support for an OmniFish branded runtime. That’s also something that the EMO has suggested.

---

From: Arjan Tijms <arjan.tijms@xxxxxxxxxxx>
Sent: Sunday, March 23, 2025 6:10 PM
To: glassfish developer discussions <glassfish-dev@xxxxxxxxxxx>
Cc: Reza Rahman <reza_rahman@xxxxxxxx>; cargotracker developer discussions <cargotracker-dev@xxxxxxxxxxx>
Subject: Re: [glassfish-dev] GlassFish Fails with Arquillian

 

Hi,

I haven't tried anything yet, but from a cursory glance at master the first thing that stands out is that for GlassFish an alpha version of the unmaintained JBoss Arquillian connector is used, that was only suited for GlassFish 6:

 <dependency>
          <groupId>org.jboss.arquillian.container</groupId>
          <artifactId>arquillian-glassfish-managed-6</artifactId>
          <version>1.0.0.Alpha1</version>
          <scope>test</scope>
</dependency>

Maybe that could explain some?

You could try with:

<dependency>
            <groupId>org.omnifaces.arquillian</groupId>
            <artifactId>arquillian-glassfish-server-managed</artifactId>
            <version>1.7</version>
            <scope>test</scope>
</dependency>

Kind regards,

Arjan Tijms

On Sun, 23 Mar 2025 at 20:47, Reza Rahman via glassfish-dev <glassfish-dev@xxxxxxxxxxx> wrote:

Hello,

The Cargo Tracker project (https://github.com/eclipse-ee4j/cargotracker)
has tried to support GlassFish but once again I am running into seeming
stability issues. While running tests with Arquillian, the attached
issue now suddenly crops up. I have absolutely no idea what is going on.
Honestly, this sort of thing never happens with Payara or Liberty, which
are the two other runtimes Cargo Tracker supports.

Is there someone here that can help? Otherwise I will remove GlassFish
support from Cargo Tracker for now.

Thanks,

Reza
_______________________________________________
glassfish-dev mailing list
glassfish-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/glassfish-dev

_______________________________________________
glassfish-dev mailing list
glassfish-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/glassfish-dev

_______________________________________________
glassfish-dev mailing list
glassfish-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/glassfish-dev

_______________________________________________
glassfish-dev mailing list
glassfish-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/glassfish-dev

_______________________________________________
glassfish-dev mailing list
glassfish-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/glassfish-dev

_______________________________________________
glassfish-dev mailing list
glassfish-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/glassfish-dev