Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [glassfish-dev] Issues with Jenkins
Never mind about there being an OpenShift outage, my mistake as the status looks green.  There was an outage on something else.

On Tue, Jul 15, 2025 at 1:45 PM Scott Marlow <scott.marlow.opensource@xxxxxxxxx> wrote:
I looked at https://status.redhat.com which shows that there is a (database) problem mentioned today but am not sure how that impacts GlassFish testing.  There is also a history of prior incidents listed as well.  Eclipse CI has its own infrastructure layer as well.  Continuing the discussion in https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/6369 is the right path to follow but sometimes it helps to know if there is an OpenShift outage.

In previous years we talked about moving GlassFish TCK testing to the GlassFish project which I think would require increasing the number of resource packs assigned to the GlassFish project.  If that seems important now, please start a new GlassFish thread for discussing how to do that.  Perhaps that discussion could be part of a new thread about EE 12 TCK testing for GlassFish.  

Scott

On Tue, Jul 15, 2025 at 10:46 AM David Matejcek via glassfish-dev <glassfish-dev@xxxxxxxxxxx> wrote:

Hi all, 

as you probably noticed last two weeks or even more, we are struggling with Jenkins performance, which randomly significantly decreases.

The slowdown is not so much random, as I started watching it, however at this moment I don't see any simple solution working without reducing automated quality assurance.

What we could do:

Prefered Solution
Backup Alternative 1
  • We will change the Jenkinsfile, so it will stop running expensive parts
    • Compiling asciidoc documentation
      • The most frequently killed part of the build
      • I tried to improve JVM settings, tried to reproduce issues from Jenkins, but nothing fixed that.
      • Documentation should be up to date and should pass the processing.
    • "Ant based zombie army" how I am calling that for several years
      • Obsoleted slow expensive part of the build, however covering some critical features
      • Has probably the highest score of "new bug kills"
      • Is slowly converted to the application-tests maven module, which is much faster. However our conversion tempo is slow - help is welcome!
Backup Alternative 2
  • We can temporarily configure Jenkinsfile to mark failed builds just as warnings.
  • Then reviewers could rerun builds locally and report the result, or builds can be reexecuted on Jenkins too, which is much slower.
Backup Alternative 3
  • Move from Jenkins to GitHub Actions
  • I am already refusing it, but just for the record
    • The problem is that on GHA we have just 4 thread CPU for the build and it is not suitable for integration testing. 
    • It is even slower than Jenkins CI.
    • GHA applies some fairness policy, so jobs can be suspended for a while if they "eat" too much.
Combination

As I am thinking about it now, we can probably also combine all three alternatives:

  1. Move the compilation of the documentation to GitHub Actions.
  2. Temporarily configure Jenkinsfile to mark failures as warnings.
  3. Reviewers then decide how to interpret failed steps and what to do with that.
  4. Jenkins CI admins will fix that without blocking us.

What do you think about it? Did I forgot something? Yeah, maybe I could check Jenkins plugins, if we can add some configuration ie. for monitoring the performance ...

-- 
David Matejcek | OmniFish
david.matejcek@xxxxxxxxxxx
_______________________________________________
glassfish-dev mailing list
glassfish-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/glassfish-dev

Back to the top