| Re: [openj9-dev] Pull Request builds are changing |
The main purpose of this change is to align the PR builds with the other builds (Nightly, OMR Acceptance, etc) in terms of how the Jenkins jobs are structured. The two most notable changes are:- Pull Request builds will now be a "pipeline of jobs" instead of a single job- Pull request builds will now use the AdoptOpenJDK Test frameworkOther notable changes:- Pull Request trigger Test target syntax change- "Jenkins test sanity.functional <platform> <version>"- Note: "sanity" and "extended" are still supported but will map to sanity.functional and extended.functional respectively.- System test PR support- Jenkins test sanity.system, or extended.system- Please continue to be mindful of the amount of testing you run, how long each test bucket takes to execute, and how busy the farm is.- JITaaS PR support- Jenkins test <target>+jitaas <platform> <version>- Note the pipelines will support this on all platforms and versions but I'm not sure of the status of support on the JITaaS side.- CMake PR support- Jenkins compile xlinuxcmake jdk11- No testing yet- Only supported on xlinux jdk 8,11 at the moment- Separated Compile and Test jobs means 1 compile feeds an sdk into many test jobs- Github PR status updates per job- Very similar to how it is already. The difference being PRs will receive separate updates for compiles and tests.- Only 1 top-level job will trigger. This job will trigger all the necessary downstream jobs.- You will get a PR status update for this top-level job as well as all the downstream jobs.- Since there is only 1 top level job, you can only run 1 "build" at a time (per PR). Launching another build will cancel the previous one.- For example if you run "Jenkins compile zlinux,plinux jdk8" and then realize you also needed xlinux, you will have to issue another comment "Jenkins compile zlinux,plinux,xlinux jdk8". Ie. you cannot simply launch another build with just xlinux because it will cancel the first build.- Launching multiple builds from the same PR is supported. If the same combination of Spec, Version and Test level are launched, the previous test will be cancelled regardless of what SHA is being tested in the existing build. Note compiles will not be cancelled if subsequent ones are launched.
Thanks Adam for reflecting the state of the discussions into https://github.com/eclipse/openj9/pull/2836#issuecomment-425212169 I've summarized it here - please chime in with any corrections.For those following along - two concerns were raised:1) Will this increase the disk usage on jenkins master?> Adam's current belief and rough back of the envelop math indicates disk usage may actually go down with this approach even though the SDKS will need to be kept around to pass from the compiles to the test jobs.2) Auto canceling of builds works on about the right granularity today. Will this change affect the cancelation?> The original proposal would have changed the cancelation granularity. Alternatives would have required committers to manually cancel builds which is error prone. Current proposals are to keep the many PR jobs and have act as simple forwarders to the common pipeline job. This shouldn't change the committer workflow while still moving towards the standardized build/test pipelines.Let's move the discussion into issue 2836.--Dan----- Original message -----
From: Adam Brousseau <adam.brousseau88@xxxxxxxxx>
Sent by: openj9-dev-bounces@xxxxxxxxxxx
To: openj9 developer discussions <openj9-dev@xxxxxxxxxxx>
Cc:
Subject: Re: [openj9-dev] Pull Request builds are changing
Date: Tue, Sep 25, 2018 2:59 PM
I agree it is preferably to be able to drive all interactions through Github as you say. I think it would be possible to introduce a "cancel" option, and possibly a "restart" as well. Can I suggest we raise this as an enhancement issue so it can be prioritized accordingly. I don't believe it should block this initial work from going through.On Mon, Sep 24, 2018 at 4:11 PM, Daniel Heidinga <Daniel_Heidinga@xxxxxxxxxx> wrote:Adam, thanks for taking another look at this.> That is, if they are launching new builds to replace old ones, they should determine if the old ones are still running and> if they should be cancelled manually as they will not be cancelled automatically once we change over to this model.Can we introduce a "jenkins cancel [test level] [platform] [jdk]" form? Or a "jenkins restart test sanity plinux jdk10" style query? My preference is for committers to drive all interactions with jenkins through the PR builder as much as possible. This makes the workflow clear and leaves a record of what happened to builds, providing people the opportunity to indicate *why* a build was cancelled / restarted.Is adding a restart or cancel tag to the PR syntax doable?--Dan----- Original message -----
From: Adam Brousseau <adam.brousseau88@xxxxxxxxx>
Sent by: openj9-dev-bounces@xxxxxxxxxxx
To: openj9-dev@xxxxxxxxxxx
Cc:Subject: Re: [openj9-dev] Pull Request builds are changing
Date: Mon, Sep 24, 2018 1:27 PM
Dan,You are correct. With the proposed changes in my original email you would have to wait for each build (or set of builds) to complete before you launched the next one to avoid them being cancelled.The Jenkins Github Pull Request plugin has an option called "Cancel build on update". I couldn't figure out why when I "uncheck" it in the build configuration and then save, it reloaded with it "checked" again.Luckily I remembered that we "checked" this in the global configure a while back. I was able to disable it globally and then disable it in the job. Now you will be able to concurrently launch builds. The caveat being you will need to do it in separate PR comments.While it is usually desired to "cancel build on update" when the builds are all separate, I understand it being undesirable when you only have 1 job as far as Jenkins<->Github is concerned.This change in behaviour require some committer diligence. That is, if they are launching new builds to replace old ones, they should determine if the old ones are still running and if they should be cancelled manually as they will not be cancelled automatically once we change over to this model.AdamOn Mon, Sep 24, 2018 at 12:15 AM Daniel Heidinga <Daniel_Heidinga@xxxxxxxxxx> wrote:Adam, I'm a little concerned by this:> - Since there is only 1 top level job, you can only run 1 "build" at a time (per PR). Launching another build will cancel the previous one.I often want to launch builds on different platforms / jdk levels that vary in the amount of testing. From simple compiles on some releases to deep testing on other platforms. I"m not sure how to do that with this scheme without having to wait for each build to complete before launching the next.ie:jenkins compile win jdk8jenkins test sanity plinux jdk10jenkins test extended xlinux jdk11Hopefully I've overlooked something here and the system doesn't turn our parallel build launches into a serial process.--Dan----- Original message -----
From: Adam Brousseau <adam.brousseau88@xxxxxxxxx>
Sent by: openj9-dev-bounces@xxxxxxxxxxx
To: openj9-dev@xxxxxxxxxxx
Cc:
Subject: [openj9-dev] Pull Request builds are changing
Date: Fri, Sep 21, 2018 3:59 PM
Coming soon to a Pull Request near you...Joe sent an email last week proposing the Pull Request (PR) triggers be adjusted so that 'all' would no longer be valid for test level or jdk version (only for platform). Since there has been no feedback or objection to this, we will go ahead and implement this change in #2836.PLEASE NOTE there are many additional changes that are coming with #2836. Since there is no major change to how the PRs are launched, I have not set in stone a "cut-over" date, and will likely announce on the OpenJ9 slack once the change is in place.The main purpose of this change is to align the PR builds with the other builds (Nightly, OMR Acceptance, etc) in terms of how the Jenkins jobs are structured. The two most notable changes are:- Pull Request builds will now be a "pipeline of jobs" instead of a single job- Pull request builds will now use the AdoptOpenJDK Test frameworkOther notable changes:- Test target syntax change- "Jenkins test sanity.functional <platform> <version>"- Note: "sanity" and "extended" are still supported but will map to sanity.functional and extended.functional respectively.- System test support- Jenkins test sanity.system, or extended.system- Please continue to be mindful of the amount of testing you run, how long each test bucket takes to execute, and how busy the farm is.- Cmake platform support- Jenkins compile xlinuxcmake jdk9- No testing yet- Only supported on jdk9 at the moment- Separated Compile and Test jobs means 1 compile feeds an sdk into many test jobs- Github PR status updates per job- Very similar to how it is already. The difference being PRs will receive separate updates for compiles and tests.- Only 1 top-level job will trigger. This job will trigger all the necessary downstream jobs.- You will get a PR status update for this top-level job as well as all the downstream jobs.- Since there is only 1 top level job, you can only run 1 "build" at a time (per PR). Launching another build will cancel the previous one.- For example if you run "Jenkins compile zlinux,plinux jdk8" and then realize you also needed xlinux, you will have to issue another comment "Jenkins compile zlinux,plinux,xlinux jdk8". Ie. you cannot simply launch another build with just xlinux because it will cancel the first build.If you have any questions or concerns please let me know. Otherwise I expect to be making the switch sometime next week.Thanks,Adam_______________________________________________
openj9-dev mailing list
openj9-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/openj9-dev
_______________________________________________
openj9-dev mailing list
openj9-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/openj9-dev_______________________________________________
openj9-dev mailing list
openj9-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/openj9-dev
_______________________________________________
openj9-dev mailing list
openj9-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/openj9-dev
_______________________________________________
openj9-dev mailing list
openj9-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/openj9-dev
_______________________________________________
openj9-dev mailing list
openj9-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/openj9-dev