Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cbi-dev] git operations failing on CI - workaround!
Thanks Jonah for the great details. The PR has been merged and we are now waiting for it to be released upstream. We hope that we will be able to roll out a new version during the upgrade week (see https://status.eclipse.org/incidents/1jvztmyy9jnl).

Are you ok if I update the wiki (https://wiki.eclipse.org/Jenkins) with parts of the text you've written below?

Thanks again for your great help with this.

Cheers,

Mikaël Barbero 
Team Lead - Release Engineering | Eclipse Foundation
🐦 @mikbarbero
Eclipse Foundation: The Platform for Open Innovation and Collaboration

Le 8 juin 2020 à 17:16, Denis Roy <denis.roy@xxxxxxxxxxxxxxxxxxxxxx> a écrit :

Thanks, Jonah.

Denis

On 2020-06-05 5:03 p.m., Jonah Graham wrote:
Hi folks,

Those of us (like CDT) working with larger git repositories have been having lots of git failures on the new Jiro based Jenkins CI. There have been slow clones, and full on failing checkouts. 

For many of the examples it turns out this is a resource allocation problem in k8s and how Jenkins interacts with it. You can read about that in Bug 560283. Mikaël Barbero has an approved PR in Jenkins so hopefully the problems will be resolved soon. 

In the meantime you can do this workaround in your pipelines. Before the git/checkout pipeline step, fetch the changes from git in a sh step. This sh step can run in your main container, instead of the git/checkout which always runs in the JNLP container. Running in your main container means that much more resources are available to git and the git fetch will run faster.

For CDT our normal fetch time was 4+ minutes, and regularly timed out after 10 or even 20 minutes. With the workaround the fetch time is ~1 minute.

An example of what you need to change is this:

stage('Git Clone') {
steps {
container('cdt') {
checkout([$class: 'GitSCM', branches: [[name: '*/master']], ...
}
}
}


to

stage('Git Clone') {
steps {
container('cdt') {
timeout(activity: true, time: 20) {
/* Running the git fetch command manually is a workaround. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=560283#c16 */
sh 'git config remote.origin.url https://git.eclipse.org/r/cdt/org.eclipse.cdt.git'
sh 'git fetch --no-tags --force --progress -- https://git.eclipse.org/r/cdt/org.eclipse.cdt.git +refs/heads/*:refs/remotes/origin/*'
}
checkout([$class: 'GitSCM', branches: [[name: '*/master']], ...
}
}
}
You want to leave the git/checkout step in place as that is how Jenkins knows what to query and what to display changes on. With the change it won't have to do the heavy lifting of actually fetching the changes. 

See commit 5080b30 in cdt-infra for CDT's change in practice.
 
HTH,
Jonah


~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com

_______________________________________________
cbi-dev mailing list
cbi-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/cbi-dev
--
Denis Roy
Director, IT Services | Eclipse Foundation, Inc.
Eclipse Foundation: The Platform for Open Innovation and Collaboration
Twitter: @droy_eclipse
_______________________________________________
cbi-dev mailing list
cbi-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/cbi-dev

Attachment: signature.asc
Description: Message signed with OpenPGP

Back to the top