Hi folks,
Those of us (like CDT) working with larger git repositories have been having lots of git failures on the new Jiro based Jenkins CI. There have been slow clones, and full on failing checkouts.
For many of the examples it turns out this is a resource allocation problem in k8s and how Jenkins interacts with it. You can read about that in
Bug 560283. Mikaël Barbero has an approved PR in Jenkins so hopefully the problems will be resolved soon.
In the meantime you can do this workaround in your pipelines. Before the git/checkout pipeline step, fetch the changes from git in a sh step. This sh step can run in your main container, instead of the git/checkout which always runs in the JNLP container. Running in your main container means that much more resources are available to git and the git fetch will run faster.
For CDT our normal fetch time was 4+ minutes, and regularly timed out after 10 or even 20 minutes. With the workaround the fetch time is ~1 minute.
An example of what you need to change is this:
stage('Git Clone') {
steps {
container('cdt') {
checkout([$class: 'GitSCM', branches: [[name: '*/master']], ...
}
}
}
to
stage('Git Clone') {
steps {
container('cdt') {
timeout(activity: true, time: 20) {
}
checkout([$class: 'GitSCM', branches: [[name: '*/master']], ...
}
}
}
You want to leave the git/checkout step in place as that is how Jenkins knows what to query and what to display changes on. With the change it won't have to do the heavy lifting of actually fetching the changes.
HTH,
Jonah
