Hi everyone,
I came across the error exceptionWhileReadingPack today: "ERROR: Exception caught while accessing pack file {0}, the pack file might be corrupt, {1}. Caught {2} consecutive errors while trying to read this pack." Could somebody explain in which circumstance this can happen? I saw Luca's commit message back to 2017[2], but I would like to have more detail from you.
which version of JGit do you use ?
The FileNotFoundException is typically raised in three conditions:
1. file doesn't exist
2. incompatible read vs. read/write open modes
3. filesystem locking
4. temporary lack of resources (e.g. too many open files)
1. is already managed, 2. would never happen as packs are not
overwritten while with 3. and 4. it is worth logging the exception and
retrying to read the pack again.
I suspect that our installation is in the 3rd case because we're using EFS of AWS. Any suggestion will be welcome. Thanks a lot!
In general jgit expects (on *nix) a posix compliant filesystem. NFS is not posix compliant.
NFS client side caching is causing issues when multiple jgit instances access data stored on nfs
via different nfs client instances, e.g. if multiple machines access the same shared NFS volume.
NFS does not support atomic file rename which jgit uses to implement transactions.
Switching off nfs client caching is typically too slow to be useful.
There are a couple of workarounds in jgit to workaround some of the nfs issues.
Some of these workarounds need to be enabled by setting corresponding JGit options.
Note that these options have a negative impact on performance.
## NFS workarounds
- core.trustFolderStat: boolean
- default: true
- Whether to trust the pack folder's modification time. If set to false we will
always scan the .git/objects/pack folder to check for new pack files. If set to
true (default) we use the lastmodified attribute of the folder and assume that
no new pack files can be in this folder if his modification time has not
changed
- core.supportsAtomicFileCreation: boolean
- supported on Posix
- default: true
- if core.supportsAtomicCreateNewFile = false then after
successful creation of the lock file a hard link to that lock file is
created and the attribute nlink of the lock file is checked to be 2. If
multiple clients manage to create the same lock file nlink would be
greater than 2 showing the error. The hard link needs to be retained
until the corresponding file is no longer needed in order to prevent that
another process can create the same file concurrently using another NFS
client which might not yet see the file due to caching.
- see [
https://www.time-travellers.org/shane/papers/NFS_considered_harmful.html]
---
By curiosity, I would like to fix the incorrect text in message exceptionWhileReadingPack, but I think Matthias already did it[3].
[3] is available since JGit release v5.1.6.201903130242-r
Do you think it is worth to improve the existing error message handling mechanism? I have two ideas in mind:
- Remove unused messages. I found 62 of them
patches welcome, unused messages should be removed since there is no need to maintain them
and removing them will reduce size of jgit binary artefact
- Refactor the code to avoid similar placeholder problems, like this one [4], so that the relationship between arguments and pattern can be easier to see.
we are close to releasing final 5.5.0, so first of all I'd like to fix all known issues in error message texts
