Hello,
We found a similar bug.
But now it relates to the GC.deleteOrphans() method.
This method also unable to remove orphaned files on Windows if the
files have R/O attribute set. The produced error is the next:
===============================================================================
2020-10-20 01:39:21,294-0400 ERROR [o.e.j.i.storage.file.GC]
E:\xxx\objects\pack\pack-d849246de612f3d29812703e482773ee5b1453c6.idx
java.nio.file.AccessDeniedException:
E:\xxx\objects\pack\pack-d849246de612f3d29812703e482773ee5b1453c6.idx
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at
sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at
org.eclipse.jgit.internal.storage.file.GC.deleteOrphans(GC.java:981)
at
org.eclipse.jgit.internal.storage.file.GC.repack(GC.java:882)
at org.eclipse.jgit.internal.storage.file.GC.doGc(GC.java:267)
at org.eclipse.jgit.internal.storage.file.GC.gc(GC.java:218)
[...]
===============================================================================
Thus if for any reasons some orphaned files remain in the 'pack'
folder -- they will remain here infinitely and will flood the logs
with the above error messages.
On 10.06.2020 3:56, Denis Malyshkin wrote:
Hello Matthias,
Thank you a lot.
I've tested with both JGit '5.8.0-SNAPSHOT' and '5.8.0.202005061305-m2'
– and I confirm that the bug is not reproduced anymore.
On 09.06.2020 19:57, Matthias Sohn wrote:
Hello,
The simple example of code, mentioned in the
previous email.
class Main {
private static File REPO_DIR = new
File("testRepo");
private static final String REPO_URL = "https://github.com/denis-bigbrassband-com/testing.git";
public static final String REF_SPEC_BARE =
"+refs/heads/*:refs/heads/*";
public static final String TAG_SPEC =
"+refs/tags/*:refs/tags/*";
public static void main(String[] args) {
System.out.println("Start cloning...");
CloneCommand cloneCommand = new
CloneCommand();
cloneCommand.setBare(true);
cloneCommand.setCloneAllBranches(true);
cloneCommand.setCloneSubmodules(true);
cloneCommand.setURI(REPO_URL);
cloneCommand.setDirectory(REPO_DIR);
cloneCommand.setTimeout(60);
try (Git git = cloneCommand.call()) {
Repository repo = git.getRepository();
System.out.println("Cloned. Press Enter
key when the repository is changed...");
System.in.read();
System.out.println("Fetching...");
try (Transport tn = Transport.open(repo,
REPO_URL)) {
tn.setTimeout(60);
ProgressMonitor monitor = new
TextProgressMonitor();
tn.fetch(monitor, Arrays.asList(new
RefSpec(REF_SPEC_BARE), new RefSpec(TAG_SPEC)));
}
System.out.println("Fetched. Starting
GC...");
GC gc = new GC((FileRepository) repo);
PackConfig pConfig = new
PackConfig(repo);
pConfig.setBuildBitmaps(false); //
disable bitmap indexes
gc.setPackConfig(pConfig);
gc.setPackExpireAgeMillis(0);
gc.gc();
System.out.println("GC completed!");
} catch (Exception e) {
e.printStackTrace();
}
}
}
Hello,
We found an issue with the JGit GC on Windows.
I've attached a simple example to expose this
issue. Steps to reproduce:
- Clone some repository using JGit.
- Add commits to the repository on the git
server side (for example, push commits from
another computer).
- Fetch the cloned repository using JGit.
- Call the JGit GC on the cloned repository.
- After the GC the cloned repository
'objects/pack' folder contains all packs --
both existed before the GC and created during
the GC.
- If you repeat steps 2-4 then new pack files
appears on every iteration. And all old pack
files remain in the repository 'objects/pack'
folder.
It seems this is a bug.
The logic of the bug is the following:
- The JGit clone & fetch operations mark
the created pack files as 'read-only' in the ObjectDirectoryPackParser.parse
method:
- JGit GC operation also marks the created
pack files as 'read-only' in the GC.writePack
method:
- JGit uses Files.delete() method in
the GC.removeOldPack to remove old
pack files:
- This Files.delete() method unable
to remove 'read-only' files on Windows (please
see https://stackoverflow.com/questions/12139482/difference-between-filesdeletepath-and-filedelete),
throws an exception, but this exception is
silently ignored:
- As a result all pack files remain in the
repository, but nobody knows that something
goes wrong:
It seems that the R/O attribute should be
cleared before deleting old pack files.
Use the nightly build 5.8.0-SNAPSHOT until 5.8 is
released next Wednesday.
-Matthias
--
|
Best regards,
Denis Malyshkin,
Senior C++ Developer
ISS Art, LLC - custom software development company
Skype: dmalyshkin
Phone: +73812909808
|
IMPORTANT:
The contents of this email and any attachments are
confidential and intended for the named recipient(s) only. If
you have received this email by mistake, please notify the
sender immediately and delete it from your system. You may not
copy this email or disclose its contents to anyone. Thank you.
_______________________________________________