Hi Matthias,
Thank you.
I've tried the fix -- and it works well. Orphaned files have been
removed, there are no errors in logs.
On 22.10.2020 13:26, Matthias Sohn wrote:
Can you try this fix ?
Hello,
We found a similar bug.
But now it relates to the GC.deleteOrphans() method.
This method also unable to remove orphaned files on
Windows if the files have R/O attribute set. The
produced error is the next:
===============================================================================
2020-10-20 01:39:21,294-0400 ERROR
[o.e.j.i.storage.file.GC]
E:\xxx\objects\pack\pack-d849246de612f3d29812703e482773ee5b1453c6.idx
java.nio.file.AccessDeniedException:
E:\xxx\objects\pack\pack-d849246de612f3d29812703e482773ee5b1453c6.idx
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at
sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at
org.eclipse.jgit.internal.storage.file.GC.deleteOrphans(GC.java:981)
at
org.eclipse.jgit.internal.storage.file.GC.repack(GC.java:882)
at
org.eclipse.jgit.internal.storage.file.GC.doGc(GC.java:267)
at
org.eclipse.jgit.internal.storage.file.GC.gc(GC.java:218)
[...]
===============================================================================
Thus if for any reasons some orphaned files remain in
the 'pack' folder -- they will remain here infinitely
and will flood the logs with the above error messages.
On 10.06.2020 3:56, Denis Malyshkin wrote:
Hello Matthias,
Thank you a lot.
I've tested with both JGit '5.8.0-SNAPSHOT'
and '5.8.0.202005061305-m2' – and I confirm
that the bug is not reproduced anymore.
On 09.06.2020 19:57, Matthias Sohn wrote:
Hello,
The simple example of code, mentioned in
the previous email.
class
Main {
private static File REPO_DIR = new
File("testRepo");
private static final String REPO_URL =
"https://github.com/denis-bigbrassband-com/testing.git";
public static final String
REF_SPEC_BARE =
"+refs/heads/*:refs/heads/*";
public static final String TAG_SPEC =
"+refs/tags/*:refs/tags/*";
public static void main(String[] args)
{
System.out.println("Start
cloning...");
CloneCommand cloneCommand = new
CloneCommand();
cloneCommand.setBare(true);
cloneCommand.setCloneAllBranches(true);
cloneCommand.setCloneSubmodules(true);
cloneCommand.setURI(REPO_URL);
cloneCommand.setDirectory(REPO_DIR);
cloneCommand.setTimeout(60);
try (Git git =
cloneCommand.call()) {
Repository repo =
git.getRepository();
System.out.println("Cloned.
Press Enter key when the repository is
changed...");
System.in.read();
System.out.println("Fetching...");
try (Transport tn =
Transport.open(repo, REPO_URL)) {
tn.setTimeout(60);
ProgressMonitor monitor =
new TextProgressMonitor();
tn.fetch(monitor,
Arrays.asList(new RefSpec(REF_SPEC_BARE),
new RefSpec(TAG_SPEC)));
}
System.out.println("Fetched.
Starting GC...");
GC gc = new
GC((FileRepository) repo);
PackConfig pConfig = new
PackConfig(repo);
pConfig.setBuildBitmaps(false); //
disable bitmap indexes
gc.setPackConfig(pConfig);
gc.setPackExpireAgeMillis(0);
gc.gc();
System.out.println("GC
completed!");
} catch (Exception e) {
e.printStackTrace();
}
}
}
Hello,
We found an issue with the JGit GC on
Windows.
I've attached a simple example to expose
this issue. Steps to reproduce:
- Clone some repository using JGit.
- Add commits to the repository on
the git server side (for example,
push commits from another computer).
- Fetch the cloned repository using
JGit.
- Call the JGit GC on the cloned
repository.
- After the GC the cloned
repository 'objects/pack' folder
contains all packs -- both existed
before the GC and created during the
GC.
- If you repeat steps 2-4 then new
pack files appears on every
iteration. And all old pack files
remain in the repository
'objects/pack' folder.
It seems this is a bug.
The logic of the bug is the following:
- The JGit clone & fetch
operations mark the created pack
files as 'read-only' in the ObjectDirectoryPackParser.parse
method:
- JGit GC operation also marks the
created pack files as 'read-only' in
the GC.writePack method:
- JGit uses Files.delete()
method in the GC.removeOldPack
to remove old pack files:
- This Files.delete()
method unable to remove 'read-only'
files on Windows (please see https://stackoverflow.com/questions/12139482/difference-between-filesdeletepath-and-filedelete),
throws an exception, but this
exception is silently ignored:
- As a result all pack files remain
in the repository, but nobody knows
that something goes wrong:
It seems that the R/O attribute
should be cleared before deleting old
pack files.
Use the nightly build 5.8.0-SNAPSHOT until
5.8 is released next Wednesday.
-Matthias
--
|
Best regards,
Denis Malyshkin,
Senior C++ Developer
ISS Art, LLC - custom software development company
Skype: dmalyshkin
Phone: +73812909808
|
IMPORTANT:
The contents of this email and any attachments are
confidential and intended for the named recipient(s) only. If
you have received this email by mistake, please notify the
sender immediately and delete it from your system. You may not
copy this email or disclose its contents to anyone. Thank you.
|