Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit
Alright, here is the change:



From: Matthias Sohn <matthias.sohn@xxxxxxxxx>
To: R�diger Herrmann <ruediger.herrmann@xxxxxx>
Cc: "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 5:27 PM
Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

I meant push it for review to Gerrit, if we don't want to keep it as a test we can still abandon it

On Thu, Apr 7, 2016 at 5:16 PM, R�diger Herrmann <ruediger.herrmann@xxxxxx> wrote:
N.B. Note sure what you mean with' contribute'. While the test from the previous post is good enough to demonstrate the bug in JGit (if any), I don't think it would qualify for a _real_ test to be included in any test suite.




From: R�diger Herrmann <ruediger.herrmann@xxxxxx>
To: Matthias Sohn <matthias.sohn@xxxxxxxxx>
Cc: "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 4:59 PM

Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

Thanks for looking into this. The three methods shown below are all that is necessary to run in the GarbageCollectCommandTest.

>>> BEGIN

@Test
public void testPruneOldOrphanCommit() throws Exception {
StoredConfig config = git.getRepository().getConfig();
config.setString("gc", null, "prunePackExpire", "1.second.ago");
config.setString("gc", null, "pruneExpire", "1.second.ago");
config.save();
ObjectId initial = git.getRepository().resolve("HEAD");
RevCommit orphan = git.commit().setMessage("orphan").call();
changeLastModified(orphan, subtractDays(new Date(), 365));
RefUpdate refUpdate = git.getRepository()
.updateRef("refs/heads/master");
refUpdate.setNewObjectId(initial);
refUpdate.forceUpdate();
FileUtils.delete(new File(git.getRepository().getDirectory(), "logs"),
FileUtils.RECURSIVE | FileUtils.RETRY);

git.gc().setExpire(new Date()).call();
Thread.sleep(4000);
git.gc().setExpire(new Date()).call();

assertNull(git.getRepository().resolve(orphan.name()));
}

private static Date subtractDays(Date date, int days) {
Calendar calendar = Calendar.getInstance();
calendar.setTime(date);
calendar.add(Calendar.DAY_OF_MONTH, days * (-1));
return calendar.getTime();
}

private void changeLastModified(ObjectId commitId, Date date) {
File objectsDirectory = new File(git.getRepository().getDirectory(),
"objects");
File commitObjectDirectory = new File(objectsDirectory,
commitId.name().substring(0, 2));
File commitObjectFile = new File(commitObjectDirectory,
commitId.name().substring(2));
commitObjectFile.setLastModified(date.getTime());
}

<<< END


From: Matthias Sohn <matthias.sohn@xxxxxxxxx>
To: R�diger Herrmann <ruediger.herrmann@xxxxxx>
Cc: Shawn Pearce <spearce@xxxxxxxxxxx>; "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 4:32 PM
Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

can you contribute this test as a new test in GarbageCollectCommandTest ?
I'll ask Christian to have a look and investigate why this still doesn't work.

On Thu, Apr 7, 2016 at 4:21 PM, R�diger Herrmann <ruediger.herrmann@xxxxxx> wrote:
Thanks for your insights. So setting gc.prunePackExpire = 1.second.ago (or pruneExpire for JGit < 4.3) and running GC twice with some seconds delay should prune the unreferenced object, right?

I changed the code accordingly but the commit is still not garbage collected (note, I am running against HEAD)
@Test
public void testPruneOldOrphanCommit() throws Exception {
StoredConfig config = git.getRepository().getConfig();
config.setString("gc", null, "prunePackExpire", "1.second.ago");
config.setString("gc", null, "pruneExpire", "1.second.ago");
config.save();
ObjectId initial = git.getRepository().resolve("HEAD");
RevCommit orphan = git.commit().setMessage("orphan").call();
changeLastModified(orphan, subtractDays(new Date(), 365));
RefUpdate refUpdate = git.getRepository()
.updateRef("refs/heads/master");
refUpdate.setNewObjectId(initial);
refUpdate.forceUpdate();
FileUtils.delete(new File(git.getRepository().getDirectory(), "logs"),
FileUtils.RECURSIVE | FileUtils.RETRY);

git.gc().setExpire(new Date()).call();
Thread.sleep(4000);
git.gc().setExpire(new Date()).call();

assertNull(git.getRepository().resolve(orphan.name()));
}



From: Matthias Sohn <matthias.sohn@xxxxxxxxx>
To: R�diger Herrmann <ruediger.herrmann@xxxxxx>
Cc: Shawn Pearce <spearce@xxxxxxxxxxx>; "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 1:26 PM

Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

in order to avoid race conditions when a pack arrives while a gc is running jgit doesn't immediately prune
garbage objects but puts them in a garbage pack which is only pruned with the next gc after a grace period
defined by gc.prunePackExpire [1] which is 1 hour by default. Before [1] this reused gc.pruneExpire
which is 2 weeks by default which led to pack file explosion, this was fixed by [1] recently.
[1] will be contained in 4.3 which I am going to release today.

[1] https://git.eclipse.org/r/#/c/69628/

-Matthias

On Thu, Apr 7, 2016 at 10:23 AM, R�diger Herrmann <ruediger.herrmann@xxxxxx> wrote:
Thanks for the hint. I wasn't aware that commits are reflogged. However, if I delete the logs directory before collecting garbage, the orphan commit is still not pruned.

Here the updated test that deletes the reflog:

@Test
public void testPruneOldOrphanCommit() throws Exception {
ObjectId initial = git.getRepository().resolve("HEAD");
RevCommit orphan = git.commit().setMessage("orphan").call();
changeLastModified(orphan, subtractDays(new Date(), 365));
RefUpdate refUpdate = git.getRepository()
.updateRef("refs/heads/master");
refUpdate.setNewObjectId(initial);
refUpdate.forceUpdate();
FileUtils.delete(new File(git.getRepository().getDirectory(), "logs"),
FileUtils.RECURSIVE | FileUtils.RETRY);

git.gc().setExpire(new Date()).call();

assertNull(git.getRepository().resolve(orphan.name()));
}



From: Matthias Sohn <matthias.sohn@xxxxxxxxx>
To: Shawn Pearce <spearce@xxxxxxxxxxx>
Cc: R�diger Herrmann <ruediger.herrmann@xxxxxx>; "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 12:51 AM
Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

On Wed, Apr 6, 2016 at 11:19 PM, Shawn Pearce <spearce@xxxxxxxxxxx> wrote:
On Wed, Apr 6, 2016 at 2:14 PM, R�diger Herrmann
<ruediger.herrmann@xxxxxx> wrote:
> You are right of course, the shown code isn't quite what I tired. Sorry for
> the confusion. If I change it to 'unreference' the orphan commit the result
> is still the same.
>
> The correct version would reset HEAD to the previous commit so that 'orphan'
> is actually unreferenced like this:
>
> @Test
> public void testPruneOldOrphanCommit() throws Exception {
> ObjectId initial = git.getRepository().resolve("HEAD");
> RevCommit orphan = git.commit().setMessage("orphan").call();
> changeLastModified(orphan, subtractDays(new Date(), 365));
> RefUpdate refUpdate = git.getRepository()
> .updateRef("refs/heads/master");
> refUpdate.setNewObjectId(initial);
> refUpdate.forceUpdate();

Is orphan still in the reflog for master? or HEAD?

I tried your test and the orphaned commit is still referenced from both the
master's and HEAD's reflogs.

-Matthias 









_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jgit-dev





Back to the top