[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [jgit-dev] Implementing shallow clones
|
Ok, that makes way more sense with RevWalk always needing commits from
its own parse*() functions. Apparently I didn't RTFM closely enough.
With that in hand, the wrapping approach works fine. I've rewritten
the patch to take advantage of it, and it gets rid of a lot of the
grossness. I also managed to use the filter framework when generating
the pack, so no extra flags are needed. Attached is the new version
of the patch, which gets a lot closer to being checkin-worthy. It
still has a couple problems/bits of ugliness, though:
1) Lots of wrapper classes (DepthRevCommit, DepthWalk,
DepthObjectWalk, etc.) I know this is how we do most of this stuff in
Java, but some of these (especially the Walk classes) seem a little
silly for the amount of code that's in them. Would it be better to
just fold them into the base classes, or do you prefer to leave them
as separate subclasses?
2) StartGenerator makes use of a new DepthGenerator to tag the
commits, but it has to be told to include this generator. At the
moment, I just did this by adding a new setter on StartGenerator,
which gets the job done, but feels kind of ugly.
3) Most significantly, I'm still not sure I'm doing the depth tagging
stuff correctly. I'm inserting the DepthGenerator directly after the
PendingGenerator, which is correct as long as PendingGenerator always
produces topologically sorted commits. The right thing to do is to
put it after the TopoSortGenerator, but if I do that, things don't
work. This is because the FIFORevQueue tries to suck in the entire
commit chain right up front, before passing anything along to the
subsequent generators. If the DepthGenerator is later in the chain,
this means the entire tree gets added to the FIFORevQueue without
having been properly depth-tagged. However, since I've also
implemented a DepthFilter to cut the tree off at the right places, and
the filters are always run from PendingGenerator, the filter sees all
of the untagged commits, and does bad things with them. It seems like
for this to work, there would need to be some way to implement a
filter which doesn't perform its check until after the whole generator
chain has run. This is probably a consequence of the fact that until
now, none of the Generators actually modified their commits in ways
that the filters cared about. Do you have any thoughts on how to best
get around this?
Sorry to keep going back and forth on this. I think I'm pretty close,
though--as far as I can tell, it runs just fine, I just want to be
sure I understand the framework well enough that I'm actually
implementing it in the correct way. Thanks for all your help so far,
hopefully this can be put to bed fairly soon.
--Matt
On Tue, Jul 27, 2010 at 10:26 AM, Shawn O. Pearce <spearce@xxxxxxxxxxx> wrote:
> Matt Fischer <mattfischer84@xxxxxxxxx> wrote:
>>
>> The biggest problem I ran into is that I don't think wrapping the
>> RevCommits using the createCommit() function of RevWalk is going to
>> work.
>
> Sure it will. You mentioned that C Git does this by adding the
> depth into the "->util" field of the struct commit. The way we do
> the same thing in JGit is by overriding createCommit() on RevWalk
> and returning a subclass of RevCommit which has the additional data
> values we need. This gives us more flexibility as we can allocate
> any amount of storage we need inline with the object itself.
>
>> The reason is that you seed the RevWalk with start commits
>> which you get from elsewhere, so those commits haven't yet been
>> wrapped.
>
> Somewhat true.
>
> Our API permits the caller to pass us anything to markStart()
> and markUninteresting(). But the calling convention requires
> that the RevCommit passed *MUST* come from the same RevWalk.
> Well behaved callers already do that. If they are crossing
> RevWalk pools they pass a RevCommit from the one pool into
> the other pool's parseCommit() method, which will ensure the
> proper createCommit() implementation is used.
>
> If its really an issue to you, override markStart() to type
> check the input argument:
>
> class ShallowWalk extends RevWalk {
> @Override
> public void markStart(RevCommit c) {
> if (!(c instanceof ShallowCommit))
> throw new IllegalArgumentException("Only ShallowCommit supported");
> super.markStart(c);
> }
>
> @Override
> protected ShallowCommit createCommit(AnyObjectId id) {
> return new ShallowCommit(id);
> }
> }
>
> class ShallowCommit extends RevCommit {
> int depth = -1;
>
> ShallowCommit(AnyObjectId id) {
> super(id);
> }
>
> @Override
> public void reset() {
> super.reset();
> depth = -1;
> }
> }
>
>> You could assume that all such commits would have depth 0,
>> and treat them as such if they aren't wrapped, but these commits may
>> have been parsed already, meaning their parents have been constructed,
>> and those haven't been wrapped either.
>
> You can't assume depth of 0 just because it was passed to
> markStart(). We only want a depth of 0 on things that were directly
> reachable from a ref. So the caller probably needs to denote for
> us which commits were at depth 0.
>
>> RevCommit, and having its setter function automatically propagate the
>> depth down to any parents which have already been parsed. This is
>> hacky and ugly, but at least it illustrates the pattern that will have
>> to exist in a correct implementation.
>
> True. This is similar to the carryFlags code which dives down into
> the parents when they are present. Note that we take an optimization
> for a string-of-pearls here by avoiding recursion when there is
> only one parent. This helps to keep our stack from exploding on
> really large histories... but we still run a risk of stack overflow.
>
> Its cleaner to just set the depth as the commits come out of the topo
> sort generator. We don't have to worry about stack depth, because
> we just take this commit's depth and set it into every parent.
> By defaulting the depth to -1 we'll promote to the smallest depth
> if it hasn't been set yet:
>
> for (RevCommit p : c.parents)
> p.depth = Math.min(c.depth, p.depth);
>
> That's a pretty simple pass to insert in front. You may not
> even need to mess around with the StartGenerator. Though a
> RevSort.REVERSE in front of this pass would totally confuse the
> results since parents would emit before their children. So we
> might still need to inject this pass into StartGenerator anyway.
>
>> As far as getting the pack built correctly, I went ahead with my
>> previous proposal to add a SHALLOW flag to RevWalk,
>
> I'm pretty against adding new reserved flags. Those bits are
> really precious. We only have 32 of them, and right now RevWalk
> reserves 6. That leaves applications only 26 bits for their own use.
> There are some algorithms like `git branch --contains` that benefit
> from having a lot of bits available to them.
>
> I am however OK with a ShallowWalk that extends RevWalk with just
> doing its own flag reservation in its constructor:
>
> final RevFlag SHALLOW = newFlag("SHALLOW");
>
> The generator would just need to pull this flag off the ShallowWalk
> instance to find out what bit it was assigned. That way we only
> lose the bit in a shallow walker, which is a lot less likely to
> need a lot of application specific flag bits.
>
> --
> Shawn.
>
From 335b9688bed8da8bc74548ab971c7d74fd34fa96 Mon Sep 17 00:00:00 2001
From: Matt Fischer <matt.fischer@xxxxxxxxxx>
Date: Mon, 26 Jul 2010 22:39:37 -0500
Subject: [PATCH] Implemented shallow clones
This implements the server side of shallow clones only (i.e. git-upload-pack),
not the client side.
---
.../src/org/eclipse/jgit/lib/PackWriter.java | 40 ++++++++-
.../org/eclipse/jgit/revwalk/DepthGenerator.java | 91 ++++++++++++++++++++
.../org/eclipse/jgit/revwalk/DepthObjectWalk.java | 71 +++++++++++++++
.../org/eclipse/jgit/revwalk/DepthRevCommit.java | 79 +++++++++++++++++
.../src/org/eclipse/jgit/revwalk/DepthWalk.java | 71 +++++++++++++++
.../org/eclipse/jgit/revwalk/StartGenerator.java | 9 ++
.../eclipse/jgit/revwalk/filter/DepthFilter.java | 81 +++++++++++++++++
.../src/org/eclipse/jgit/transport/UploadPack.java | 82 +++++++++++++++++-
8 files changed, 517 insertions(+), 7 deletions(-)
create mode 100644 org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthGenerator.java
create mode 100644 org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthObjectWalk.java
create mode 100644 org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthRevCommit.java
create mode 100644 org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthWalk.java
create mode 100644 org.eclipse.jgit/src/org/eclipse/jgit/revwalk/filter/DepthFilter.java
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/lib/PackWriter.java b/org.eclipse.jgit/src/org/eclipse/jgit/lib/PackWriter.java
index 48f41a5..3e8ea01 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/lib/PackWriter.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/lib/PackWriter.java
@@ -57,10 +57,13 @@
import org.eclipse.jgit.JGitText;
import org.eclipse.jgit.errors.IncorrectObjectTypeException;
import org.eclipse.jgit.errors.MissingObjectException;
+import org.eclipse.jgit.revwalk.DepthObjectWalk;
import org.eclipse.jgit.revwalk.ObjectWalk;
+import org.eclipse.jgit.revwalk.DepthRevCommit;
import org.eclipse.jgit.revwalk.RevFlag;
import org.eclipse.jgit.revwalk.RevObject;
import org.eclipse.jgit.revwalk.RevSort;
+import org.eclipse.jgit.revwalk.filter.DepthFilter;
import org.eclipse.jgit.transport.PackedObjectInfo;
import org.eclipse.jgit.util.NB;
@@ -488,19 +491,33 @@ public void preparePack(final Iterator<RevObject> objectsSource)
* @param uninterestingObjects
* collection of objects to be marked as uninteresting (end
* points of graph traversal).
+ * @param depth Depth
* @throws IOException
* when some I/O problem occur during reading objects.
*/
public void preparePack(
final Collection<? extends ObjectId> interestingObjects,
- final Collection<? extends ObjectId> uninterestingObjects)
+ final Collection<? extends ObjectId> uninterestingObjects,
+ int depth)
throws IOException {
ObjectWalk walker = setUpWalker(interestingObjects,
- uninterestingObjects);
+ uninterestingObjects, depth);
findObjectsToPack(walker);
}
/**
+ * @param interestingObjects
+ * @param uninterestingObjects
+ * @throws IOException
+ */
+ public void preparePack(
+ final Collection<? extends ObjectId> interestingObjects,
+ final Collection<? extends ObjectId> uninterestingObjects)
+ throws IOException {
+ preparePack(interestingObjects, uninterestingObjects, 0);
+ }
+
+ /**
* Determine if the pack file will contain the requested object.
*
* @param id
@@ -814,18 +831,30 @@ private void writeChecksum() throws IOException {
private ObjectWalk setUpWalker(
final Collection<? extends ObjectId> interestingObjects,
- final Collection<? extends ObjectId> uninterestingObjects)
+ final Collection<? extends ObjectId> uninterestingObjects,
+ int depth)
throws MissingObjectException, IOException,
IncorrectObjectTypeException {
- final ObjectWalk walker = new ObjectWalk(db);
+ final ObjectWalk walker;
+ if (depth == 0) {
+ walker = new ObjectWalk(db);
+ } else {
+ walker = new DepthObjectWalk(db);
+ walker.setRevFilter(new DepthFilter(depth));
+ }
+
walker.setRetainBody(false);
walker.sort(RevSort.TOPO);
walker.sort(RevSort.COMMIT_TIME_DESC, true);
if (thin)
walker.sort(RevSort.BOUNDARY, true);
-
for (ObjectId id : interestingObjects) {
RevObject o = walker.parseAny(id);
+
+ // Mark this as a root
+ if(depth != 0) {
+ ((DepthRevCommit)o).setDepth(0);
+ }
walker.markStart(o);
}
if (uninterestingObjects != null) {
@@ -841,6 +870,7 @@ private ObjectWalk setUpWalker(
walker.markUninteresting(o);
}
}
+
return walker;
}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthGenerator.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthGenerator.java
new file mode 100644
index 0000000..b5992b4
--- /dev/null
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthGenerator.java
@@ -0,0 +1,91 @@
+/*
+ * Copyright (C) 2010, Garmin International
+ * Copyright (C) 2010, Matt Fischer <matt.fischer@xxxxxxxxxx>
+ * and other copyright owners as documented in the project's IP log.
+ *
+ * This program and the accompanying materials are made available
+ * under the terms of the Eclipse Distribution License v1.0 which
+ * accompanies this distribution, is reproduced below, and is
+ * available at http://www.eclipse.org/org/documents/edl-v10.php
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.eclipse.jgit.revwalk;
+
+import java.io.IOException;
+
+import org.eclipse.jgit.errors.IncorrectObjectTypeException;
+import org.eclipse.jgit.errors.MissingObjectException;
+
+/**
+ * Tag all incoming commits with a Depth tag. This tag indicates
+ * the number of commits between this commit and the closest root.
+ */
+public class DepthGenerator extends Generator {
+ Generator pending;
+
+ /**
+ * @param s
+ */
+ public DepthGenerator(Generator s) {
+ pending = s;
+ }
+
+ @Override
+ int outputType() {
+ return pending.outputType();
+ }
+
+ @Override
+ RevCommit next() throws MissingObjectException,
+ IncorrectObjectTypeException, IOException {
+ RevCommit c = pending.next();
+
+ if (c == null) {
+ return null;
+ }
+
+ DepthRevCommit cd = (DepthRevCommit)c;
+
+ for (RevCommit p : cd.parents) {
+ DepthRevCommit pd = (DepthRevCommit)p;
+
+ pd.setDepth(Math.min(pd.getDepth(), cd.getDepth() + 1));
+ }
+
+ return c;
+ }
+
+}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthObjectWalk.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthObjectWalk.java
new file mode 100644
index 0000000..df6649e
--- /dev/null
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthObjectWalk.java
@@ -0,0 +1,71 @@
+/*
+ * Copyright (C) 2010, Garmin International
+ * Copyright (C) 2010, Matt Fischer <matt.fischer@xxxxxxxxxx>
+ * and other copyright owners as documented in the project's IP log.
+ *
+ * This program and the accompanying materials are made available
+ * under the terms of the Eclipse Distribution License v1.0 which
+ * accompanies this distribution, is reproduced below, and is
+ * available at http://www.eclipse.org/org/documents/edl-v10.php
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+
+package org.eclipse.jgit.revwalk;
+
+import org.eclipse.jgit.lib.AnyObjectId;
+import org.eclipse.jgit.lib.Repository;
+
+/**
+ * Subclass of ObjectWalk which tags all commits with the depth
+ * to their nearest root.
+ *
+ */
+public class DepthObjectWalk extends ObjectWalk {
+
+ /**
+ * @param repo
+ * the repository the walker will obtain data from.
+ */
+ public DepthObjectWalk(Repository repo) {
+ super(repo);
+ ((StartGenerator)pending).setGenerateDepth(true);
+ }
+
+ @Override
+ protected RevCommit createCommit(final AnyObjectId id) {
+ return new DepthRevCommit(id);
+ }
+}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthRevCommit.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthRevCommit.java
new file mode 100644
index 0000000..a7afd56
--- /dev/null
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthRevCommit.java
@@ -0,0 +1,79 @@
+/*
+ * Copyright (C) 2010, Garmin International
+ * Copyright (C) 2010, Matt Fischer <matt.fischer@xxxxxxxxxx>
+ * and other copyright owners as documented in the project's IP log.
+ *
+ * This program and the accompanying materials are made available
+ * under the terms of the Eclipse Distribution License v1.0 which
+ * accompanies this distribution, is reproduced below, and is
+ * available at http://www.eclipse.org/org/documents/edl-v10.php
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.eclipse.jgit.revwalk;
+
+import org.eclipse.jgit.lib.AnyObjectId;
+
+/**
+ * Wrapper object for RevCommit which adds a depth tag.
+ *
+ */
+public class DepthRevCommit extends RevCommit {
+
+ private int depth;
+
+ /**
+ * @param id
+ */
+ protected DepthRevCommit(AnyObjectId id) {
+ super(id);
+ depth = Integer.MAX_VALUE;
+ }
+
+ /**
+ * Set the depth of this commit
+ * @param depth Distance to nearest root in commit graph
+ */
+ public void setDepth(int depth) {
+ this.depth = depth;
+ }
+
+ /**
+ * @return The depth of this commit
+ */
+ public int getDepth() {
+ return depth;
+ }
+}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthWalk.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthWalk.java
new file mode 100644
index 0000000..7faaa8d
--- /dev/null
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/DepthWalk.java
@@ -0,0 +1,71 @@
+/*
+ * Copyright (C) 2010, Garmin International
+ * Copyright (C) 2010, Matt Fischer <matt.fischer@xxxxxxxxxx>
+ * and other copyright owners as documented in the project's IP log.
+ *
+ * This program and the accompanying materials are made available
+ * under the terms of the Eclipse Distribution License v1.0 which
+ * accompanies this distribution, is reproduced below, and is
+ * available at http://www.eclipse.org/org/documents/edl-v10.php
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.eclipse.jgit.revwalk;
+
+import org.eclipse.jgit.lib.AnyObjectId;
+import org.eclipse.jgit.lib.Repository;
+
+/**
+ * Subclass of RevWalk which tags all commits with the depth
+ * to their nearest root.
+ *
+ */
+public class DepthWalk extends RevWalk {
+
+ /**
+ * @param repo
+ * the repository the walker will obtain data from.
+ */
+ public DepthWalk(Repository repo) {
+ super(repo);
+
+ ((StartGenerator)pending).setGenerateDepth(true);
+ }
+
+ @Override
+ protected RevCommit createCommit(final AnyObjectId id) {
+ return new DepthRevCommit(id);
+ }
+}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/StartGenerator.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/StartGenerator.java
index 5e778a4..6c8a109 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/StartGenerator.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/StartGenerator.java
@@ -65,9 +65,11 @@
*/
class StartGenerator extends Generator {
private final RevWalk walker;
+ private boolean generateDepth;
StartGenerator(final RevWalk w) {
walker = w;
+ generateDepth = false;
}
@Override
@@ -140,6 +142,9 @@ RevCommit next() throws MissingObjectException,
((PendingGenerator) g).canDispose = false;
}
+ if (generateDepth)
+ g = new DepthGenerator(g);
+
if ((g.outputType() & NEEDS_REWRITE) != 0) {
// Correction for an upstream NEEDS_REWRITE is to buffer
// fully and then apply a rewrite generator that can
@@ -172,4 +177,8 @@ else if (uninteresting) {
w.pending = g;
return g.next();
}
+
+ public void setGenerateDepth(boolean generateDepth) {
+ this.generateDepth = generateDepth;
+ }
}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/filter/DepthFilter.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/filter/DepthFilter.java
new file mode 100644
index 0000000..a8dfc83
--- /dev/null
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/filter/DepthFilter.java
@@ -0,0 +1,81 @@
+/*
+ * Copyright (C) 2010, Garmin International
+ * Copyright (C) 2010, Matt Fischer <matt.fischer@xxxxxxxxxx>
+ * and other copyright owners as documented in the project's IP log.
+ *
+ * This program and the accompanying materials are made available
+ * under the terms of the Eclipse Distribution License v1.0 which
+ * accompanies this distribution, is reproduced below, and is
+ * available at http://www.eclipse.org/org/documents/edl-v10.php
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Eclipse Foundation, Inc. nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.eclipse.jgit.revwalk.filter;
+
+import java.io.IOException;
+
+import org.eclipse.jgit.errors.IncorrectObjectTypeException;
+import org.eclipse.jgit.errors.MissingObjectException;
+import org.eclipse.jgit.errors.StopWalkException;
+import org.eclipse.jgit.revwalk.RevCommit;
+import org.eclipse.jgit.revwalk.DepthRevCommit;
+import org.eclipse.jgit.revwalk.RevWalk;
+
+/**
+ * Includes a commit only if it is less than the specified depth
+ */
+public class DepthFilter extends RevFilter {
+ private int depth;
+
+ /**
+ * @param depth
+ */
+ public DepthFilter(int depth) {
+ this.depth = depth;
+ }
+
+ @Override
+ public boolean include(RevWalk walker, RevCommit cmit)
+ throws StopWalkException, MissingObjectException,
+ IncorrectObjectTypeException, IOException {
+ return ((DepthRevCommit)cmit).getDepth() < depth;
+ }
+
+ @Override
+ public RevFilter clone() {
+ return this;
+ }
+
+}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java b/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java
index 77cc1a6..f079ed4 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java
@@ -63,12 +63,15 @@
import org.eclipse.jgit.lib.ProgressMonitor;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
+import org.eclipse.jgit.revwalk.DepthWalk;
import org.eclipse.jgit.revwalk.RevCommit;
+import org.eclipse.jgit.revwalk.DepthRevCommit;
import org.eclipse.jgit.revwalk.RevFlag;
import org.eclipse.jgit.revwalk.RevFlagSet;
import org.eclipse.jgit.revwalk.RevObject;
import org.eclipse.jgit.revwalk.RevTag;
import org.eclipse.jgit.revwalk.RevWalk;
+import org.eclipse.jgit.revwalk.filter.DepthFilter;
import org.eclipse.jgit.transport.BasePackFetchConnection.MultiAck;
import org.eclipse.jgit.transport.RefAdvertiser.PacketLineOutRefAdvertiser;
import org.eclipse.jgit.util.io.InterruptTimer;
@@ -146,6 +149,12 @@
/** Objects on both sides, these don't have to be sent. */
private final List<RevObject> commonBase = new ArrayList<RevObject>();
+ /** Shallow commits that we will be sending. */
+ private final List<RevCommit> localShallowCommits = new ArrayList<RevCommit>();
+
+ /** Shallow commits the remote already has. */
+ private final List<RevCommit> remoteShallowCommits = new ArrayList<RevCommit>();
+
/** null if {@link #commonBase} should be examined again. */
private Boolean okToGiveUp;
@@ -165,6 +174,8 @@
private MultiAck multiAck = MultiAck.OFF;
+ private int depth = 0;
+
/**
* Create a new pack upload for an open repository.
*
@@ -330,8 +341,64 @@ else if (options.contains(OPTION_MULTI_ACK))
else
multiAck = MultiAck.OFF;
- if (negotiate())
+ if (depth != 0) {
+ processShallow();
+ }
+
+ if (negotiate()) {
sendPack();
+ }
+ }
+
+ private void processShallow() throws IOException {
+ RevWalk walk = new DepthWalk(db);
+
+ // Find all the commits which will be shallow
+ walk.setRevFilter(new DepthFilter(depth));
+ for (RevCommit c : wantCommits) {
+ RevCommit commit = walk.parseCommit(c);
+ ((DepthRevCommit)commit).setDepth(0);
+ walk.markStart(commit);
+ }
+
+ RevObject o;
+
+ // Find all commits at the boundary, and mark them as shallow
+ while ((o = walk.next()) != null) {
+ if (((DepthRevCommit)o).getDepth() != depth - 1)
+ continue;
+
+ localShallowCommits.add((RevCommit)o);
+
+ boolean found = false;
+ for (RevCommit c : remoteShallowCommits) {
+ if (c.name() == o.name()) {
+ found = true;
+ break;
+ }
+ }
+
+ if (!found) {
+ pckOut.writeString("shallow " + o.name());
+ }
+ }
+
+ // Unshallow any commits which we're expanding on
+ for (RevCommit remote : remoteShallowCommits) {
+ boolean found = false;
+ for(RevCommit local : localShallowCommits) {
+ if(remote.name() == local.name()) {
+ found = true;
+ break;
+ }
+ }
+
+ if (!found) {
+ pckOut.writeString("unshallow " + remote.name());
+ }
+ }
+
+ pckOut.end();
}
/**
@@ -354,6 +421,7 @@ public void sendAdvertisedRefs(final RefAdvertiser adv) throws IOException {
adv.advertiseCapability(OPTION_NO_PROGRESS);
adv.setDerefTags(true);
refs = refFilter.filter(db.getAllRefs());
+
adv.send(refs);
adv.end();
}
@@ -372,6 +440,16 @@ private void recvWants() throws IOException {
if (line == PacketLineIn.END)
break;
+
+ if (line.startsWith("deepen ")) {
+ depth = Integer.parseInt(line.substring(7));
+ continue;
+ }
+ if (line.startsWith("shallow ")) {
+ final ObjectId id = ObjectId.fromString(line.substring(8));
+ remoteShallowCommits.add(walk.parseCommit(id));
+ continue;
+ }
if (!line.startsWith("want ") || line.length() < 45)
throw new PackProtocolException(MessageFormat.format(JGitText.get().expectedGot, "want", line));
@@ -568,7 +646,7 @@ private void sendPack() throws IOException {
pw = new PackWriter(db, pm, NullProgressMonitor.INSTANCE);
pw.setDeltaBaseAsOffset(options.contains(OPTION_OFS_DELTA));
pw.setThin(thin);
- pw.preparePack(wantAll, commonBase);
+ pw.preparePack(wantAll, commonBase, depth);
if (options.contains(OPTION_INCLUDE_TAG)) {
for (final Ref r : refs.values()) {
final RevObject o;
--
1.6.0.6
