[jgit-dev] JGit corrupts blobs or hangs forever

[Dmitry Pavlenko <dmit10@xxxxxxxxx>]

Hello all!

I've encountered into the problem when using JGit. It happens if
 * the blob is packed
 * the blob is kept as delta within the pack
 * there's no corresponding loose object for the blob (so JGit is forced to read the pack file)

With some older JGit version the test fails on comparison of expected and actual values at i=8192.
With the latest JGit master the test below hangs forever with the stack:

  at java.util.zip.Deflater.deflateBytes(Deflater.java:-1)
	  at java.util.zip.Deflater.deflate(Deflater.java:306)
	  at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)
	  at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118)
	  at org.eclipse.jgit.util.io.TeeInputStream.read(TeeInputStream.java:113)
	  at org.eclipse.jgit.util.io.TeeInputStream.read(TeeInputStream.java:87)
	  at org.eclipse.jgit.lib.ObjectStream$Filter.read(ObjectStream.java:204)

My Git version is 1.7.7.3, from the Debian wheezy/sid package.
Maybe with your version Git will use another packing strategy and the test will fail on reading blob1Id not blob2Id as for me.

	@Test
	public void testLargeDeltaPackedBlobsJGit() throws Exception {
		final File directory = createEmptyDirectory();

		final Repository gitRepository = new FileRepository(directory);
		gitRepository.create(true);

		//create a buffer that could be poorly compressed
		final int bufferSize = 40000000; // ~40 Mb
		final byte[] buffer = new byte[bufferSize];

		for (int i = 0; i < bufferSize; i++) {
			buffer[i] = (byte) (i*i*i);
		}

		ObjectInserter objectInserter;

		//create a blob with this contents
		final ObjectId blob1Id;
		objectInserter = gitRepository.newObjectInserter();
		try {
			blob1Id = objectInserter.insert(Constants.OBJ_BLOB, buffer);
		}
		finally {
			objectInserter.release();
		}

		//slightly modify the buffer to force Git use delta-compression
		for (int i = 0; i < 3; i++) {
			buffer[i] = 0;
		}

		//create a blob with this contents
		final ObjectId blob2Id;
		objectInserter = gitRepository.newObjectInserter();
		try {
			blob2Id = objectInserter.insert(Constants.OBJ_BLOB, buffer);
		}
		finally {
			objectInserter.release();
		}

		//create a tag for blob 1 to survive after pruning
		final RefUpdate refUpdateBlob1 = gitRepository.updateRef("refs/tags/blob1");
		refUpdateBlob1.setNewObjectId(blob1Id);
		refUpdateBlob1.forceUpdate();

		//create a tag for blob 2 to survive after pruning
		final RefUpdate refUpdateBlob2 = gitRepository.updateRef("refs/tags/blob2");
		refUpdateBlob2.setNewObjectId(blob2Id);
		refUpdateBlob2.forceUpdate();

		//run "git gc" + "git prune" externally to create a pack without loose objects

		final Process gitGcProcess = new ProcessBuilder("git", "gc").directory(directory).start();
		gitGcProcess.waitFor();

		final Process gitPruneProcess = new ProcessBuilder("git", "prune").directory(directory).start();
		gitPruneProcess.waitFor();

		//let's read blob 2 (with another Git version maybe the problem will appear while reading blob 1)
		final ObjectLoader objectLoader = gitRepository.open(blob2Id);
		final ObjectStream inputStream = objectLoader.openStream();
		try {
			int i = 0;
			while (true) {
				final int b = inputStream.read();
				if (b == -1) {
					return;
				}

				final byte value = (byte) (b & 0xFF);
				if (i > 3) {
					final byte expected = (byte) (i * i * i);
					final byte actual = value;
					if (expected != actual) {
						Assert.fail("Expected = " + expected + ", actual=" + actual + ", i=" + i);
					}
				}
				i++;
			}
		}
		finally {
			inputStream.close();
		}
	}