Re: [jgit-dev] JGit corrupts blobs or hangs forever

[Roberto Tyley <roberto.tyley@xxxxxxxxx>]

Could you specify which JVM you're using? Just want to check it isn't related to this old issue on Harmony:

https://issues.apache.org/jira/browse/HARMONY-6637

Roberto

On 24 November 2011 22:20, Dmitry Pavlenko <dmit10@xxxxxxxxx> wrote:

Hello all!

I've encountered into the problem when using JGit. It happens if
 * the blob is packed
 * the blob is kept as delta within the pack
 * there's no corresponding loose object for the blob (so JGit is forced to read the pack file)

With some older JGit version the test fails on comparison of expected and actual values at i=8192.
With the latest JGit master the test below hangs forever with the stack:

 at java.util.zip.Deflater.deflateBytes(Deflater.java:-1)
         at java.util.zip.Deflater.deflate(Deflater.java:306)
         at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)
         at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118)
         at org.eclipse.jgit.util.io.TeeInputStream.read(TeeInputStream.java:113)
         at org.eclipse.jgit.util.io.TeeInputStream.read(TeeInputStream.java:87)
         at org.eclipse.jgit.lib.ObjectStream$Filter.read(ObjectStream.java:204)

My Git version is 1.7.7.3, from the Debian wheezy/sid package.
Maybe with your version Git will use another packing strategy and the test will fail on reading blob1Id not blob2Id as for me.

       @Test
       public void testLargeDeltaPackedBlobsJGit() throws Exception {
               final File directory = createEmptyDirectory();

               final Repository gitRepository = new FileRepository(directory);
               gitRepository.create(true);

               //create a buffer that could be poorly compressed
               final int bufferSize = 40000000; // ~40 Mb
               final byte[] buffer = new byte[bufferSize];

               for (int i = 0; i < bufferSize; i++) {
                       buffer[i] = (byte) (i*i*i);
               }

               ObjectInserter objectInserter;

               //create a blob with this contents
               final ObjectId blob1Id;
               objectInserter = gitRepository.newObjectInserter();
               try {
                       blob1Id = objectInserter.insert(Constants.OBJ_BLOB, buffer);
               }
               finally {
                       objectInserter.release();
               }

               //slightly modify the buffer to force Git use delta-compression
               for (int i = 0; i < 3; i++) {
                       buffer[i] = 0;
               }

               //create a blob with this contents
               final ObjectId blob2Id;
               objectInserter = gitRepository.newObjectInserter();
               try {
                       blob2Id = objectInserter.insert(Constants.OBJ_BLOB, buffer);
               }
               finally {
                       objectInserter.release();
               }

               //create a tag for blob 1 to survive after pruning
               final RefUpdate refUpdateBlob1 = gitRepository.updateRef("refs/tags/blob1");
               refUpdateBlob1.setNewObjectId(blob1Id);
               refUpdateBlob1.forceUpdate();

               //create a tag for blob 2 to survive after pruning
               final RefUpdate refUpdateBlob2 = gitRepository.updateRef("refs/tags/blob2");
               refUpdateBlob2.setNewObjectId(blob2Id);
               refUpdateBlob2.forceUpdate();

               //run "git gc" + "git prune" externally to create a pack without loose objects

               final Process gitGcProcess = new ProcessBuilder("git", "gc").directory(directory).start();
               gitGcProcess.waitFor();

               final Process gitPruneProcess = new ProcessBuilder("git", "prune").directory(directory).start();
               gitPruneProcess.waitFor();

               //let's read blob 2 (with another Git version maybe the problem will appear while reading blob 1)
               final ObjectLoader objectLoader = gitRepository.open(blob2Id);
               final ObjectStream inputStream = objectLoader.openStream();
               try {
                       int i = 0;
                       while (true) {
                               final int b = inputStream.read();
                               if (b == -1) {
                                       return;
                               }

                               final byte value = (byte) (b & 0xFF);
                               if (i > 3) {
                                       final byte expected = (byte) (i * i * i);
                                       final byte actual = value;
                                       if (expected != actual) {
                                               Assert.fail("Expected = " + expected + ", actual=" + actual + ", i=" + i);
                                       }
                               }
                               i++;
                       }
               }
               finally {
                       inputStream.close();
               }
       }
_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jgit-dev