Re: [jgit-dev] NegativeArraySizeException in PackIndexV2

[Marc Strapetz <marc.strapetz@xxxxxxxxxxx>]

I've pushed a small patch, addressing both problems:

https://git.eclipse.org/r/#/c/31501/

-Marc

On 13.08.2014 03:42, Alex Blewitt wrote:
> See also 
> http://stackoverflow.com/questions/3038392/do-java-arrays-have-a-maximum-size
> 
> Which suggests that MAX_VALUE might be the wrong test here. 
> 
> Alex 
> 
> Sent from my iPhone 5
> 
>> On 12 Aug 2014, at 14:34, Robin Rosenberg <robin.rosenberg.lists@xxxxxxxxxx> wrote:
>>
>>
>> I missed, since there is an explicit overflow check that seems ok, I'd guess corruption now.
>>
>> -- robin
>>
>> ----- Ursprungligt meddelande -----
>>> Från: "Marc Strapetz" <marc.strapetz@xxxxxxxxxxx>
>>> Till: "Robin Rosenberg" <robin.rosenberg.lists@xxxxxxxxxx>
>>> Kopia: jgit-dev@xxxxxxxxxxx
>>> Skickat: tisdag, 12 aug 2014 15:09:27
>>> Ämne: Re: [jgit-dev] NegativeArraySizeException in PackIndexV2
>>>
>>>> More than 107 million objects. That would cause this behavior. That would
>>>> imply a idx file of nearly 4 GB (36 bytes per object). See
>>>> https://www.kernel.org/pub/software/scm/git/docs/technical/pack-format.txt
>>>>
>>>> By contrast, the linux kernel repo idx file contains only 2.6 M objects and
>>>> occupies ~73 MB on disk.
>>>>
>>>> Corruption is a possibility. C Git would probably choke on the file if it
>>>> was corrupted.
>>>> The index file can be regenerated from the pack using git index-pack.
>>>
>>> Thanks, Robin. For now, I'll try to distinguish between both cases and
>>> throw different Exceptions here; once we get a new bug report, I'll come
>>> back with additional information.
>>>
>>> -Marc
>>>
>>>
>>>
>>>> On 12.08.2014 14:57, Robin Rosenberg wrote:
>>>>
>>>>
>>>> ----- Ursprungligt meddelande -----
>>>>> Från: "Marc Strapetz" <marc.strapetz@xxxxxxxxxxx>
>>>>> Till: "Robin Rosenberg" <robin.rosenberg.lists@xxxxxxxxxx>
>>>>> Kopia: jgit-dev@xxxxxxxxxxx
>>>>> Skickat: tisdag, 12 aug 2014 13:24:19
>>>>> Ämne: Re: [jgit-dev] NegativeArraySizeException in PackIndexV2
>>>>>
>>>>>> Is this a ridiculously large pack?
>>>>>
>>>>> Unfortunately I don't have any more information than the pure
>>>>> stacktrace. How large would be too large? Could the fanoutTable be
>>>>> corrupted, so bucketCount < 0?
>>>>
>>>> More than 107 million objects. That would cause this behavior. That would
>>>> imply a idx file of nearly 4 GB (36 bytes per object). See
>>>> https://www.kernel.org/pub/software/scm/git/docs/technical/pack-format.txt
>>>>
>>>> By contrast, the linux kernel repo idx file contains only 2.6 M objects and
>>>> occupies ~73 MB on disk.
>>>>
>>>> Corruption is a possibility. C Git would probably choke on the file if it
>>>> was corrupted.
>>>> The index file can be regenerated from the pack using git index-pack.
>>>>
>>>> -- robin
>>>>
>>>>> -Marc
>>>>>
>>>>>> On 12.08.2014 11:28, Robin Rosenberg wrote:
>>>>>>
>>>>>>
>>>>>> ----- Ursprungligt meddelande -----
>>>>>>> Från: "Marc Strapetz" <marc.strapetz@xxxxxxxxxxx>
>>>>>>> Till: jgit-dev@xxxxxxxxxxx
>>>>>>> Skickat: tisdag, 12 aug 2014 10:50:22
>>>>>>> Ämne: [jgit-dev] NegativeArraySizeException in PackIndexV2
>>>>>>>
>>>>>>> Any ideas regarding this error? Our jgit is based on 3.3.x.
>>>>>>>
>>>>>>> java.lang.NegativeArraySizeException
>>>>>>>    at
>>>>>>> org.eclipse.jgit.internal.storage.file.PackIndexV2.<init>(PackIndexV2.java:123)
>>>>>>>    at
>>>>>>> org.eclipse.jgit.internal.storage.file.PackIndex.read(PackIndex.java:137)
>>>>>>>    at
>>>>>>>    org.eclipse.jgit.internal.storage.file.PackIndex.open(PackIndex.java:96)
>>>>>>
>>>>>> Is this a ridiculously large pack?
>>>>>>
>>>>>> -- robin
>>>>>>
>>>>>>>
>>>>>>> The offending code is:
>>>>>>>
>>>>>>> final long nameLen = bucketCnt * Constants.OBJECT_ID_LENGTH;
>>>>>>> if (nameLen > Integer.MAX_VALUE)
>>>>>>>  throw new IOException(JGitText.get().indexFileIsTooLargeForJgit);
>>>>>>>
>>>>>>> final int intNameLen = (int) nameLen;
>>>>>>>>>> final byte[] raw = new byte[intNameLen];
>>>>>>>
>>>>>>> -Marc
>>>>>>> _______________________________________________
>>>>>>> jgit-dev mailing list
>>>>>>> jgit-dev@xxxxxxxxxxx
>>>>>>> To change your delivery options, retrieve your password, or unsubscribe
>>>>>>> from
>>>>>>> this list, visit
>>>>>>> https://dev.eclipse.org/mailman/listinfo/jgit-dev
>> _______________________________________________
>> jgit-dev mailing list
>> jgit-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/jgit-dev
>