Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] exception when adding big files (urgent)

2012/3/15 Markus Duft <markus.duft@xxxxxxxxxx>
On 03/15/2012 09:52 AM, Markus Duft wrote:
> Hey!
>
[snip]
>
> with current jgit/egit master (yes, we're stupid enough to run live builds productively :D). any hints on what this could be caused by? i'll start debugging now...
>
> i could reproduce the issue by creating a new repo and a new project in a fresh workspace, and then adding two (text!) files with approx 3 MB each to the repo. if i try to commit, i get the above exception.

uh - i found out, that tempBuffer in ObjectDirectoryInserter is a zero length array, which is _bad_.. i traced this:

* ObjectDirectoryInserter.insert calls buffer(len) with len = 3157208;
* since len > tempBufSize (which is max 1000000), a zero length buffer is created, memorized and returned.
* since returned buf.length < len, insert() chooses to go through a temporary file (toTemp()).
* toTemp() calls buffer() which returns the previously allocated zero sized buffer.
* it then calls is.read(buf, 0, Math.min(len, buf.length)) where len is still 3157208, and bug.length = 0
* thus, 0 is returnes, and toTemp calls throw shortInput(len).

is there _any_ good reason for buffer(len) to return a zero sized buffer? shouldn't it instead return new byte[tempBufSize] if len > tempBufSize?

i have a change [1] ready, it also includes a test demoing the problem.

[1] https://git.eclipse.org/r/#/c/5363/

this was introduced by Robin [2] as a quick hack to improve imperfect support
for autocrlf handling, I am sure he has some hints on how to further improve that.

[2] https://git.eclipse.org/r/#/c/4998/ 

--
Matthias

Back to the top