Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] How to deal with LargeObjectException in RevWalk.parseCommit?

On Wed, Apr 4, 2012 at 10:48, Marc Strapetz <marc.strapetz@xxxxxxxxxxx> wrote:
> One of our users is running into a LargeObjectException$ExceedsLimit for
> RevWalk.parseCommit. I'm not sure what can make a commit-object itself
> that large, but I'm wondering if it's reasonable to have a limit here at
> all?
> org.eclipse.jgit.errors.LargeObjectException$ExceedsLimit: Object
> 8260e144113486b44c81bed39eb10dedd834804c exceeds 5 242 880 limit, actual
> size is 26 693 640

Ouch. They have a commit object whose message is ~26 MiB in size? That
is huge. Or the object information is corrupt.

> Personally, I'd prefer to let the application run into an OOME
> here. Should we make the limit configurable by a system property?

This is a hard limit in RevWalk right now:

	byte[] getCachedBytes(RevObject obj, ObjectLoader ldr)
			throws LargeObjectException, MissingObjectException, IOException {
		try {
			return ldr.getCachedBytes(5 * MB);
		} catch (LargeObjectException tooBig) {
			throw tooBig;

We could make this a repository level configuration setting, with the
default being the current 5 MiB. If users need it bigger, they could
set a larger limit in .git/config or ~/.gitconfig. If they set it very
large (e.g. 2048m) it would effectively disable this check and instead
throw LargeObjectException$OutOfMemory when the JRE can't perform the

IIRC I put this limit in here of 5 MiB because the ObjectLoader needs
an upper limit target. It could be changed to Integer.MAX_VALUE to
really make it as big as an array can go in Java. But I'm not sure its
reasonable to attempt to parse a commit that is nearly 2 GiB in size.
Most applications would want to abort before trying to get the GC to
give them that big of a contiguous memory block.

Back to the top