Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EGit / JGit » Reduce walk memory usage(Possibilities for reducing memory usage.)
icon3.gif  Reduce walk memory usage [message #985739] Thu, 15 November 2012 21:46 Go to next message
Doru Chiulan is currently offline Doru ChiulanFriend
Messages: 2
Registered: August 2012
Junior Member
Hy guys.

I am using a PlotWalk for a git client application.

I have to open some pretty large repositories like linux-2.6, and i had to implement a "sliding window" algorhitm for my table which loads "chunks" of commits from the PlotWalk, because if i would load all commits the memory usage was huge.

Even if i use this, i want to reduce even more the memory usage. The application uses ~100mb of ram until the PlotWalk is used. Then, the PlotWalk uses 300mb of memory + 100 used by the rest of the application.

I read all possibilities of reducing PlotWalk memory usage in JGit User's guide, but i can't find a way.

I saw that the memory usage on PlotWalk started to rise when next() method is invoked.

Do you have any suggestions, maybe extending the PlotWalk, or it's Generators ?

Report message to a moderator

Re: Reduce walk memory usage [message #985795 is a reply to message #985739] Fri, 16 November 2012 08:37 Go to previous messageGo to next message
Matthias Sohn is currently offline Matthias SohnFriend
Messages: 1268
Registered: July 2009
Senior Member
Maybe this discussion [1] helps to gain more insight.

[1] http://dev.eclipse.org/mhonarc/lists/jgit-dev/msg01588.html

Report message to a moderator

Re: Reduce walk memory usage [message #985796 is a reply to message #985739] Fri, 16 November 2012 08:37 Go to previous message
Stefan Lay is currently offline Stefan LayFriend
Messages: 342
Registered: July 2009
Senior Member
Hi,

we already tried something in that direction for EGit. It turned out that the problem is that PlotWalk uses RevSort.TOPO. In that case the first call of next() triggers a walk of the whole history internally in order to do the topological sorting, loading all the commits into memory. See the discussion on http://dev.eclipse.org/mhonarc/lists/jgit-dev/msg01589.html and its predecessor for more details.

In Egit, we did two things:

- Load commits in batches (https://git.eclipse.org/r/#/c/5654/ for the gerrit change)
- Lazy loading of the commit body (https://git.eclipse.org/r/#/c/6149/ in gerrit)


We didn't investigate further. If you find out anything more, please let us know (or contribute to jgit/egit Wink. I would suggest to use the jgit mailing list for that (https://dev.eclipse.org/mailman/listinfo/jgit-dev).

Cheers,
Stefan

Report message to a moderator

Previous Topic:Commit/amend not possible
Next Topic:GIT Hooks
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Tue Apr 23 09:49:54 GMT 2024

Powered by FUDForum. Page generated in 0.03145 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly