Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EGit / JGit » Performance bad, possible to use native git?(EGit / JGit bad performance)
Performance bad, possible to use native git? [message #1445848] Thu, 16 October 2014 02:07 Go to next message
rupert THURNER is currently offline rupert THURNERFriend
Messages: 5
Registered: July 2009
Junior Member
We try to put a rather large code base into eclipse, 100'000 files, 40'000 directories. Most of us use Eclipse on Windows, some use Idea IntelliJ.

While the IntelliJ, commandline/TortoiseGit persons are happy with Git, the Eclipse persons complain that Git support is unusable for such a big code base. "git status" takes 10 sec with (newest) native msysgit. It takes minutes with EGit/Jgit. The difference is even worse in subdirectories, <1sec with msysgit, minutes Eclipse. As the code base contains multiple eclpise projects developers only have opened subdirectories, i.e. a small part of the code. A developer mostly works on a private branch, one special module. He fetches changes, rebases the own change on top of the private branch and commits.

Would it be possible to configure native Git just like in IntelliJ?

best,

Rupert.

ps:
We are aware that the code base is legacy, Coming from CVS, we want to migrate it first to a new repository and after that break it up.
Re: Performance bad, possible to use native git? [message #1446379 is a reply to message #1445848] Thu, 16 October 2014 19:17 Go to previous messageGo to next message
Robin Rosenberg is currently offline Robin RosenbergFriend
Messages: 332
Registered: July 2009
Senior Member
rupert THURNER skrev 2014-10-16 14.04:
> We try to put a rather large code base into eclipse, 100'000 files, 40'000 directories. Most of us use Eclipse on Windows, some use Idea IntelliJ.
>
> While the IntelliJ, commandline/TortoiseGit persons are happy with Git, the Eclipse persons complain that Git support is unusable for such a big code base. "git status"
> takes 10 sec with (newest) native msysgit. It takes minutes with EGit/Jgit. The difference is even worse in subdirectories, <1sec with msysgit, minutes Eclipse. As the code
> base contains multiple eclpise projects developers only have opened subdirectories, i.e. a small part of the code. A developer mostly works on a private branch, one special
> module. He fetches changes, rebases the own change on top of the private branch and commits.
>
> Would it be possible to configure native Git just like in IntelliJ?

No.

There are other plugins that use native git, but I'm not up-to-date on which one to use.

There may be different reasons for this slowness. One might be memory, giving Eclipse more of it may help. Turning off linefeed conversion
may help (Eclipse doesn't need CRLF on Windows, though the default is). Packing the repo may help.

Having a profile of Eclipse doing the long operations may help in figuring out why things are so slow.

We can do Windows specific patches sometimes, but we don't dogfood them, since that would mean using Windows daily, which I believe none of
the active EGit/JGit developers do. It does take an effort from the Windows community in order to improve Windows specific issues, including
grabbing patches from Gerrit (which isn't very hard) rather than only testing releases. So far patching for Windows hasn't turned out well.

-- robin

> best,
>
> Rupert.
>
> ps:
> We are aware that the code base is legacy, Coming from CVS, we want to migrate it first to a new repository and after that break it up.
Re: Performance bad, possible to use native git? [message #1446662 is a reply to message #1446379] Fri, 17 October 2014 05:54 Go to previous messageGo to next message
rupert THURNER is currently offline rupert THURNERFriend
Messages: 5
Registered: July 2009
Junior Member
msysgit was also quite slow, and compared to git on linux it still is. what makes a huge impact in msysgit are two things. first, "core.fscache" [1] makes "git status" minimum 5 times faster. second, operations can be restricted to subdirectories. e.g. "git status .". depending on the number of subdirectories, this is 10 times faster. with both in place git status goes down from close to a minute to a second in a subdirectory.

is this something which jgit (core.fscache, restrict to directory) and egit (restrict to directory) could do as well?

[1] github.com/msysgit/git/commit/64d63240762df22e92b287b145d75a0d68a66988 (sorry for the broken link - do not have enough posts here to post links to other sites than eclipse *blush*)

[Updated on: Fri, 17 October 2014 09:23]

Report message to a moderator

Re: Performance bad, possible to use native git? [message #1446834 is a reply to message #1446379] Fri, 17 October 2014 11:28 Go to previous messageGo to next message
Rüdiger Herrmann is currently offline Rüdiger HerrmannFriend
Messages: 581
Registered: July 2009
Senior Member
On 16.10.2014 21:17, Robin Rosenberg wrote:
> Windows specific issues, including
> grabbing patches from Gerrit (which isn't very hard) rather than only
> testing releases. So far patching for Windows hasn't turned out well.
Are there open Windows patches? I haven't found any but if there are I
would be interested in helping to test them.

- Rüdiger
Re: Performance bad, possible to use native git? [message #1448240 is a reply to message #1446834] Sun, 19 October 2014 16:37 Go to previous messageGo to next message
Robin Rosenberg is currently offline Robin RosenbergFriend
Messages: 332
Registered: July 2009
Senior Member
Rüdiger Herrmann skrev 2014-10-17 13.28:
> On 16.10.2014 21:17, Robin Rosenberg wrote:
>> Windows specific issues, including
>> grabbing patches from Gerrit (which isn't very hard) rather than only
>> testing releases. So far patching for Windows hasn't turned out well.
> Are there open Windows patches? I haven't found any but if there are I would be interested in helping to test them.

https://git.eclipse.org/r/#/c/4942/

and

https://git.eclipse.org/r/#/c/1614/

In theory these aren't Windows specific, but in practice only Windows users care.

Anything newline-related had better be checked for Windows too.

e.g:
https://git.eclipse.org/r/#/c/33219/

Code that has already been merged may need some love, e.g. Symbolic links stuff "seems to work",
but there may be undetected issue (if not because it seem to work better than I anticipated, can't be
that easy....).

CRLF handling could probably be more efficient, and I suspect smudge handling could be more
efficient (matters for autocrlf).

-- robin
Re: Performance bad, possible to use native git? [message #1449507 is a reply to message #1448240] Tue, 21 October 2014 13:13 Go to previous message
Rüdiger Herrmann is currently offline Rüdiger HerrmannFriend
Messages: 581
Registered: July 2009
Senior Member
Hi Robin,

thanks for the links. I will see if I can spend some time on them,
though I can't offer much practcical experience with the CRLF patches.
We use Unix LF on all platforms.

Rüdiger

On 19.10.2014 18:37, Robin Rosenberg wrote:
> Rüdiger Herrmann skrev 2014-10-17 13.28:
>> On 16.10.2014 21:17, Robin Rosenberg wrote:
>>> Windows specific issues, including
>>> grabbing patches from Gerrit (which isn't very hard) rather than only
>>> testing releases. So far patching for Windows hasn't turned out well.
>> Are there open Windows patches? I haven't found any but if there are I
>> would be interested in helping to test them.
>
> https://git.eclipse.org/r/#/c/4942/
>
> and
>
> https://git.eclipse.org/r/#/c/1614/
>
> In theory these aren't Windows specific, but in practice only Windows
> users care.
>
> Anything newline-related had better be checked for Windows too.
>
> e.g:
> https://git.eclipse.org/r/#/c/33219/
>
> Code that has already been merged may need some love, e.g. Symbolic
> links stuff "seems to work",
> but there may be undetected issue (if not because it seem to work better
> than I anticipated, can't be
> that easy....).
>
> CRLF handling could probably be more efficient, and I suspect smudge
> handling could be more
> efficient (matters for autocrlf).
>
> -- robin
>
>
Previous Topic:How to call share Project command from my plugin java code
Next Topic:Does EGit 3.4.1 supports SSLv3 or TLS?
Goto Forum:
  


Current Time: Sun Sep 15 13:03:47 GMT 2019

Powered by FUDForum. Page generated in 0.02232 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top