Eclipse Community Forums: EGit / JGit » Slow performance of add command

Help

Home

Home » Eclipse Projects » EGit / JGit » Slow performance of add command

Show: Today's Messages :: Show Polls :: Message Navigator

Slow performance of add command [message #1828385]

Mon, 08 June 2020 21:19

Bob Tinsman

Messages: 1
Registered: June 2020

Junior Member

I'm embedding JGit 5.6 in a tool that manages installed files, and if I run AddCommand.call() with a directory with about 300M in 10k files, it takes about a minute, even if few files are changing. If I run binary git, the same operation returns in a couple seconds.
I profiled this operation, and most of the time is taken by ObjectDirectoryInserter.toTemp()
index.php/fa/38287/0/

It appears that the whole directory is getting copied to a temp ZIP file before getting processed. Could this possibly done in-memory? Is there any config option around this? I'd be interested in coming up with an alternative faster implementation of this, but I'd like to get some more info and/or feedback.
Thanks, Bob

Report message to a moderator

Re: Slow performance of add command [message #1828398 is a reply to message #1828385]

Tue, 09 June 2020 07:55

Christian Halstrick

Messages: 274
Registered: July 2009

Senior Member

When you add a directory to the index then jgit has to check whether each of the files has new content. If you have 10k files this means it would has to check for 10k files which was modified since it was last added to the index. And even worse: sometimes if you are working on filesystems with not very accurate timer resolution its hard to detect that a file is unchanged. And then you really have to read the file content and compare the checksum to what is stored in the index to find out the file is unchanged. And when you say most of the time is in ObjectDirectoryInserter.toTemp() then this means jgit is trying to add all the files to the index because it maybe fails to detect that they are unchanged. ObjectDirectoryInserter.toTemp() is called when adding new content for a single file to the index.
Can you share your code (only that parts constructing and dealing with the AddCommand) maybe some testData?

Ciao
Chris

Report message to a moderator

Previous Topic:	Errors in Eclipse 2020-03 with latest ADT
Next Topic:	egit clone throws exception

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Tue Dec 05 02:40:15 GMT 2023

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter