Skip to main content

Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EGit / JGit » Slow performance of add command
Slow performance of add command [message #1828385] Mon, 08 June 2020 21:19 Go to next message
Bob Tinsman is currently offline Bob TinsmanFriend
Messages: 1
Registered: June 2020
Junior Member
I'm embedding JGit 5.6 in a tool that manages installed files, and if I run with a directory with about 300M in 10k files, it takes about a minute, even if few files are changing. If I run binary git, the same operation returns in a couple seconds.
I profiled this operation, and most of the time is taken by ObjectDirectoryInserter.toTemp()

It appears that the whole directory is getting copied to a temp ZIP file before getting processed. Could this possibly done in-memory? Is there any config option around this? I'd be interested in coming up with an alternative faster implementation of this, but I'd like to get some more info and/or feedback.
Thanks, Bob
Re: Slow performance of add command [message #1828398 is a reply to message #1828385] Tue, 09 June 2020 07:55 Go to previous message
Christian Halstrick is currently offline Christian HalstrickFriend
Messages: 274
Registered: July 2009
Senior Member
When you add a directory to the index then jgit has to check whether each of the files has new content. If you have 10k files this means it would has to check for 10k files which was modified since it was last added to the index. And even worse: sometimes if you are working on filesystems with not very accurate timer resolution its hard to detect that a file is unchanged. And then you really have to read the file content and compare the checksum to what is stored in the index to find out the file is unchanged. And when you say most of the time is in ObjectDirectoryInserter.toTemp() then this means jgit is trying to add all the files to the index because it maybe fails to detect that they are unchanged. ObjectDirectoryInserter.toTemp() is called when adding new content for a single file to the index.
Can you share your code (only that parts constructing and dealing with the AddCommand) maybe some testData?

Previous Topic:Errors in Eclipse 2020-03 with latest ADT
Next Topic:egit clone throws exception
Goto Forum:

Current Time: Tue Dec 05 02:40:15 GMT 2023

Powered by FUDForum. Page generated in 0.01812 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top