Skip to main content


Eclipse Community Forums

      Home
Home » Eclipse Projects » EGit / JGit » Slow performance of add command
Slow performance of add command [message #1828385] Mon, 08 June 2020 17:19 Go to next message
Eclipse UserFriend
I'm embedding JGit 5.6 in a tool that manages installed files, and if I run AddCommand.call() with a directory with about 300M in 10k files, it takes about a minute, even if few files are changing. If I run binary git, the same operation returns in a couple seconds.
I profiled this operation, and most of the time is taken by ObjectDirectoryInserter.toTemp()
index.php/fa/38287/0/

It appears that the whole directory is getting copied to a temp ZIP file before getting processed. Could this possibly done in-memory? Is there any config option around this? I'd be interested in coming up with an alternative faster implementation of this, but I'd like to get some more info and/or feedback.
Thanks, Bob
Re: Slow performance of add command [message #1828398 is a reply to message #1828385] Tue, 09 June 2020 03:55 Go to previous message
Eclipse UserFriend
When you add a directory to the index then jgit has to check whether each of the files has new content. If you have 10k files this means it would has to check for 10k files which was modified since it was last added to the index. And even worse: sometimes if you are working on filesystems with not very accurate timer resolution its hard to detect that a file is unchanged. And then you really have to read the file content and compare the checksum to what is stored in the index to find out the file is unchanged. And when you say most of the time is in ObjectDirectoryInserter.toTemp() then this means jgit is trying to add all the files to the index because it maybe fails to detect that they are unchanged. ObjectDirectoryInserter.toTemp() is called when adding new content for a single file to the index.
Can you share your code (only that parts constructing and dealing with the AddCommand) maybe some testData?
Previous Topic:Errors in Eclipse 2020-03 with latest ADT
Next Topic:egit clone throws exception
Goto Forum:
  


Current Time: Tue May 13 08:41:15 EDT 2025

Powered by FUDForum. Page generated in 0.06397 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly