On 2020-10-06 16:23, Luca Milanesio wrote:On 7 Oct 2020, at 00:17, kaushikl@xxxxxxxxxxxxxx wrote:
On 2020-10-06 12:50, Luca Milanesio wrote:
On 6 Oct 2020, at 20:48, kaushikl@xxxxxxxxxxxxxx wrote:
On 2020-10-06 12:32, Luca Milanesio wrote:
On 6 Oct 2020, at 20:16, kaushikl@xxxxxxxxxxxxxx wrote:
Hi,
We have noticed that creating a new ref takes ~600ms on a repository
with around 100k packed-refs.
My test repository:
$ git count-objects -v
count: 0
size: 0
in-pack: 100042
packs: 1
size-pack: 9734
prune-packable: 0
garbage: 0
size-garbage: 0
$ find refs/ -type f | wc -l
0
Specs:
Machine has 32 cores and 250G RAM.
$ uname -r
4.4.0-165-generic
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial
Jgit version is 5.9
$ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build
1.8.0_242-8u242-b08-0ubuntu3~16.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
Consider a small program[1] which creates a ref 'simple'. On
executing the program on my test repository, I see output:
simple: 677 ms
This seems slow. Is this expected behavior with jgit?
Can you share the generated .gitconfig?
(When you use JGit on a filesystem, it performs the computation of
the
filesystem latency)
$ cat config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
trustFolderStat = false
I had trustFolderStat set to false in my test repositories, which causes packed-refs file to be parsed over and over even if no modifications have been made to it, which I think explains why I have been seeing long ref creation times.
I believe it isn’t needed if you are using a local filesystem, isn’t it? Also for an NFS mount, I typically do not set trustFolderStat to false, otherwise the packed-refs are not cached at all: for a very large repo with lots of refs, that is a killer.
Luca. [repack]
usedeltabaseoffset = true
[pack]
compression = 9
indexversion = 2
threads = 6
windowmemory = 2g
window = 250
depth = 50
[gc]
autopacklimit = 4
packrefs = true
reflogexpire = never
reflogexpireunreachable = never
auto = 0
[receive]
denyNonFastForwards = false
No, I mean the ~/.gitconfig of the user. If you are running Gerrit
v3.1 or later, then it is $GERRIT_SITE/etc/jgit.config.
Ah, OK, I see below in ~/.gitconfig.
[filesystem "Private Build|1.8.0_232|/dev/mapper/xxxxxx_local_mnt"]
timestampResolution = 1001 microseconds
minRacyThreshold = 5289 microseconds
[filesystem "Private
Build|1.8.0_242|/dev/mapper/xxxxxxx-xxxxxxx_local_mnt"]
timestampResolution = 1001 microseconds
minRacyThreshold = 5486 microseconds
Mmm … interesting: the timestamp resolution is quite fine (1ms) but
still it takes 600 ms to calculate?
Is this a local disk or a mounted NFS share?
SSD or spinning?
This is on a local spinning disk.
Is the disk fragmented?
I don't know how to check that, however, I copied the repository to
another machine with similar configuration and noticed that it was
showing similar performance( i.e greater than ~600ms) for ref
creation.
I believe you can short-circuit in JGit the filesystem timestamp
resolution, so that you can measure the pure filesystem-level
performance.
Luca.
Luca.
Luca.
[1]
package test;
import java.io.File;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevWalk;
public class Test {
public static void main(String[] args) {
try {
String path = null;
if (args.length == 1) {
path = args[0];
} else {
System.out.println("Repo path must be specified.");
System.exit(1);
}
String branch = "simple";
try (Git git = Git.open(new File(path))) {
Repository repo = git.getRepository();
RevWalk walk = new RevWalk(repo);
RevCommit commit =
walk.parseCommit(repo.exactRef("refs/heads/master").getObjectId());
long startTimeInNanoSecs = System.nanoTime();
git.branchCreate().setName(branch).setStartPoint(commit).call();
long estimatedTimeInNanoSecs = System.nanoTime() -
startTimeInNanoSecs;
System.out.println(branch + ": " +
TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");
}
} catch (IllegalStateException | GitAPIException | IOException e)
{
e.printStackTrace();
}
}
}
_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
To unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jgit-dev
|
