Re: [jgit-dev] Creating new refs on a repository with 100k packed-refs i

[kaushikl@xxxxxxxxxxxxxx]

On 2020-10-06 12:32, Luca Milanesio wrote:
> > On 6 Oct 2020, at 20:16, kaushikl@xxxxxxxxxxxxxx wrote:
> >
> > Hi,
> >
> > We have noticed that creating a new ref takes ~600ms on a repository 
> > with around 100k packed-refs.
> >
> > My test repository:
> >
> >    $ git count-objects -v
> >    count: 0
> >    size: 0
> >    in-pack: 100042
> >    packs: 1
> >    size-pack: 9734
> >    prune-packable: 0
> >    garbage: 0
> >    size-garbage: 0
> >
> >    $ find refs/ -type f | wc -l
> >    0
> >
> > Specs:
> >
> >    Machine has 32 cores and 250G RAM.
> >
> >    $ uname -r
> >    4.4.0-165-generic
> >
> >    $ lsb_release -a
> >    No LSB modules are available.
> >    Distributor ID:    Ubuntu
> >    Description:    Ubuntu 16.04.6 LTS
> >    Release:    16.04
> >    Codename:    xenial
> >
> >    Jgit version is 5.9
> >
> >    $ java -version
> >    openjdk version "1.8.0_242"
> >    OpenJDK Runtime Environment (build 
> > 1.8.0_242-8u242-b08-0ubuntu3~16.04-b08)
> >    OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
> >
> > Consider a small program[1] which creates a ref 'simple'. On executing 
> > the program on my test repository, I see output:
> >
> > simple: 677 ms
> >
> > This seems slow. Is this expected behavior with jgit?
>
> Can you share the generated .gitconfig?
> (When you use JGit on a filesystem, it performs the computation of the
> filesystem latency)

$ cat config
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
	trustFolderStat = false
[repack]
	usedeltabaseoffset = true
[pack]
	compression = 9
	indexversion = 2
	threads = 6
	windowmemory = 2g
	window = 250
	depth = 50
[gc]
	autopacklimit = 4
	packrefs = true
	reflogexpire = never
	reflogexpireunreachable = never
	auto = 0
[receive]
	denyNonFastForwards = false

> Is this a local disk or a mounted NFS share?
> SSD or spinning?

This is on a local spinning disk.

> Luca.
>
> > [1]
> > package test;
> >
> > import java.io.File;
> > import java.io.IOException;
> > import java.util.concurrent.TimeUnit;
> >
> > import org.eclipse.jgit.api.Git;
> > import org.eclipse.jgit.api.errors.GitAPIException;
> > import org.eclipse.jgit.lib.Repository;
> > import org.eclipse.jgit.revwalk.RevCommit;
> > import org.eclipse.jgit.revwalk.RevWalk;
> >
> > public class Test {
> >  public static void main(String[] args) {
> >    try {
> >      String path = null;
> >      if (args.length == 1) {
> >        path = args[0];
> >      } else {
> >        System.out.println("Repo path must be specified.");
> >        System.exit(1);
> >      }
> >      String branch = "simple";
> >      try (Git git = Git.open(new File(path))) {
> >        Repository repo = git.getRepository();
> >        RevWalk walk = new RevWalk(repo);
> >        RevCommit commit = 
> > walk.parseCommit(repo.exactRef("refs/heads/master").getObjectId());
> >        long startTimeInNanoSecs = System.nanoTime();
> >        
> > git.branchCreate().setName(branch).setStartPoint(commit).call();
> >        long estimatedTimeInNanoSecs = System.nanoTime() - 
> > startTimeInNanoSecs;
> >        System.out.println(branch + ": " + 
> > TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");
> >      }
> >    } catch (IllegalStateException | GitAPIException | IOException e) {
> >      e.printStackTrace();
> >    }
> >  }
> > }
> > _______________________________________________
> > jgit-dev mailing list
> > jgit-dev@xxxxxxxxxxx
> > To unsubscribe from this list, visit 
> > https://www.eclipse.org/mailman/listinfo/jgit-dev