[jgit-dev] Refs creation slowness with Jgit

[kaushikl@xxxxxxxxxxxxxx]

Hello,

When investigating slow Gerrit NoteDB migration times, we noticed a 
slowness with jgit creating refs. During the migration a ref is created 
for each Gerrit change, for example ‘changes/01/1/meta’, 
‘changes/02/2/meta’, ‘changes/03/3/meta’ and so on. We see the slowness 
when creating refs in a new path. This behaviour is repeatable, i.e each 
time a ref is created in a new path, it is slow. We noticed it is fast 
on some machines, but we couldn't quite nail down why.

I have a small sample program[1] which illustrates the slowness. It 
creates two refs 'refs/heads/test_simple' and 'refs/heads/test/foo'. On 
executing the program, I see output:

test/foo: 325 ms
test_simple: 4 ms

My expectation is that 'test/foo' will also be created in order of 
milliseconds and not in order of few hundred milliseconds. The slowness 
seems to be stemming from the Files.getFileStore(dir) call in 
FS.FileStoreAttributes.getFileStoreAttributes(Path dir).

Specs:

Jgit version is 5.9

$ java -version # also tried with version 1.8.0_252
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 
1.8.0_232-8u232-b09-0ubuntu1~16.04.1-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

$ uname -r
4.15.0-54-generic

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.6 LTS
Release:        16.04
Codename:       xenial

Filesystem type is ext4



[1]
package test;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.concurrent.TimeUnit;

import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevWalk;

public class Test {
  public static void main(String[] args) {
    try {

      String path = null;
     if (args.length == 1) {
        path = args[0];
      } else {
        System.out.println("Repo path must be specified.");
        System.exit(1);
      }
      Path repoPath = Paths.get(path);
      long startTimeInNanoSecs;
      long estimatedTimeInNanoSecs;

      Git git = Git.init().setDirectory(repoPath.toFile()).call();
      Repository repo = git.getRepository();
      RevWalk walk = new RevWalk(repo);

      Files.write(repoPath.resolve("file"), Arrays.asList("line"), 
StandardCharsets.UTF_8);
      git.add().addFilepattern("file").call();
      git.commit().setMessage("create file").setAuthor("author", 
"author@xxxxxxxxx").call();
      RevCommit commit = 
walk.parseCommit(repo.exactRef("refs/heads/master").getObjectId());

      String branch = "test/foo";
      startTimeInNanoSecs = System.nanoTime();
      git.branchCreate().setName(branch).setStartPoint(commit).call();
      estimatedTimeInNanoSecs = System.nanoTime() - startTimeInNanoSecs;
      System.out.println(branch + ": " + 
TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");

      branch = "test_simple";
      startTimeInNanoSecs = System.nanoTime();
      git.branchCreate().setName(branch).setStartPoint(commit).call();
      estimatedTimeInNanoSecs = System.nanoTime() - startTimeInNanoSecs;
      System.out.println(branch + ": " + 
TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");
    } catch (IllegalStateException | GitAPIException | IOException e) {
     e.printStackTrace();
    }
  }
}