Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] Refs creation slowness with Jgit 
CC: repo-discuss@xxxxxxxxxxxxxxxx

On 2020-09-23 09:13, kaushikl@xxxxxxxxxxxxxx wrote:

Hello,

When investigating slow Gerrit NoteDB migration times, we noticed a
slowness with jgit creating refs. During the migration a ref is
created for each Gerrit change, for example ‘changes/01/1/meta’,
‘changes/02/2/meta’, ‘changes/03/3/meta’ and so on. We see the
slowness when creating refs in a new path. This behaviour is
repeatable, i.e each time a ref is created in a new path, it is slow.
We noticed it is fast on some machines, but we couldn't quite nail
down why.

I have a small sample program[1] which illustrates the slowness. It
creates two refs 'refs/heads/test_simple' and 'refs/heads/test/foo'.
On executing the program, I see output:

test/foo: 325 ms
test_simple: 4 ms

My expectation is that 'test/foo' will also be created in order of
milliseconds and not in order of few hundred milliseconds. The
slowness seems to be stemming from the Files.getFileStore(dir) call in
FS.FileStoreAttributes.getFileStoreAttributes(Path dir).

Specs:

Jgit version is 5.9

$ java -version # also tried with version 1.8.0_252
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-8u232-b09-0ubuntu1~16.04.1-b09) 
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

$ uname -r
4.15.0-54-generic

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.6 LTS
Release:        16.04
Codename:       xenial

Filesystem type is ext4



[1]
package test;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.concurrent.TimeUnit;

import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevWalk;

public class Test {
  public static void main(String[] args) {
    try {

      String path = null;
     if (args.length == 1) {
        path = args[0];
      } else {
        System.out.println("Repo path must be specified.");
        System.exit(1);
      }
      Path repoPath = Paths.get(path);
      long startTimeInNanoSecs;
      long estimatedTimeInNanoSecs;

      Git git = Git.init().setDirectory(repoPath.toFile()).call();
      Repository repo = git.getRepository();
      RevWalk walk = new RevWalk(repo);

      Files.write(repoPath.resolve("file"), Arrays.asList("line"),
StandardCharsets.UTF_8);
      git.add().addFilepattern("file").call();
      git.commit().setMessage("create file").setAuthor("author",
"author@xxxxxxxxx").call();
      RevCommit commit =
walk.parseCommit(repo.exactRef("refs/heads/master").getObjectId());

      String branch = "test/foo";
      startTimeInNanoSecs = System.nanoTime();
      git.branchCreate().setName(branch).setStartPoint(commit).call();
estimatedTimeInNanoSecs = System.nanoTime() - startTimeInNanoSecs; 
      System.out.println(branch + ": " +
TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");

      branch = "test_simple";
      startTimeInNanoSecs = System.nanoTime();
      git.branchCreate().setName(branch).setStartPoint(commit).call();
estimatedTimeInNanoSecs = System.nanoTime() - startTimeInNanoSecs; 
      System.out.println(branch + ": " +
TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");
    } catch (IllegalStateException | GitAPIException | IOException e) {
     e.printStackTrace();
    }
  }
}
_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
To unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jgit-dev

Back to the top