| Re: [jgit-dev] Refs creation slowness with Jgit |
Hello Matthias,
Thanks for your response. Here are the results of the benchmark on the
machine.
Benchmark Mode Cnt
Score Error Units
CreateFileSnapshotBenchmark.testCreateFile avgt 25
62917.171 ± 131776.325 ns/op
CreateFileSnapshotBenchmark.testCreateFileSnapshot avgt 25
3263.106 ± 126.821 ns/op
LookupFileStoreBenchmark.testLookupFileStore avgt 25
292185770.706 ± 1830241.558 ns/op
SimpleLruCacheBenchmark.readwrite avgt 25
533.560 ± 36.500 ns/op
SimpleLruCacheBenchmark.readwrite:testCacheRead avgt 25
292.723 ± 30.698 ns/op
SimpleLruCacheBenchmark.readwrite:testCacheWrite avgt 25
774.397 ± 84.068 ns/op
On 2020-09-23 15:26, Matthias Sohn wrote:
> On Wed, Sep 23, 2020 at 8:05 PM <kaushikl@xxxxxxxxxxxxxx> wrote:
>
>> CC: repo-discuss@xxxxxxxxxxxxxxxx
>>
>> On 2020-09-23 09:13, kaushikl@xxxxxxxxxxxxxx wrote:
>>> Hello,
>>>
>>> When investigating slow Gerrit NoteDB migration times, we noticed
>> a
>>> slowness with jgit creating refs. During the migration a ref is
>>> created for each Gerrit change, for example
>> ‘changes/01/1/meta’,
>>> ‘changes/02/2/meta’, ‘changes/03/3/meta’ and so on. We see
>> the
>>> slowness when creating refs in a new path. This behaviour is
>>> repeatable, i.e each time a ref is created in a new path, it is
>> slow.
>>> We noticed it is fast on some machines, but we couldn't quite nail
>>> down why.
>>>
>>> I have a small sample program[1] which illustrates the slowness.
>> It
>>> creates two refs 'refs/heads/test_simple' and
>> 'refs/heads/test/foo'.
>>> On executing the program, I see output:
>>>
>>> test/foo: 325 ms
>>> test_simple: 4 ms
>>>
>>> My expectation is that 'test/foo' will also be created in order of
>>> milliseconds and not in order of few hundred milliseconds. The
>>> slowness seems to be stemming from the Files.getFileStore(dir)
>> call in
>>> FS.FileStoreAttributes.getFileStoreAttributes(Path dir).
>
> Can you run the LookupFileStoreBenchmark [1] on this system ?
> On my Mac this yields 63us to lookup a FileStore for a new file path.
>
> After running the Maven build the benchmarks can be run using Maven:
>
> $ cd jgit
> $ mvn clean install -DskipTests -Dmaven.javadoc.skip=true
>
> $ java -jar org.eclipse.jgit.benchmarks/target/benchmarks.jar
>
> or in Eclipse run the main method of the LookupFileStoreBenchmark
> class.
>
> [1]
> https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/refs/heads/master/org.eclipse.jgit.benchmarks/src/org/eclipse/jgit/benchmarks/LookupFileStoreBenchmark.java
>
> -Matthias
>
>>> Specs:
>>>
>>> Jgit version is 5.9
>>>
>>> $ java -version # also tried with version 1.8.0_252
>>> openjdk version "1.8.0_232"
>>> OpenJDK Runtime Environment (build
>>> 1.8.0_232-8u232-b09-0ubuntu1~16.04.1-b09)
>>> OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
>>>
>>> $ uname -r
>>> 4.15.0-54-generic
>>>
>>> $ lsb_release -a
>>> No LSB modules are available.
>>> Distributor ID: Ubuntu
>>> Description: Ubuntu 16.04.6 LTS
>>> Release: 16.04
>>> Codename: xenial
>>>
>>> Filesystem type is ext4
>>>
>>>
>>>
>>> [1]
>>> package test;
>>>
>>> import java.io.IOException;
>>> import java.nio.charset.StandardCharsets;
>>> import java.nio.file.Files;
>>> import java.nio.file.Path;
>>> import java.nio.file.Paths;
>>> import java.util.Arrays;
>>> import java.util.concurrent.TimeUnit;
>>>
>>> import org.eclipse.jgit.api.Git;
>>> import org.eclipse.jgit.api.errors.GitAPIException;
>>> import org.eclipse.jgit.lib.Repository;
>>> import org.eclipse.jgit.revwalk.RevCommit;
>>> import org.eclipse.jgit.revwalk.RevWalk;
>>>
>>> public class Test {
>>> public static void main(String[] args) {
>>> try {
>>>
>>> String path = null;
>>> if (args.length == 1) {
>>> path = args[0];
>>> } else {
>>> System.out.println("Repo path must be specified.");
>>> System.exit(1);
>>> }
>>> Path repoPath = Paths.get(path);
>>> long startTimeInNanoSecs;
>>> long estimatedTimeInNanoSecs;
>>>
>>> Git git = Git.init().setDirectory(repoPath.toFile()).call();
>>> Repository repo = git.getRepository();
>>> RevWalk walk = new RevWalk(repo);
>>>
>>> Files.write(repoPath.resolve("file"), Arrays.asList("line"),
>>> StandardCharsets.UTF_8);
>>> git.add().addFilepattern("file").call();
>>> git.commit().setMessage("create file").setAuthor("author",
>>> "author@xxxxxxxxx").call();
>>> RevCommit commit =
>>>
>> walk.parseCommit(repo.exactRef("refs/heads/master").getObjectId());
>>>
>>> String branch = "test/foo";
>>> startTimeInNanoSecs = System.nanoTime();
>>>
>> git.branchCreate().setName(branch).setStartPoint(commit).call();
>>> estimatedTimeInNanoSecs = System.nanoTime() -
>>> startTimeInNanoSecs;
>>> System.out.println(branch + ": " +
>>> TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");
>>>
>>> branch = "test_simple";
>>> startTimeInNanoSecs = System.nanoTime();
>>>
>> git.branchCreate().setName(branch).setStartPoint(commit).call();
>>> estimatedTimeInNanoSecs = System.nanoTime() -
>>> startTimeInNanoSecs;
>>> System.out.println(branch + ": " +
>>> TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");
>>> } catch (IllegalStateException | GitAPIException | IOException
>> e) {
>>> e.printStackTrace();
>>> }
>>> }
>>> }
>>> _______________________________________________
>>> jgit-dev mailing list
>>> jgit-dev@xxxxxxxxxxx
>>> To unsubscribe from this list, visit
>>> https://www.eclipse.org/mailman/listinfo/jgit-dev
>> _______________________________________________
>> jgit-dev mailing list
>> jgit-dev@xxxxxxxxxxx
>> To unsubscribe from this list, visit
>> https://www.eclipse.org/mailman/listinfo/jgit-dev
_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jgit-dev