[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] Endless loop when detecting filesystem timestamps resolution

Hi,
today I merged MatthiasÂhttps://git.eclipse.org/r/#/c/144897/Âwhich changes the implementation of touch

On Wed, Jul 17, 2019 at 4:32 PM Dmitry Pavlenko <pavlenko@xxxxxxxxxxxxx> wrote:
Hello!

We're using JGit library internally and for one of our clients it entered into an endless loop.

The stack trace obtained by 'jstack' is

 Âjava.lang.Thread.State: TIMED_WAITING (sleeping)
    at java.lang.Thread.sleep(Native Method)
    at java.lang.Thread.sleep(Thread.java:340)
    at java.util.concurrent.TimeUnit.sleep(TimeUnit.java:386)
    at org.eclipse.jgit.util.FS$FileStoreAttributeCache.<init>(FS.java:242)
    at org.eclipse.jgit.util.FS$FileStoreAttributeCache.getFsTimestampResolution(FS.java:213)
    at org.eclipse.jgit.util.FS.getFsTimerResolution(FS.java:323)
    at org.eclipse.jgit.internal.storage.file.FileSnapshot.<init>(FileSnapshot.java:186)
    at org.eclipse.jgit.internal.storage.file.FileSnapshot.save(FileSnapshot.java:122)
    at org.eclipse.jgit.storage.file.FileBasedConfig.load(FileBasedConfig.java:159)
    at org.eclipse.jgit.internal.storage.file.FileRepository.loadUserConfig(FileRepository.java:261)
    at org.eclipse.jgit.internal.storage.file.FileRepository.<init>(FileRepository.java:197)
    at org.eclipse.jgit.lib.RepositoryCache$FileKey.getGitRepository(RepositoryCache.java:519)
    at org.eclipse.jgit.lib.RepositoryCache$FileKey.isGitRepository(RepositoryCache.java:500)

So the problem happens in the following code:

                FileTime startTime = Files.getLastModifiedTime(probe);
                FileTime actTime = startTime;
                long sleepTime = 512;
                while (actTime.compareTo(startTime) <= 0) {
                    TimeUnit.NANOSECONDS.sleep(sleepTime);
                    FileUtils.touch(probe);
                    actTime = Files.getLastModifiedTime(probe);
                    // limit sleep time to max. 100ms
                    if (sleepTime < 100_000_000L) {
                        sleepTime = sleepTime * 2;
                    }
                }

JGit creates a file named .probe-<UUID> and the 'touches' it until its 'mtime' is greater than the creation time.
And for the user it seems that FileUtils#touch is no-op and hence the cycle never ends.

The user also tried different filesystems and the problem happens on all filesystems. So it doesn't seem to be
a problem of a particular filesystem. The mounting options also don't have anything special.

Moreover, when the user 'touches' the .probe-<UUID> file with 'touch' command line,
the cycle finishes and the program continues. So I would conclude that the problem is in implementation
of FileUtils#touch (JVM is OpenJDK 25.212-b03):

    public static void touch(Path f) throws IOException {
        try (OutputStream fos = Files.newOutputStream(f)) {
            // touch the file
        }
    }

Also one of my colleagues recalled that he had similar problem with SVNKit: 'mtime' wasn't updated with just
opening+closing the file, but writing something was necessary; and the problem was really rare.

The user experienced JGit problem has Ubuntu, we tried to reproduce the problem on Ubuntu and the same filesystems and
we couldn't. I cannot reproduce the problem on my machine as well.

There's also an interesting longread regarding 'mtime':
 Âhttps://apenwarr.ca/log/20181113

I'll cite a piece:
Â* Is mtime always nonzero?
 No. Various cheaply-written virtual filesystems, like many fuse-based ones, don't bother setting mtime.

So I would suggest to
 - limit the number of iterations for this cycle to prevent the endless loop;
 - implement 'touch' in a different way, e.g. by writing something into the file (this doesn't solves the problem of hypothetical
    "cheaply-written virtual filesystems" though).

Here's the corresponding issue on our tracker: https://issues.tmatesoft.com/issue/SGT-1308
--
Dmitry Pavlenko,
TMate Software,
http://subgit.com/ - git-svn bridge



_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jgit-dev