Re: [jgit-dev] Endless loop when detecting filesystem timestamps resolut

["Sohn, Matthias" <matthias.sohn@xxxxxxx>]

This is bug 548598 [1], a patch for this bug was merged on the stable-5.1 branch recently.
I am working on a series of patches [2], as soon as this is merged service releases will be created from 5.1.9 up to 5.4.1

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=548598
[2] https://git.eclipse.org/r/#/c/146085/

-Matthias

On 17.07.19, 16:33, "jgit-dev-bounces@xxxxxxxxxxx on behalf of Dmitry Pavlenko" <jgit-dev-bounces@xxxxxxxxxxx on behalf of pavlenko@xxxxxxxxxxxxx> wrote:

    Hello!
    
    We're using JGit library internally and for one of our clients it entered into an endless loop.
    
    The stack trace obtained by 'jstack' is
    
       java.lang.Thread.State: TIMED_WAITING (sleeping)
            at java.lang.Thread.sleep(Native Method)
            at java.lang.Thread.sleep(Thread.java:340)
            at java.util.concurrent.TimeUnit.sleep(TimeUnit.java:386)
            at org.eclipse.jgit.util.FS$FileStoreAttributeCache.<init>(FS.java:242)
            at org.eclipse.jgit.util.FS$FileStoreAttributeCache.getFsTimestampResolution(FS.java:213)
            at org.eclipse.jgit.util.FS.getFsTimerResolution(FS.java:323)
            at org.eclipse.jgit.internal.storage.file.FileSnapshot.<init>(FileSnapshot.java:186)
            at org.eclipse.jgit.internal.storage.file.FileSnapshot.save(FileSnapshot.java:122)
            at org.eclipse.jgit.storage.file.FileBasedConfig.load(FileBasedConfig.java:159)
            at org.eclipse.jgit.internal.storage.file.FileRepository.loadUserConfig(FileRepository.java:261)
            at org.eclipse.jgit.internal.storage.file.FileRepository.<init>(FileRepository.java:197)
            at org.eclipse.jgit.lib.RepositoryCache$FileKey.getGitRepository(RepositoryCache.java:519)
            at org.eclipse.jgit.lib.RepositoryCache$FileKey.isGitRepository(RepositoryCache.java:500)
    
    So the problem happens in the following code:
    
    				FileTime startTime = Files.getLastModifiedTime(probe);
    				FileTime actTime = startTime;
    				long sleepTime = 512;
    				while (actTime.compareTo(startTime) <= 0) {
    					TimeUnit.NANOSECONDS.sleep(sleepTime);
    					FileUtils.touch(probe);
    					actTime = Files.getLastModifiedTime(probe);
    					// limit sleep time to max. 100ms
    					if (sleepTime < 100_000_000L) {
    						sleepTime = sleepTime * 2;
    					}
    				}
    
    JGit creates a file named .probe-<UUID> and the 'touches' it until its 'mtime' is greater than the creation time.
    And for the user it seems that FileUtils#touch is no-op and hence the cycle never ends.
    
    The user also tried different filesystems and the problem happens on all filesystems. So it doesn't seem to be
    a problem of a particular filesystem. The mounting options also don't have anything special.
    
    Moreover, when the user 'touches' the .probe-<UUID> file with 'touch' command line,
    the cycle finishes and the program continues. So I would conclude that the problem is in implementation
    of FileUtils#touch (JVM is OpenJDK 25.212-b03):
    
    	public static void touch(Path f) throws IOException {
    		try (OutputStream fos = Files.newOutputStream(f)) {
    			// touch the file
    		}
    	}
    
    Also one of my colleagues recalled that he had similar problem with SVNKit: 'mtime' wasn't updated with just
    opening+closing the file, but writing something was necessary; and the problem was really rare.
    
    The user experienced JGit problem has Ubuntu, we tried to reproduce the problem on Ubuntu and the same filesystems and
    we couldn't. I cannot reproduce the problem on my machine as well.
    
    There's also an interesting longread regarding 'mtime':
       https://apenwarr.ca/log/20181113 
    
    I'll cite a piece:
     * Is mtime always nonzero?
      No. Various cheaply-written virtual filesystems, like many fuse-based ones, don't bother setting mtime.
    
    So I would suggest to 
      - limit the number of iterations for this cycle to prevent the endless loop;
      - implement 'touch' in a different way, e.g. by writing something into the file (this doesn't solves the problem of hypothetical
            "cheaply-written virtual filesystems" though).
    
    Here's the corresponding issue on our tracker: https://issues.tmatesoft.com/issue/SGT-1308
    -- 
    Dmitry Pavlenko,
    TMate Software,
    http://subgit.com/ - git-svn bridge
    
    
    
    _______________________________________________
    jgit-dev mailing list
    jgit-dev@xxxxxxxxxxx
    To change your delivery options, retrieve your password, or unsubscribe from this list, visit
    https://www.eclipse.org/mailman/listinfo/jgit-dev