RE: [egit-dev] Concurrent access to Git Index - synchronization required

["Baumgart, Jens" <jens.baumgart@xxxxxxx>]

We still have the two critical bugs 311051 and 308506. I'd like to discuss how to continue.
I think there a two scenarios:

1) In Process (e.g. Eclipse)

Here the user expects all threads to cooperate without error (e.g. Decorator vs. Branch operation).

2) Cross-Process (e.g. Eclipse vs. msysgit)

I think in this scenario the user accepts failing EGit operations if e.g. an msysgit operation is running concurrently on the command line.

To solve the issue in 1) I would prefer using a Java read write lock to synchronize the access on the index file. I have some doubts on the retry approach because it might happen that e.g. decoration is long-running and an operation like branch then might fail after 10 attempts.

There could be a central lock manager (singleton) managing one read/write lock per index file.
This lock manager could be used by DirCache.read / DirCache.write / GitIndex.read / GitIndex.write.
This ugly stuff could be restricted to the Windows platform or there could be a central switch to enable the use of locks.

Comments?

--Jens



-----Original Message-----
From: Shawn O. Pearce [mailto:spearce@xxxxxxxxxxx] 
Sent: Dienstag, 4. Mai 2010 18:29
To: Baumgart, Jens
Cc: egit-dev@xxxxxxxxxxx
Subject: Re: [egit-dev] Concurrent access to Git Index - synchronization required?

"Baumgart, Jens" <jens.baumgart@xxxxxxx> wrote:
> in Bug 311051 a Branch operation failed due to parallel access to the GIT index file by 2 threads:
> 
> Thread 1: Git decorator reads the index using DirCache.readFrom
> Thread 2: BranchOperation fails in GitIndex.write because it cannot delete the index file
> 
> Bug 308506 seems to be a similar problem.
> 
> My question is: do we need synchronization for accessing the index file?
> One could e.g. think of introducing a read/write lock (ReadWriteLock) to protect access to the index file.

Yikes.

A reader shouldn't need to lock the index, its atomically updated
all-or-nothing on disk.  Except that on Windows you can't replace
the file while its being read.  Hmm.

An in-JVM read/write lock can help you work around this most of the
time, but only within this JVM process.  It won't do anything for
you when an external program is running `git status` at the same
time that Eclipse is updating the index file.

I wonder if we shouldn't try waiting a very short time on challenged
filesystems (*cough* Win32 *cough*) and try again inside of the
atomic write code paths.  Basically teach LockFile to wait and
retry the replacement every 200 ms for up to 2000 ms if the original
rename or delete fails on Win32.


I think I'm fine with adding some sort of read-write lock to
Repository to fairly serialize readers and writers of the index
file in memory and improve their chances of success when accessing
the file.  But I think we need to look at it from the more general
world-view and account for external processes interferring with
our own access too, hence the wait-retry suggestion above.

-- 
Shawn.