Re: Local file system abstraction still a work in progress ? (was: [jgit

[Shawn Pearce <spearce@xxxxxxxxxxx>]

On Fri, Aug 13, 2010 at 1:21 PM, Thomas Sauzedde <yaourt@xxxxxxxxxxxxxx> wrote:
> Since I read http://dev.eclipse.org/mhonarc/lists/jgit-dev/msg00486.html
> (Git repo on Google AppEngine), I've in mind to implement a git bare
> repository with its smart http transport on GAE(Java).
>
> I didn't dive into Git / JGit internals before, so I began to learn how git
> is working and especially the smart http protocol.
>
> Now that I'm back to the code, I was wondering if Shawn's refactoring was a
> finished job (to abstract the local file system) ?

The refactoring that the message pointed at was merged into JGit's
master branch.  So its mostly present now.

> Why this question ?
> Bcz ASAIU, a lot of "interesting" code is present in the jgit.transport
> package but this package is still coupled with the local file system.

That's true for the code that needs IndexPack, which is the
ReceivePack server side code for push, and the fetch client code.  But
the server side UploadPack is fully abstracted off the local
filesystem now, and so are other interesting parts like computing a
diff.

> For example, the IndexPack class could be nearly reused as it is on GAE, but
> it use File and FileOutputstream to store packs and pack indexes ...

Right, IndexPack isn't done yet.  I haven't started to refactor it to
support a non-file storage system.  To  be honest the other stuff
burned me out, and I've been afraid of touching IndexPack, its a big
hairy class.

> So I would like to know if there are some work in progress on this topic or
> if it is considered nearly finished and so if I could modestly try to
> contribute and propose my own refactoring.

Its "most of the way there".  I know I have to do IndexPack still.
There's also a small series of patches pending for review that I came
up with related to doing bulk or asynchronous lookups against higher
latency storage systems than the local filesystem.

FWIW I do have JGit UploadPack running over smart HTTP coming from a
non-file storage system.  But I'm not yet at a point where I can open
source the implementation, I still want to get it more stable, because
its a very substantial amount of code.  It still has a lot of really
critical bugs (e.g. delta chains can create a cycle and lock up the
server chasing them), and its not fast enough... yet.

> BTW, my goal is to provide a "GAE ready" library under GPLv3, but as IANAL,
> I would like to know also if this is possible as I would provide something
> that embed / is linked to jgit ?

Yes, the EDL is compatible with most open source licenses, including
GPLv3.  You just need to abide by the notification clauses in the EDL
(which is generally quite simple, just listing the notice in
documentation/README files).

-- 
Shawn.