Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[jgit-dev] DFS reftable at Google

So I posted two long topics, reftable and dfs-reftable, because Google
is moving to reftable to back our DFS reference storage.

Due to hilarious mistakes I made ~7 years ago, our internal database
system is no longer suitable for storing huge numbers of references
(e.g. 866k) in Gerrit Code Review repositories. We could fix this and
keep using a Google database, but it doesn't solve all of the problems
that reftable solves. Or we could build reftable, include it in JGit,
and solve a lot of problems. I chose the latter. :)

One of the great features of reftable is readers will see atomic
updates of multi-ref batches. This is great for Gerrit Code Review
when a patch set is both added, and the /meta ref is updated. Readers
won't have any risk of seeing a partial state.


I'm hoping I can get git-core to also adopt reftable, such that Gerrit
Code Review can eventually recommend admins use reftable instead of
$GIT_DIR/{refs,packed-refs} for reference storage. But that is still a
work in progress.

No matter what happens in git-core, Google is moving its system to
reftable, and will be maintaining reftable within DFS.

If git-core rejects reftable, Gerrit Code Review may support reftable
as an option, but we'll continue to encourage Gerrit to be compatible
with git-core, meaning it stays with $GIT_DIR/{refs,packed-refs}. Or
switches to some other format that git-core moves to, such as LMDB.


Ketch's RefTree may be a bad idea in the long run. It doesn't scale
with references the way reftable does. I've got an unpublished draft
document how to efficiently embed reftable within a canonical tree
structure, to replace RefTree for Ketch.

I haven't published it yet because Ketch is very much a back-burner
project to us, and reftable needs to land and be stable before the
Ketch prototype could really start to depend on an embedded reftable
for the consensus system.


Back to the top