| Re: [stellation-res] A database proposal, for fixing 22135. |
On Fri, Aug 09, 2002 at 09:20:56PM +0000, Mark C. Chu-Carroll wrote: > On Friday 09 August 2002 03:18 pm, Florin Iucha wrote: > > No matter how you smart you create an AST and how smart the > > serialization mechanism is, you still end up with a bucket of bits and > > you still need to parse (even if you call it "unserialization") that to > > get the AST back in memory. > > That's what I meant by a "happy medium" point - where you have > a storage mechanism that maps a significant amount of artifact > structure into the database structure - so your structure is encoded > partially into chunks that need to be parsed, and part of the structure > is explicitly represented by the layout structure in the database. Good point. > > > > Why not store the artifact data in the file system and keep a cookie in > > > > the database? > > > > > > Because file systems are non-transactional. > > > > Yet. By year's end, reiserfs will have transactions implemented at the > > file system level. > > But we don't want Stellation to only be reliable on ReiserFS. We > want a system that's safe, robust, and reliable on all versions > of Unix, regardless of what filesystem you choose; and on windows, > regardless of what filesystem you use. safe, robust, windows. Choose two 8^) > > > Transactionality is one > > > of the most under-appreciated, under-estimated pieces of functionality > > > that you get from using a database. Transactionality is *the* key to > > > keeping your code safe in the repository. > > > > > > Databases give you transactionality for free. > > > > Not exactly "free". It costs more to insert rows in a database than to > > dump data from the network socket into a buffer and mmap that to the > > disk. > > I meant free in the sense of "not requiring additional code or design > work". Hrm... and stuffing all data in a XML document using DOM and squeezing that through a socked is free in the same sense. But BAD. > > > You *can* implement > > > transactionality in a filesystem. But to do it *right* is very hard. To > > > be *sure* that you've got it completely right - that you haven't missed > > > any end-cases, that there's no scenario where an interrupted operation > > > will corrupt data in your filesystem - that's very, very hard. For every > > > hundred people who've tried to implement transactionality, more than > > > 99 of them get it wrong. And you don't know that you got it wrong until > > > some data gets lost. > > > > > > I don't trust anyone involved in this project to really implement > > > transactionality *right*. That's not a dig at anyone involved in > > > the project. It's a reflection of how hard the problem is - and none > > > of us are experts on the subject. > > > > Except that you don't need to solve the transactionality problem in the > > general case, but in a special case: > > > > > > for a SCM, no information is ever _changed_ - information is only > > _added_ to a repository > > > > Adding stuff transactionally is much easier than updating/deleting it. > > > > Note, I am talking here about the artifact data, not metadata. > > Easi*er* can be very different from easy. Even when you're > assuming strict additivity - to make absolutely sure that things are safe > at all times is not a simple task. > > But even if we can implement additive transactionality - is that > really good enough? I don't really want to commit everyone > who ever extends Stellation to strict additivity. Additivity is fine > for code. But it's not fine for metadata. Metadata is very > important for the long-term goals of Stellation, and the line > between primary data and metadata can become extremely fuzzy. Please read my previous sentence again. > I strongly believe that the advantages of the relational database > are important enough that we should be using it. If a storage > format in the database turns out to be overly wasteful of space, ... and time... > I'd rather spend the time to change the schema so that it isn't > so bad, than to give up on the database for storage. OK. Let's move on. florin -- "If it's not broken, let's fix it till it is." 41A9 2BDE 8E11 F1C5 87A6 03EE 34B3 E075 3B90 DFE4
Attachment:
pgpGbl6woKs2P.pgp
Description: PGP signature