Re: Project Model Improvements Re: [cdt-dev] CDT Summit Report
I agree that a proper relational database is a better solution for storing and retrieving data than an XML file. However there may be a problem with going this route. Users should be able to check their entire project into version control, including the .project and .cproject files. (There are problems with sharing .cproject, but ideally it should work.) Now, if we use a relational db instead, I'd assume the db would be stored in some kind of binary file, and that may be a problem for some version control systems.
So in the end we may be stuck using a text based format. I don't think XML is inherently evil, its just the way we are handing the XML that is clearly flawed. I'm not very experienced in this area but there may be better approaches to processing XML, like xpath queries or something. Another solution might be to write a SAX parser that directly builds the project description AST, without loading the entire DOM into memory.
There are ways of dealing with the problem of a crash during serialization. Whenever a change needs to be saved we could first rename the .cproject file, then write out a new version of the .cproject file, then delete the old one after. That way if power is cut during serialization the old file is still there.
IBM Eclipse CDT Team
"James Blackburn" ---10/01/2008 07:00:11 AM---Hi Doug,
"James Blackburn" <jamesblackburn@xxxxxxxxx>
"CDT General developers list." <cdt-dev@xxxxxxxxxxx>
10/01/2008 07:00 AM
Project Model Improvements Re: [cdt-dev] CDT Summit Report
> Not yet. We didn't get very much time on build as we did on the other
> topics. I think the summary of it is to redo the Project model to simplify
> it and to break the dependency on managed build that was introduced with it
> in 4.0.
Is there any indication on what's planned, who might be doing this
work, and in what time frame?
Time's come full circle again for me and I'm once again focusing on my
users with larger projects. The problem with the current
implementation is twofold:
1) For an XML file >3M CDT exceeds a 512M HEAP and performance is
really bad (BUG238421)
2) The current XML model is not threadsafe (get the indexer going,
open settings page and hit apply rapidly... BUG239627).
I think this all boils down to choice of XML as the data structure.
It's verbose, inherently not threadsafe and, for all its verbosity,
it's still not human readable. The tree duplication is expensive in
terms of time and memory, and the end result is that changes made to
the project description from different threads can easily be lost
And all this before we consider what happens if a powercut or crash
happens during serialization.
It's my feeling that for something as important as the project model
we actually need a data store with ACID properties with reasonable
performance -- a very lightweight db would seem to be an ideal
I've got some time in my schedule to start work on this, but am keen
not to tread on anyone else's toes if they're intending on working on
the project model. My first aim would be to port the existing project
model to use sqllite as its db backend, and any changes to the actual
structure of the model could be made in parallel or after.
If this all sounds like a really bad idea then someone please say!
Otherwise it's the only solution I can see that would ensure we have a
scalable project description with ACID properties.
cdt-dev mailing list