[CDO] Implement an alternative store [message #809937] |
Wed, 29 February 2012 08:18  |
Eclipse User |
|
|
|
Hi all,
I am looking into the possibility to implement an alternative IStore for CDO. At this tage I am mainly considering feasibility and the effort required. Can you point me to any useful documentation? Do you suggest to just take a look at the code of DBStore & other already existing stores?
Cheers,
Federico
|
|
|
Re: [CDO] Implement an alternative store [message #810000 is a reply to message #809937] |
Wed, 29 February 2012 09:49   |
Eclipse User |
|
|
|
Am 29.02.2012 14:18, schrieb Federico Tomassetti:
> Hi all,
> I am looking into the possibility to implement an alternative IStore for CDO. At this tage I am mainly considering
> feasibility
That's definitely feasible and I know of some who've managed.
> and the effort required.
That's impossible to tell in general, i.e., without knowing for what type of backend technology. As an example, it took
me a little less than 10 days to implement the MongoDBStore. But the effort could have been lower if MongoDB offered the
concept of transactions.
And then it depends on the set of optional CDO functionalities you want to support. Here is an excerpt from a former
conversation:
Base functionality
=================
- MongoDB implements IStore
- ID strategy (long IDs? UUID? both? pluggable?)
- Accessor management (connection pooling?)
- System info storage (creation time, first time flag, 2x ID counters, 2x branch counters, last commit time)
- Application property storage
- MongoDBAccessor implements IStoreAccessor
- Storage of package units
- Preloading of package infos
- Lazy loading of packages
- Revision loading
- Revision iteration
- Resource queries
- Commit (including TempID mapping for client-side assigned IDs)
- Rollback
--> All that is comparingly easy but high volume. The main challenge is to find a good and performant mapping to a
MongoDB schema. If we want to have this flexible to a certain degree, it could be a little harder.
Auditing
========
Also not too hard.
Branching
=========
Medium. Hardest impact probably efficient detection of duplicate resources and revision loading. Also high volume as it
normally includes merging support and changeset calculation.
Offline / Raw Access
==================
Medium. (But possibly high volume)
Partial Collection Loading
=======================
Probably very hard with MongoDB.
Automatic crash repair
====================
Easy.
Cross Referencing
=================
Hard to predict for MongoDB. How would MongoDB answer the question "What objects point through a set of given
EReferences to a set of target objects?"
Large Objects
=============
Low volume but making it efficient depends on MongoDB APIs.
Query Handler
==============
Probably easy. Depends on MongoDB API
Commit Infos
=============
Easy
--> That's it, I think. Note that some capabilites depend on each other, like "no offline clone commits without branching".
> Can you point me to any useful documentation? Do you suggest to just take a look at the code of DBStore & other
> already existing stores?
We should have more SPI documentation but as of now the existing store implementations are really your best friend in
understanding. If you have specific questions regarding those or your own attempts, don't hesitate to ask here.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
|
|
|
Re: [CDO] Implement an alternative store [message #815152 is a reply to message #815137] |
Wed, 07 March 2012 04:37   |
Eclipse User |
|
|
|
Hi Federico,
Some comments below
Federico Tomassetti escribió:
> At the moment I implemented the "write down data" part, so new
> metamodels and new revisions of objects are saved.
>
> Now it comes the next part... loading the data when the repository is
> restarted.
>
> My strategy is to load informations about revisions as soon as the store
> is loaded.
>
>
> @Override
> protected void doAfterActivate() throws Exception {
> System.out.println("LOADING STORE...");
> File dataDir = getDataDir();
> for (File file : dataDir.listFiles()){
> if (file.getName().endsWith(".emf_dump")){
> System.out.println("Loading "+file);
> String sn = file.getName();
> String regex = "CDOID_OID(\\d+)_version_(\\d+)\\.emf_dump";
> Pattern p = Pattern.compile(regex);
> Matcher m = p.matcher(sn);
> if (m.matches()){
> int cdoid = Integer.parseInt(m.group(1));
> int version = Integer.parseInt(m.group(2));
> loadCDOObject(cdoid, version, file);
> }
>
> }
> }
> }
>
>
> Does it seem reasonable?
> My problem currently is that I don't know how to instantiate
> InternalCDORevision.
My experience with custom IStores is the DB4OStore which I developed
myself. You should find there some examples of the several parts of
IStore implementation.
Regarding CDORevision instantiation, you could take a look at:
org.eclipse.emf.cdo.server.internal.db4o.DB4ORevision.getCDORevision(IStore,
DB4ORevision)
You only need to get the CDORevisionFactory instance, provided by the
IRepository:
IRepository repository = store.getRepository();
CDORevisionFactory factory =
((InternalCDORevisionManager)repository.getRevisionManager()).getFactory();
.....
InternalCDORevision revision =
(InternalCDORevision)factory.createRevision(eClass);
> @Victor maybe we can join forces
I'm working with Postgresql DBStore support at the moment, but yeah, it
should be feasible. Are you planning to publish your implementation?
Have you thought about contributing to CDO? It would be easier to
collaborate if we had it somewhere like a git repository (yours or
cdo's). Just let me know.
Cheers,
Víctor.
|
|
|
|
|
|
|
|
Re: [CDO] Implement an alternative store [message #932506 is a reply to message #932037] |
Thu, 04 October 2012 02:51  |
Eclipse User |
|
|
|
Hi Jim!
Let's say I am almost there. Actually I think the plugins could share most of the code: the hard-work is to store and load nodes as files, the actual commit/checkout commands can be added later.
If you are interested we could put our efforts together.
Federico
|
|
|
Powered by
FUDForum. Page generated in 0.05356 seconds