Home » Modeling » EMF » [CDO] Questions about CDO capabilities
|
| Re: [CDO] Questions about CDO capabilities [message #1009522 is a reply to message #1009303] |
Fri, 15 February 2013 09:16   |
|
Am 14.02.2013 22:27, schrieb Pierre Gaufillet:
> Hello CDO team!
>
> After some time of experimentation, I have a few questions about CDO capabilities:
> - is the workspace/project feature compatible with the security manager?
I've not tested it. I imagine that some extra thought needs to be put in on the RepositorySynchronizer.
> - when using CDO projects, is there an equivalent to team/synchronize?
I think that would be this API: org.eclipse.emf.cdo.workspace.CDOWorkspace.update(CDOMerger)
> - even if the CDO server is stopped using the osgi console and 'close' command, the database is detected as "crashed"
> during the next start. Am I doing something wrong?
That shouldn't happen if you do everything right. I'd need to be able to reproduce that.
> - what about the robustness of CDO model repositories? When using XMI, it is not so difficult to fix anomalies by
> hand. But is it easy to fix a database that has been sabotaged by a rogue client, by a bug in CDO, in the database
> server or in anything else?
What would be easy? It all depends on the used IStore implementation. Often you can use the CDOServerExporter /
CDOServerImporter to analyze, change and re-import the complete contents as XML. The DBStore offers an optional
CDOServerBrowser that you can use to inspect the DB via a local web browser. Often you can also use arbitrary SQL front
ends to debug the data.
> - is there a UI to tag a set of objects?
Not even an API, although it would be a nice thing to have. We're very reluctant, though, to introduce instances of what
I call "the partial evil" problem. If/when we implement the (already existing) CDOBranchTag interface then certainly not
in a way that it would support the tagging of an arbitrary (possibly inconsistent, not self-contained) subset of the
entire object graph.
> - is there a provision to "clean up" a repository from time to time, for example by removing non tagged versions of
> objects between 2 tagged versions?
No.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Questions about CDO capabilities [message #1009539 is a reply to message #1009522] |
Fri, 15 February 2013 09:42 |
 Christophe Bouhier
Messages: 937
Registered: July 2009 |
Senior Member |
|
|
Just to add to point on DB maintenance.
On 15-02-13 10:16, Eike Stepper wrote:
> Am 14.02.2013 22:27, schrieb Pierre Gaufillet:
>> Hello CDO team!
>>
>> After some time of experimentation, I have a few questions about CDO
>> capabilities:
>> - is the workspace/project feature compatible with the security manager?
> I've not tested it. I imagine that some extra thought needs to be put in
> on the RepositorySynchronizer.
>
>> - when using CDO projects, is there an equivalent to team/synchronize?
> I think that would be this API:
> org.eclipse.emf.cdo.workspace.CDOWorkspace.update(CDOMerger)
>
>> - even if the CDO server is stopped using the osgi console and 'close'
>> command, the database is detected as "crashed" during the next start.
>> Am I doing something wrong?
> That shouldn't happen if you do everything right. I'd need to be able to
> reproduce that.
>
>> - what about the robustness of CDO model repositories? When using XMI,
>> it is not so difficult to fix anomalies by hand. But is it easy to fix
>> a database that has been sabotaged by a rogue client, by a bug in CDO,
>> in the database server or in anything else?
> What would be easy? It all depends on the used IStore implementation.
> Often you can use the CDOServerExporter / CDOServerImporter to analyze,
> change and re-import the complete contents as XML. The DBStore offers an
> optional CDOServerBrowser that you can use to inspect the DB via a local
> web browser. Often you can also use arbitrary SQL front ends to debug
> the data.
Unfortunately we were recently forced to fix anomalies what you describe
as caused by "sabotaged by rogue client",. In our case however, the
reason was a mal-configured MySQL user (Insufficient permissions) which
caused, some indexes not to be created, leading to
a DB state CDO is not built to deal with.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=396743
The trick to solve it, was to simply use a MySQL workbench, find these
anomalies and fix them.
We also have a strange case, that object are in the wrong CDOResource.
The application code didn't do that, so how did it happen? No clue to be
honest, but this also easily fixed.
Then there is the case of ObjectNotFoundException. I think that can be
considered 'normal' behaviour and has to do with consistency of the
object graph and managing references. But here also a tool to
directly peek and poke on the Objects comes in handy.
Just recently, we have done what Eike describes which is export, fix the
objects and import.. The challenge here is, it could be a multi-Gig XML
file as in our case, and you would need some tooling to check the
data offline. Perhaps an XSLT tool could execute some rules on the XML
and come up with an 'integrity' report. (We tried to do this on CDO
itself, but it's a looooooooooooong running operation).
My conclusion sofar, it's all very manageable, but some more tooling
, practice and documentation would come in handy. (Time to write them!).
BTW: CDOServerBrowser is new to me, will check that out!
Cheers Christophe
>
>> - is there a UI to tag a set of objects?
> Not even an API, although it would be a nice thing to have. We're very
> reluctant, though, to introduce instances of what I call "the partial
> evil" problem. If/when we implement the (already existing) CDOBranchTag
> interface then certainly not in a way that it would support the tagging
> of an arbitrary (possibly inconsistent, not self-contained) subset of
> the entire object graph.
>
>> - is there a provision to "clean up" a repository from time to time,
>> for example by removing non tagged versions of objects between 2
>> tagged versions?
> No.
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
|
|
|
| Re: [CDO] Questions about CDO capabilities [message #1009560 is a reply to message #1009539] |
Fri, 15 February 2013 10:10 |
|
Am 15.02.2013 10:42, schrieb Christophe Bouhier:
> Unfortunately we were recently forced to fix anomalies what you describe as caused by "sabotaged by rogue client",. In
> our case however, the reason was a mal-configured MySQL user (Insufficient permissions) which caused, some indexes not
> to be created, leading to
> a DB state CDO is not built to deal with.
>
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=396743
>
> The trick to solve it, was to simply use a MySQL workbench, find these anomalies and fix them.
>
> We also have a strange case, that object are in the wrong CDOResource.
> The application code didn't do that, so how did it happen? No clue to be honest, but this also easily fixed.
It would be good to track this down (if possible) when it happens again. I wonder if missing or wrong DB indices could
cause this?
> Then there is the case of ObjectNotFoundException. I think that can be considered 'normal' behaviour and has to do
> with consistency of the object graph and managing references. But here also a tool to
> directly peek and poke on the Objects comes in handy.
I think it depends on how/where exactly the exceptions occur. Of course you must not call, e.g., view.getObject(id) with
the id of a detached object, or expect an ObjectNotFoundException.
If you don't configure your repository with ensureReferentialIntegrity=true you must expect "problems" when
dereferencing EReferences to detached objects. You can control the behaviour in these cases with CDOStaleReferencePolicy
and CDOInvalidationPolicy:
org.eclipse.emf.cdo.view.CDOView.Options.setInvalidationPolicy(CDOInvalidationPolicy)
org.eclipse.emf.cdo.view.CDOView.Options.setStaleReferencePolicy(CDOStaleReferencePolicy)
> Just recently, we have done what Eike describes which is export, fix the objects and import.. The challenge here is,
> it could be a multi-Gig XML file as in our case, and you would need some tooling to check the
> data offline. Perhaps an XSLT tool could execute some rules on the XML
> and come up with an 'integrity' report. (We tried to do this on CDO itself, but it's a looooooooooooong running
> operation).
What is looong? The export, the import or something else? Maybe we can do something about it...
> My conclusion sofar, it's all very manageable, but some more tooling
> , practice and documentation would come in handy. (Time to write them!).
Yes, please! :P
>
>
> BTW: CDOServerBrowser is new to me, will check that out!
Have a look at org.eclipse.emf.cdo.internal.server.bundle.CDOServerApplication.doStart():
String port = OMPlatform.INSTANCE.getProperty(PROP_BROWSER_PORT);
if (port != null)
{
container.getElement("org.eclipse.emf.cdo.server.browsers", "default", port);
}
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Questions about CDO capabilities [message #1009562 is a reply to message #1009560] |
Fri, 15 February 2013 10:19 |
Christophe Bouhier
Messages: 937
Registered: July 2009 |
Senior Member |
|
|
On 15-02-13 11:10, Eike Stepper wrote:
> Am 15.02.2013 10:42, schrieb Christophe Bouhier:
>> Unfortunately we were recently forced to fix anomalies what you
>> describe as caused by "sabotaged by rogue client",. In our case
>> however, the reason was a mal-configured MySQL user (Insufficient
>> permissions) which caused, some indexes not to be created, leading to
>> a DB state CDO is not built to deal with.
>>
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=396743
>>
>> The trick to solve it, was to simply use a MySQL workbench, find these
>> anomalies and fix them.
>>
>> We also have a strange case, that object are in the wrong CDOResource.
>> The application code didn't do that, so how did it happen? No clue to
>> be honest, but this also easily fixed.
> It would be good to track this down (if possible) when it happens again.
> I wonder if missing or wrong DB indices could cause this?
>
>> Then there is the case of ObjectNotFoundException. I think that can be
>> considered 'normal' behaviour and has to do with consistency of the
>> object graph and managing references. But here also a tool to
>> directly peek and poke on the Objects comes in handy.
> I think it depends on how/where exactly the exceptions occur. Of course
> you must not call, e.g., view.getObject(id) with the id of a detached
> object, or expect an ObjectNotFoundException.
>
> If you don't configure your repository with
> ensureReferentialIntegrity=true you must expect "problems" when
Yes, what I understand referential integrity comes at a (high?) cost.
so it's off in our case.
> dereferencing EReferences to detached objects. You can control the
> behaviour in these cases with CDOStaleReferencePolicy and
> CDOInvalidationPolicy:
>
> org.eclipse.emf.cdo.view.CDOView.Options.setInvalidationPolicy(CDOInvalidationPolicy)
>
Yes, the Relaxed policy solved many UI issues related to this:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=303868
(this bug is still marked *New* BTW :P)
> org.eclipse.emf.cdo.view.CDOView.Options.setStaleReferencePolicy(CDOStaleReferencePolicy)
>
>
>> Just recently, we have done what Eike describes which is export, fix
>> the objects and import.. The challenge here is, it could be a
>> multi-Gig XML file as in our case, and you would need some tooling to
>> check the
>> data offline. Perhaps an XSLT tool could execute some rules on the XML
>> and come up with an 'integrity' report. (We tried to do this on CDO
>> itself, but it's a looooooooooooong running operation).
> What is looong? The export, the import or something else? Maybe we can
> do something about it...
>
>> My conclusion sofar, it's all very manageable, but some more tooling
>> , practice and documentation would come in handy. (Time to write them!).
> Yes, please! :P
>>
>>
>> BTW: CDOServerBrowser is new to me, will check that out!
> Have a look at
> org.eclipse.emf.cdo.internal.server.bundle.CDOServerApplication.doStart():
>
> String port = OMPlatform.INSTANCE.getProperty(PROP_BROWSER_PORT);
> if (port != null)
> {
> container.getElement("org.eclipse.emf.cdo.server.browsers",
> "default", port);
> }
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
|
|
|
| Re: [CDO] Questions about CDO capabilities [message #1009596 is a reply to message #1009562] |
Fri, 15 February 2013 11:07 |
|
Am 15.02.2013 11:19, schrieb Christophe Bouhier:
>
>>
>> If you don't configure your repository with
>> ensureReferentialIntegrity=true you must expect "problems" when
>
> Yes, what I understand referential integrity comes at a (high?) cost.
> so it's off in our case.
It requires SELECTs on multiple (possibly many) tables. The actual number of tables to consult is carefully optimized by
means of static model analysis, see XRefsQueryHandler.collectSourceCandidates(IView, Collection<EClass>, Map<EClass,
List<EReference>>).
>
>> dereferencing EReferences to detached objects. You can control the
>> behaviour in these cases with CDOStaleReferencePolicy and
>> CDOInvalidationPolicy:
>>
>> org.eclipse.emf.cdo.view.CDOView.Options.setInvalidationPolicy(CDOInvalidationPolicy)
>>
> Yes, the Relaxed policy solved many UI issues related to this:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=303868
In 4.2 they're the defaults:
public static final CDOStaleReferencePolicy DEFAULT = PROXY;
public static final CDOInvalidationPolicy DEFAULT = RELAXED;
>
> (this bug is still marked *New* BTW :P)
I can't see any votes on this Platform UI bug ;-(
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
| | |
Goto Forum:
Current Time: Tue Apr 23 11:18:02 GMT 2024
Powered by FUDForum. Page generated in 0.04298 seconds |
] 
Search
Help
Register
Login
Home
