Home » Modeling » EMF » [CDO] Querying HREFs in the repository
| [CDO] Querying HREFs in the repository [message #928676] |
Sun, 30 September 2012 21:57  |
|
Hi,
Let's say I have objects in a CDO repository that reference objects
persisted in resources in the Eclipse workspace (e.g., XMI resources).
(I know, I know. Why would I do that? Well, let's just pretend)
Now, I want to rename or move a workspace XMI resource that contains
objects referenced by some objects in my repository. Refactoring
references from other resources in the workspace is relatively easy: I
can scan every resource in the workspace as text, line by line, using
Java's efficient I/O streams to look for URIs that seem to reference
the resource being renamed/moved, then load any resources that look
like they will be affected and save them back with new HREFs. Piece of
cake, right?
But, what do I do for the CDO repository, in which I have millions upon
millions of objects that may or may not be affected by this
refactoring? I don't relish the thought of loading every object in the
repository client-side just to see whether it references some object in
my workspace resource.
So, I'm assuming that CDO's server-side query capabilities can help.
There's a suite of CDOView::queryXRefs(...) methods, but they require a
CDOObject as the thing we want to find references to. I don't have
CDOObjects because the objects I want to find references to are
persisted in workspace XMI resources. Is there an equivalent API that
accepts any EObject as the cross-reference target, or even a URI?
Because I assume that these references in the repository are all
absolute platform:/resource scheme URIs (what else could they be?).
Can the CDOQuery API help me here? Is there a query language supported
out-of-the-box that can query cross-references by URI?
Thanks,
Christian
|
|
|
| Re: [CDO] Querying HREFs in the repository [message #929075 is a reply to message #928676] |
Mon, 01 October 2012 07:37  |
|
Hi Christian,
I've implemented a solution that's based on wrapping the external EObject in a CDOExternalObject, which can be used with
CDOView.queryXRefs():
public void testXRefExternalObject() throws Exception
{
skipStoreWithoutExternalReferences();
ResourceSet resourceSet = new ResourceSetImpl();
Map<String, Object> map = resourceSet.getResourceFactoryRegistry().getExtensionToFactoryMap();
map.put("xml", new XMLResourceFactoryImpl());
PurchaseOrder externalObject = getModel1Factory().createPurchaseOrder();
Resource externalResource = resourceSet.createResource(URI.createFileURI("/com/foo/bar.xml"));
externalResource.getContents().add(externalObject);
CDOSession session = openSession();
CDOTransaction transaction = session.openTransaction(resourceSet);
Supplier supplier = getModel1Factory().createSupplier();
supplier.getPurchaseOrders().add(externalObject);
CDOResource resource = transaction.createResource(getResourcePath("/internal"));
resource.getContents().add(supplier);
transaction.commit();
CDOObject wrapper = CDOUtil.wrapExternalObject(externalObject, transaction);
List<CDOObjectReference> xRefs = transaction.queryXRefs(wrapper);
assertEquals(1, xRefs.size());
assertEquals(supplier, xRefs.get(0).getSourceObject());
}
If that suits your needs please submit a bugzilla so that I can commit the enhancement.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Am 30.09.2012 23:57, schrieb Christian W. Damus:
> Hi,
>
> Let's say I have objects in a CDO repository that reference objects persisted in resources in the Eclipse workspace
> (e.g., XMI resources).
>
> (I know, I know. Why would I do that? Well, let's just pretend)
>
> Now, I want to rename or move a workspace XMI resource that contains objects referenced by some objects in my
> repository. Refactoring references from other resources in the workspace is relatively easy: I can scan every
> resource in the workspace as text, line by line, using Java's efficient I/O streams to look for URIs that seem to
> reference the resource being renamed/moved, then load any resources that look like they will be affected and save them
> back with new HREFs. Piece of cake, right?
>
> But, what do I do for the CDO repository, in which I have millions upon millions of objects that may or may not be
> affected by this refactoring? I don't relish the thought of loading every object in the repository client-side just
> to see whether it references some object in my workspace resource.
>
> So, I'm assuming that CDO's server-side query capabilities can help.
>
> There's a suite of CDOView::queryXRefs(...) methods, but they require a CDOObject as the thing we want to find
> references to. I don't have CDOObjects because the objects I want to find references to are persisted in workspace
> XMI resources. Is there an equivalent API that accepts any EObject as the cross-reference target, or even a URI?
> Because I assume that these references in the repository are all absolute platform:/resource scheme URIs (what else
> could they be?).
>
> Can the CDOQuery API help me here? Is there a query language supported out-of-the-box that can query cross-references
> by URI?
>
> Thanks,
>
> Christian
>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
| |
| Re: [CDO] Querying HREFs in the repository [message #929480 is a reply to message #929168] |
Mon, 01 October 2012 13:44 |
|
Hi, Eike,
I woke up this morning to a sequence of Bugzilla e-mail notifications.
That's impressive customer service! Implementing features not even
requested, yet. ;-) Your help is much appreciated.
However, there is actually more to my problem than querying
cross-references to external EObjects (as incredibly useful as that
already is). The core problem is a matter of querying cross-references
to external *resources*. You see, I need to be able to find the
objects in the repository that reference *any* object in a particular
external resource. This amounts to a server-side query for
cross-reference URIs that, with the fragment trimmed off, match a given
external resource URI.
The use case is refactoring. Within the repository, all
cross-references are implemented by OIDs, so renaming or moving
resources doesn't matter. However, cross-references to external
resources are susceptible to breakage when the external resource is
renamed, moved, deleted, etc. That's why I need to find and update
objects that reference anything in some external resource.
And, now that I think some more, I wonder even how to "touch" the
affected objects so that they will update their HREFs in the
repository. In an XMI world, one would just save a resource and it
writes out new HREFs because they are always computed on-the-fly. But
with CDO, there is no "save", but only committing dirty objects. Are
these objects dirty because the URIs of objects that they reference
have changed?
Again, I prefer to keep everything in the repository as a closed world
of models, but I do want to account for use cases that persist content
in different stores. I expect the problem would be similar in
references across repositories, which is a scenario explicitly
supported by CDOXATransaction.
And note that I'm not asking for any new features yet, just whether the
tools are already available to solve my problem. If not, I will be
happy to work on a solution myself if and when I need to. It isn't
determined yet how important this use case is.
Thanks!
Christian
On 2012-10-01 09:12:29 +0000, Eike Stepper said:
> Am 01.10.2012 09:37, schrieb Eike Stepper:
>> If that suits your needs please submit a bugzilla so that I can commit
>> the enhancement.
> I already submitted this bug for you:
>
> 390804: Support external targets in CDOView.queryXRefs()
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=390804
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Querying HREFs in the repository [message #929573 is a reply to message #929480] |
Mon, 01 October 2012 15:19 |
|
Am 01.10.2012 15:44, schrieb Christian W. Damus:
> Hi, Eike,
>
> I woke up this morning to a sequence of Bugzilla e-mail notifications. That's impressive customer service!
> Implementing features not even requested, yet. ;-) Your help is much appreciated.
>
> However, there is actually more to my problem than querying cross-references to external EObjects (as incredibly
> useful as that already is). The core problem is a matter of querying cross-references to external *resources*. You
> see, I need to be able to find the objects in the repository that reference *any* object in a particular external
> resource. This amounts to a server-side query for cross-reference URIs that, with the fragment trimmed off, match a
> given external resource URI.
That sounds more like a use case for a custom CDOQuery. You have at least these 3 choices:
1) If you're using the DBStore you can create an SQL query similar to this:
CDOQuery query = transaction.createQuery("sql", "SELECT uri FROM cdo_external_refs WHERE uri LIKE ':path%' ");
query.setParameter("path", resource.getURI().path());
List<String> uris = query.getResult(Stringclass);
2) You can use an OCL query (you know better than I how to write OCL :P )
3) You can register your own IQueryHandler with the repository.
> The use case is refactoring. Within the repository, all cross-references are implemented by OIDs, so renaming or
> moving resources doesn't matter. However, cross-references to external resources are susceptible to breakage when the
> external resource is renamed, moved, deleted, etc. That's why I need to find and update objects that reference
> anything in some external resource.
I see.
> And, now that I think some more, I wonder even how to "touch" the affected objects so that they will update their
> HREFs in the repository. In an XMI world, one would just save a resource and it writes out new HREFs because they are
> always computed on-the-fly. But with CDO, there is no "save", but only committing dirty objects. Are these objects
> dirty because the URIs of objects that they reference have changed?
Interesting point. First I would try to set references again (to the same object with the new URI). I think that should
make the pointing object dirty. Can you confirm this?
> Again, I prefer to keep everything in the repository as a closed world of models, but I do want to account for use
> cases that persist content in different stores.
You shall have no other stores before CDO :P
> I expect the problem would be similar in references across repositories, which is a scenario explicitly supported by
> CDOXATransaction.
Yes, references into other repositories are no different from any other external references.
> And note that I'm not asking for any new features yet, just whether the tools are already available to solve my
> problem. If not, I will be happy to work on a solution myself if and when I need to. It isn't determined yet how
> important this use case is.
Okay, okay ;-)
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Querying HREFs in the repository [message #929639 is a reply to message #929573] |
Mon, 01 October 2012 16:20 |
|
Thanks for your reply, Eike.
It sounds like this can be implemented with the API available today. I
don't know whether the problem is interesting enough to the rest of the
world to provide an API like the Xrefs queries available in CDOView,
which would have to be supported on every back-end store with whatever
mappings are required.
A few more follow-ups are in-line, below.
Cheers,
Christian
On 2012-10-01 15:19:22 +0000, Eike Stepper said:
> Am 01.10.2012 15:44, schrieb Christian W. Damus:
>> Hi, Eike,
>>
>> I woke up this morning to a sequence of Bugzilla e-mail notifications.
>> That's impressive customer service! Implementing features not even
>> requested, yet. ;-) Your help is much appreciated.
>>
>> However, there is actually more to my problem than querying
>> cross-references to external EObjects (as incredibly useful as that
>> already is). The core problem is a matter of querying cross-references
>> to external *resources*. You see, I need to be able to find the
>> objects in the repository that reference *any* object in a particular
>> external resource. This amounts to a server-side query for
>> cross-reference URIs that, with the fragment trimmed off, match a given
>> external resource URI.
> That sounds more like a use case for a custom CDOQuery. You have at
> least these 3 choices:
>
> 1) If you're using the DBStore you can create an SQL query similar to this:
>
> CDOQuery query = transaction.createQuery("sql", "SELECT uri FROM
> cdo_external_refs WHERE uri LIKE ':path%' ");
> query.setParameter("path", resource.getURI().path());
> List<String> uris = query.getResult(Stringclass);
Oo. Gritty details of the ORM schema. I like it. :-)
> 2) You can use an OCL query (you know better than I how to write OCL :P )
Ah, but OCL doesn't have a concept of resources and URIs. It would
require extending the language, probably adding new operations to the
OclAny type or something like that.
> 3) You can register your own IQueryHandler with the repository.
I'll have to read up on IQueryHandlers.
>> The use case is refactoring. Within the repository, all
>> cross-references are implemented by OIDs, so renaming or moving
>> resources doesn't matter. However, cross-references to external
>> resources are susceptible to breakage when the external resource is
>> renamed, moved, deleted, etc. That's why I need to find and update
>> objects that reference anything in some external resource.
> I see.
>
>> And, now that I think some more, I wonder even how to "touch" the
>> affected objects so that they will update their HREFs in the
>> repository. In an XMI world, one would just save a resource and it
>> writes out new HREFs because they are always computed on-the-fly. But
>> with CDO, there is no "save", but only committing dirty objects. Are
>> these objects dirty because the URIs of objects that they reference
>> have changed?
> Interesting point. First I would try to set references again (to the
> same object with the new URI). I think that should make the pointing
> object dirty. Can you confirm this?
Sure, I'll give it a try. I would expect that "touch" notifications
don't cause objects to become dirty, which would be good in the general
case but bad for my twisted use case, but we shall see. :-)
>> Again, I prefer to keep everything in the repository as a closed world
>> of models, but I do want to account for use cases that persist content
>> in different stores.
> You shall have no other stores before CDO :P
Haha! If only it could be so.
>> I expect the problem would be similar in references across
>> repositories, which is a scenario explicitly supported by
>> CDOXATransaction.
> Yes, references into other repositories are no different from any other
> external references.
>
>> And note that I'm not asking for any new features yet, just whether the
>> tools are already available to solve my problem. If not, I will be
>> happy to work on a solution myself if and when I need to. It isn't
>> determined yet how important this use case is.
> Okay, okay ;-)
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Querying HREFs in the repository [message #929789 is a reply to message #929573] |
Mon, 01 October 2012 19:26 |
|
Hi, Eike,
As I had expected, touching an object by setting a reference to the
object that is already referenced does not dirty the referencing
object. This is consistent with EMF behaviour in general, and is a
good thing, IMO.
Removing the referenced object and putting it back (at the same index,
in the case of a list) does dirty the referencing object. The
transaction doesn't notice that the end result is the same as the
original state; internally the commit info has a CDORevisionDelta that
thinks a reference was changed from null to my external object (in
fact, it was changed from the external object to null and then back to
the external object). So, I think this bogus change was actually sent
to the repository to update the database.
It's a fairly brute-force approach to updating HREFs in the repository.
I suppose that if one is prepared to go to the effort of writing SQL
queries on the back-end to find objects that cross-reference to some
external resource URI, one could also implement a bulk update of those
HREFs in the database. The trick would be to notify connected clients
that those objects have changed, but then the references will break for
every other client because they don't have the corresponding updates in
the workspace resources.
Which is why the whole scenario is (IMHO) so problematic to begin with.
At least with cross-repository references one can expect other clients
to be connected to all of the same repositories for real-time updates.
File-based systems just seem so clunky in a CDO world.
cW
On 2012-10-01 15:19:22 +0000, Eike Stepper said:
> Am 01.10.2012 15:44, schrieb Christian W. Damus:
-------- 8< --------
>> And, now that I think some more, I wonder even how to "touch" the
>> affected objects so that they will update their HREFs in the
>> repository. In an XMI world, one would just save a resource and it
>> writes out new HREFs because they are always computed on-the-fly. But
>> with CDO, there is no "save", but only committing dirty objects. Are
>> these objects dirty because the URIs of objects that they reference
>> have changed?
> Interesting point. First I would try to set references again (to the
> same object with the new URI). I think that should make the pointing
> object dirty. Can you confirm this?
-------- 8< --------
|
|
|
| Re: [CDO] Querying HREFs in the repository [message #930217 is a reply to message #929789] |
Tue, 02 October 2012 05:37 |
|
Am 01.10.2012 21:26, schrieb Christian W. Damus:
> Hi, Eike,
>
> As I had expected, touching an object by setting a reference to the object that is already referenced does not dirty
> the referencing object. This is consistent with EMF behaviour in general, and is a good thing, IMO.
Hmm, I thought that the underlying CDORevision would end up with a different CDOID for the external object, see
CDOIDExternalImpl:
@Override
public boolean equals(Object obj)
{
if (obj == this)
{
return true;
}
// Could CDOIDTempObjectExternalImpl and CDOIDExternalImpl have the same uri. We don't want to mixed them.
if (obj != null && obj.getClass() == getClass())
{
CDOIDExternal that = (CDOIDExternal)obj;
return ObjectUtil.equals(getURI(), that.getURI());
}
return false;
}
But as you've tested it (I didn't) it seems as if equality is tested already on the EObject level.
> Removing the referenced object and putting it back (at the same index, in the case of a list) does dirty the
> referencing object. The transaction doesn't notice that the end result is the same as the original state; internally
> the commit info has a CDORevisionDelta that thinks a reference was changed from null to my external object (in fact,
> it was changed from the external object to null and then back to the external object). So, I think this bogus change
> was actually sent to the repository to update the database.
It's a known issue that redundant changes are not (always) nulled out and can lead to new revisions:
312534: Revert objects to CLEAN if new state matches clean state
https://bugs.eclipse.org/bugs/show_bug.cgi?id=312534
I consider a moved external object a different object from a repository's perspective.
> It's a fairly brute-force approach to updating HREFs in the repository. I suppose that if one is prepared to go to the
> effort of writing SQL queries on the back-end to find objects that cross-reference to some external resource URI, one
> could also implement a bulk update of those HREFs in the database. The trick would be to notify connected clients
> that those objects have changed, but then the references will break for every other client because they don't have the
> corresponding updates in the workspace resources.
A backend-independent solution would be nicer.
> Which is why the whole scenario is (IMHO) so problematic to begin with. At least with cross-repository references one
> can expect other clients to be connected to all of the same repositories for real-time updates. File-based systems
> just seem so clunky in a CDO world.
True ;-)
What about a CDO-driven URIConverter on the local ResourceSets? You could generate a URI Mapping model for CDO and store
the mappings in CDO. With a well-known location of this model within a repository you could use it to automatically
configure the URIConverter. You would move the recorded change (of external moves, renames, etc) from the pointing end
to the target end, where it belongs. I'm not very experienced with the pitfalls of URI mapping, though.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Querying HREFs in the repository [message #930985 is a reply to message #930217] |
Tue, 02 October 2012 20:31 |
|
Thanks, Eike,
See some replies in-line, below.
cW
On 2012-10-02 05:37:00 +0000, Eike Stepper said:
> Am 01.10.2012 21:26, schrieb Christian W. Damus:
>> Hi, Eike,
>>
>> As I had expected, touching an object by setting a reference to the
>> object that is already referenced does not dirty the referencing
>> object. This is consistent with EMF behaviour in general, and is a
>> good thing, IMO.
> Hmm, I thought that the underlying CDORevision would end up with a
> different CDOID for the external object, see CDOIDExternalImpl:
>
> @Override
> public boolean equals(Object obj)
> {
> if (obj == this)
> {
> return true;
> }
>
> // Could CDOIDTempObjectExternalImpl and CDOIDExternalImpl have
> the same uri. We don't want to mixed them.
> if (obj != null && obj.getClass() == getClass())
> {
> CDOIDExternal that = (CDOIDExternal)obj;
> return ObjectUtil.equals(getURI(), that.getURI());
> }
>
> return false;
> }
>
> But as you've tested it (I didn't) it seems as if equality is tested
> already on the EObject level.
I'm still on CDO 4.2 M1. Was this something that changed in your
recent fix? Perhaps I didn't actually test it ...
>> Removing the referenced object and putting it back (at the same index,
>> in the case of a list) does dirty the referencing object. The
>> transaction doesn't notice that the end result is the same as the
>> original state; internally the commit info has a CDORevisionDelta that
>> thinks a reference was changed from null to my external object (in
>> fact, it was changed from the external object to null and then back to
>> the external object). So, I think this bogus change was actually sent
>> to the repository to update the database.
> It's a known issue that redundant changes are not (always) nulled out
> and can lead to new revisions:
>
> 312534: Revert objects to CLEAN if new state matches clean state
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=312534
OK. In my case, it happens to be a useful buf, then. :-)
> I consider a moved external object a different object from a
> repository's perspective.
That makes sense, I suppose, since all that the repository knows about
the object is its URI, and that definitely changes in my scenario.
>> It's a fairly brute-force approach to updating HREFs in the repository.
>> I suppose that if one is prepared to go to the effort of writing SQL
>> queries on the back-end to find objects that cross-reference to some
>> external resource URI, one could also implement a bulk update of those
>> HREFs in the database. The trick would be to notify connected clients
>> that those objects have changed, but then the references will break for
>> every other client because they don't have the corresponding updates in
>> the workspace resources.
> A backend-independent solution would be nicer.
>
>> Which is why the whole scenario is (IMHO) so problematic to begin with.
>> At least with cross-repository references one can expect other clients
>> to be connected to all of the same repositories for real-time updates.
>> File-based systems just seem so clunky in a CDO world.
> True ;-)
>
> What about a CDO-driven URIConverter on the local ResourceSets? You
> could generate a URI Mapping model for CDO and store the mappings in
> CDO. With a well-known location of this model within a repository you
> could use it to automatically configure the URIConverter. You would
> move the recorded change (of external moves, renames, etc) from the
> pointing end to the target end, where it belongs. I'm not very
> experienced with the pitfalls of URI mapping, though.
Hunh. That's an interesting idea, actually. I'll have to give that
some thought. I think it should work. My first hesitation is that it
requires my application's special behaviour to be able to resolve
references at all, which otherwise wouldn't be necessary in a "pure"
CDO context. Definitely something to consider.
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Querying HREFs in the repository [message #931321 is a reply to message #930985] |
Wed, 03 October 2012 05:11 |
|
Am 02.10.2012 22:31, schrieb Christian W. Damus:
>>> As I had expected, touching an object by setting a reference to the object that is already referenced does not dirty
>>> the referencing object. This is consistent with EMF behaviour in general, and is a good thing, IMO.
>> Hmm, I thought that the underlying CDORevision would end up with a different CDOID for the external object, see
>> CDOIDExternalImpl:
>>
>> @Override
>> public boolean equals(Object obj)
>> {
>> if (obj == this)
>> {
>> return true;
>> }
>>
>> // Could CDOIDTempObjectExternalImpl and CDOIDExternalImpl have the same uri. We don't want to mixed them.
>> if (obj != null && obj.getClass() == getClass())
>> {
>> CDOIDExternal that = (CDOIDExternal)obj;
>> return ObjectUtil.equals(getURI(), that.getURI());
>> }
>>
>> return false;
>> }
>>
>> But as you've tested it (I didn't) it seems as if equality is tested already on the EObject level.
>
>
> I'm still on CDO 4.2 M1. Was this something that changed in your recent fix? Perhaps I didn't actually test it ...
This class has not changed a lot since it's been created in 2008. Especially no recent changes.
>
>
>>> Removing the referenced object and putting it back (at the same index, in the case of a list) does dirty the
>>> referencing object. The transaction doesn't notice that the end result is the same as the original state; internally
>>> the commit info has a CDORevisionDelta that thinks a reference was changed from null to my external object (in fact,
>>> it was changed from the external object to null and then back to the external object). So, I think this bogus
>>> change was actually sent to the repository to update the database.
>> It's a known issue that redundant changes are not (always) nulled out and can lead to new revisions:
>>
>> 312534: Revert objects to CLEAN if new state matches clean state
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=312534
>
> OK. In my case, it happens to be a useful buf, then. :-)
I just mentioned it because this "buf" can disappear eventually.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
Goto Forum:
Current Time: Fri Apr 26 21:08:41 GMT 2024
Powered by FUDForum. Page generated in 0.03584 seconds |
] 
Search
Help
Register
Login
Home
