Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Archived » COSMOS » parsing query results from multiple MDRs
parsing query results from multiple MDRs [message #11403] Mon, 13 October 2008 20:27 Go to next message
David Loewenstern is currently offline David Loewenstern
Messages: 4
Registered: July 2009
Junior Member
I apologize if this question has been answered before, but I have not
seen it in the archives.

Say I have an application that queries two _different_ MDRs with
_different_ underlying object models. The application can be configured
so it knows which model goes with which MDR. To make this concrete,
say we have one MDR that handles personnel and a different MDR that
handles IT assets, and we need to construct queries across both (e.g.,
"list all assets that are assigned to managers with pay grade above 10").

I realize it isn't Cosmos's or CMDBf's job to tear apart the query and
route the subqueries to the appropriate MDR, and I'm not worried about
that. The problem I have is specific, and has to do with handling the
query responses.

Both MDRs will send back query responses containing records. For one,
the records will be personnel, for the other, IT assets. As far as I
can tell, the CMDBf will handle both by way of the
QueryOutputStreamHandler, ultimately calling
CMDBfServicesUtil.getRecordSerializer(). There can be only one record
serializer -- it's a class variable, so even if I have two separate
instances of QueryOutputStreamHandler, they will both use the same
record serializer. But I need two serializers, one for the personnel
data model, one for the IT data model. How do I accomplish this?

It seems that there is a problem in that QueryOutputStreamHandler
accepts a user-defined ArtifactFactory but ignores it in favor of
CommonArtifactFactory when creating most object types.

David Loewenstern
davidloe@us.ibm.com
Re: parsing query results from multiple MDRs [message #11440 is a reply to message #11403] Mon, 13 October 2008 21:18 Go to previous messageGo to next message
Eclipse User
Originally posted by: david_whiteman.us.ibm.com

Hi David,

Is it possible to create a serializer aggregate that keys off something
in the query response, perhaps a namespace, to delgate to model-specific
serializers for the actual work? That might not be an elegant solution
but might work for you in the near term. We would also like to hear
your ideas on what sort of API you would like to see, maybe some sort of
serializer/datatype mapping registry?

It sounds like the last sentence of your post is regarding a separate
problem. When you say it ignores it for most object types, can you give
an example of one for which it does, or a place you think it is doing
this?

In general, I would encourage you to create defects and/or enhancements
in our Bugzilla system where appopriate, including specific details and
recreation scenarios, as this will make it more likely that we can get
the work scheduled, and offer more timely solutions. Plus, being the
creator of the defect, you will get notified as the status changes.

Thanks,
David

David Loewenstern wrote:
> I apologize if this question has been answered before, but I have not
> seen it in the archives.
>
> Say I have an application that queries two _different_ MDRs with
> _different_ underlying object models. The application can be configured
> so it knows which model goes with which MDR. To make this concrete,
> say we have one MDR that handles personnel and a different MDR that
> handles IT assets, and we need to construct queries across both (e.g.,
> "list all assets that are assigned to managers with pay grade above 10").
>
> I realize it isn't Cosmos's or CMDBf's job to tear apart the query and
> route the subqueries to the appropriate MDR, and I'm not worried about
> that. The problem I have is specific, and has to do with handling the
> query responses.
>
> Both MDRs will send back query responses containing records. For one,
> the records will be personnel, for the other, IT assets. As far as I
> can tell, the CMDBf will handle both by way of the
> QueryOutputStreamHandler, ultimately calling
> CMDBfServicesUtil.getRecordSerializer(). There can be only one record
> serializer -- it's a class variable, so even if I have two separate
> instances of QueryOutputStreamHandler, they will both use the same
> record serializer. But I need two serializers, one for the personnel
> data model, one for the IT data model. How do I accomplish this?
>
> It seems that there is a problem in that QueryOutputStreamHandler
> accepts a user-defined ArtifactFactory but ignores it in favor of
> CommonArtifactFactory when creating most object types.
>
> David Loewenstern
> davidloe@us.ibm.com
Re: parsing query results from multiple MDRs [message #11477 is a reply to message #11440] Mon, 13 October 2008 22:19 Go to previous messageGo to next message
David Loewenstern is currently offline David Loewenstern
Messages: 4
Registered: July 2009
Junior Member
Thanks for the quick response, Dave.

I'm afraid to create defects or enhancements until I understand COSMOS a
bit better. I don't want to spam defects that have already been
addressed or that really work as designed. If we establish that I have
in fact uncovered either a defect or a feature that needs to be
addressed, I'll submit a bug report/feature request.

I thought of using a serializer aggregate but I couldn't figure out what
it would key off of. The problem is that the metadata you'd need to
know which serializer to use (e.g., a schema name or an MDR identifier)
isn't available at the point that the serializer is called.

Yes, a serializer registry could work.

The idea for the last sentence came from the javadoc for
IRecord.setValueFromString:

* Set the value for this record from a string.
* This API is needed because the value coming in
* from XML will be raw text. Implementers are
* responsible for converting this raw text to
* their format of choice.

So, if I created my own implementation of IRecord, I could have it parse
the XML appropriately, using its own serializer instead of the one
registered with CMDBfServicesUtil. But how do I get
QueryOutputStreamHandler to use my IRecord implementation? Here is line
108 of QueryOutputStreamHandler.java:

currentRecord =
CommonArtifactFactory.getInstance().createRecord(currentResu ltEntity, null);

Why isn't it this?

currentRecord = getArtifactFactory().createRecord(currentResultEntity,
null);

If it were, then I could create different artifact factories for
different MDRs, although that might not be the easiest method.

Dave


David Whiteman wrote:
> Hi David,
>
> Is it possible to create a serializer aggregate that keys off something
> in the query response, perhaps a namespace, to delgate to model-specific
> serializers for the actual work? That might not be an elegant solution
> but might work for you in the near term. We would also like to hear
> your ideas on what sort of API you would like to see, maybe some sort of
> serializer/datatype mapping registry?
>
> It sounds like the last sentence of your post is regarding a separate
> problem. When you say it ignores it for most object types, can you give
> an example of one for which it does, or a place you think it is doing this?
>
> In general, I would encourage you to create defects and/or enhancements
> in our Bugzilla system where appopriate, including specific details and
> recreation scenarios, as this will make it more likely that we can get
> the work scheduled, and offer more timely solutions. Plus, being the
> creator of the defect, you will get notified as the status changes.
>
> Thanks,
> David
>
> David Loewenstern wrote:
>> I apologize if this question has been answered before, but I have not
>> seen it in the archives.
>>
>> Say I have an application that queries two _different_ MDRs with
>> _different_ underlying object models. The application can be
>> configured so it knows which model goes with which MDR. To make this
>> concrete, say we have one MDR that handles personnel and a different
>> MDR that handles IT assets, and we need to construct queries across
>> both (e.g., "list all assets that are assigned to managers with pay
>> grade above 10").
>>
>> I realize it isn't Cosmos's or CMDBf's job to tear apart the query and
>> route the subqueries to the appropriate MDR, and I'm not worried about
>> that. The problem I have is specific, and has to do with handling the
>> query responses.
>>
>> Both MDRs will send back query responses containing records. For one,
>> the records will be personnel, for the other, IT assets. As far as I
>> can tell, the CMDBf will handle both by way of the
>> QueryOutputStreamHandler, ultimately calling
>> CMDBfServicesUtil.getRecordSerializer(). There can be only one record
>> serializer -- it's a class variable, so even if I have two separate
>> instances of QueryOutputStreamHandler, they will both use the same
>> record serializer. But I need two serializers, one for the personnel
>> data model, one for the IT data model. How do I accomplish this?
>>
>> It seems that there is a problem in that QueryOutputStreamHandler
>> accepts a user-defined ArtifactFactory but ignores it in favor of
>> CommonArtifactFactory when creating most object types.
>>
>> David Loewenstern
>> davidloe@us.ibm.com
Re: parsing query results from multiple MDRs [message #11514 is a reply to message #11477] Mon, 13 October 2008 22:44 Go to previous messageGo to next message
Eclipse User
Originally posted by: david_whiteman.us.ibm.com

Dave,

To help flesh out my idea a bit further, maybe you could post examples
of the incoming XML so we could together figure out something to key off
of. If you want to disguise your model types for confidentiality,
change them to fruits & vegetables or something.

Regarding your 2nd point, I would consider that a bug, at least without
looking at it in great detail. I authorize you to create a defect for
that. ;-) It should go against the DataCollection component. Don't
worry about picking the wrong choices, we can always fix it up. And
also don't worry if there might be a duplicate defect - we'd rather you
create it and not lose time or track of the details, and we can always
close it as a duplicate with minimal effort. In general, the Eclipse
way is to err on the side of creating the defect. For this one, please
be sure to put dlwhiteman @ us.ibm.com (without the spaces) on the cc:
line.

Regards,
David

David Loewenstern wrote:
> Thanks for the quick response, Dave.
>
> I'm afraid to create defects or enhancements until I understand COSMOS a
> bit better. I don't want to spam defects that have already been
> addressed or that really work as designed. If we establish that I have
> in fact uncovered either a defect or a feature that needs to be
> addressed, I'll submit a bug report/feature request.
>
> I thought of using a serializer aggregate but I couldn't figure out what
> it would key off of. The problem is that the metadata you'd need to
> know which serializer to use (e.g., a schema name or an MDR identifier)
> isn't available at the point that the serializer is called.
>
> Yes, a serializer registry could work.
>
> The idea for the last sentence came from the javadoc for
> IRecord.setValueFromString:
>
> * Set the value for this record from a string.
> * This API is needed because the value coming in
> * from XML will be raw text. Implementers are
> * responsible for converting this raw text to
> * their format of choice.
>
> So, if I created my own implementation of IRecord, I could have it parse
> the XML appropriately, using its own serializer instead of the one
> registered with CMDBfServicesUtil. But how do I get
> QueryOutputStreamHandler to use my IRecord implementation? Here is line
> 108 of QueryOutputStreamHandler.java:
>
> currentRecord =
> CommonArtifactFactory.getInstance().createRecord(currentResu ltEntity,
> null);
>
> Why isn't it this?
>
> currentRecord = getArtifactFactory().createRecord(currentResultEntity,
> null);
>
> If it were, then I could create different artifact factories for
> different MDRs, although that might not be the easiest method.
>
> Dave
Re: parsing query results from multiple MDRs [message #11550 is a reply to message #11514] Tue, 14 October 2008 03:35 Go to previous message
Eclipse User
Originally posted by: david_whiteman.us.ibm.com

I just realized something. You are trying to convert the XML returned
from the MDRs into POJOs. The original intent for the consumer of the
transformations was the MDRs themselves. I.e. the MDR would create the
query response as POJOs, and those would be converted into XML to be
returned to the client. Since the client can be a UI that displays the
query result as XML, we had not designed the transformation code to
handle query results from multiple MDRs.

That said, I think this is a valid usage, and we can still likely make
things work via the approach we discussed previously of inspecting the
query result XMLs and routing the conversion to one of several
domain-specific handlers.

David
Re: parsing query results from multiple MDRs [message #571798 is a reply to message #11403] Mon, 13 October 2008 21:18 Go to previous message
David Whiteman is currently offline David Whiteman
Messages: 166
Registered: July 2009
Senior Member
Hi David,

Is it possible to create a serializer aggregate that keys off something
in the query response, perhaps a namespace, to delgate to model-specific
serializers for the actual work? That might not be an elegant solution
but might work for you in the near term. We would also like to hear
your ideas on what sort of API you would like to see, maybe some sort of
serializer/datatype mapping registry?

It sounds like the last sentence of your post is regarding a separate
problem. When you say it ignores it for most object types, can you give
an example of one for which it does, or a place you think it is doing
this?

In general, I would encourage you to create defects and/or enhancements
in our Bugzilla system where appopriate, including specific details and
recreation scenarios, as this will make it more likely that we can get
the work scheduled, and offer more timely solutions. Plus, being the
creator of the defect, you will get notified as the status changes.

Thanks,
David

David Loewenstern wrote:
> I apologize if this question has been answered before, but I have not
> seen it in the archives.
>
> Say I have an application that queries two _different_ MDRs with
> _different_ underlying object models. The application can be configured
> so it knows which model goes with which MDR. To make this concrete,
> say we have one MDR that handles personnel and a different MDR that
> handles IT assets, and we need to construct queries across both (e.g.,
> "list all assets that are assigned to managers with pay grade above 10").
>
> I realize it isn't Cosmos's or CMDBf's job to tear apart the query and
> route the subqueries to the appropriate MDR, and I'm not worried about
> that. The problem I have is specific, and has to do with handling the
> query responses.
>
> Both MDRs will send back query responses containing records. For one,
> the records will be personnel, for the other, IT assets. As far as I
> can tell, the CMDBf will handle both by way of the
> QueryOutputStreamHandler, ultimately calling
> CMDBfServicesUtil.getRecordSerializer(). There can be only one record
> serializer -- it's a class variable, so even if I have two separate
> instances of QueryOutputStreamHandler, they will both use the same
> record serializer. But I need two serializers, one for the personnel
> data model, one for the IT data model. How do I accomplish this?
>
> It seems that there is a problem in that QueryOutputStreamHandler
> accepts a user-defined ArtifactFactory but ignores it in favor of
> CommonArtifactFactory when creating most object types.
>
> David Loewenstern
> davidloe@us.ibm.com
Re: parsing query results from multiple MDRs [message #571825 is a reply to message #11440] Mon, 13 October 2008 22:19 Go to previous message
David Loewenstern is currently offline David Loewenstern
Messages: 4
Registered: July 2009
Junior Member
Thanks for the quick response, Dave.

I'm afraid to create defects or enhancements until I understand COSMOS a
bit better. I don't want to spam defects that have already been
addressed or that really work as designed. If we establish that I have
in fact uncovered either a defect or a feature that needs to be
addressed, I'll submit a bug report/feature request.

I thought of using a serializer aggregate but I couldn't figure out what
it would key off of. The problem is that the metadata you'd need to
know which serializer to use (e.g., a schema name or an MDR identifier)
isn't available at the point that the serializer is called.

Yes, a serializer registry could work.

The idea for the last sentence came from the javadoc for
IRecord.setValueFromString:

* Set the value for this record from a string.
* This API is needed because the value coming in
* from XML will be raw text. Implementers are
* responsible for converting this raw text to
* their format of choice.

So, if I created my own implementation of IRecord, I could have it parse
the XML appropriately, using its own serializer instead of the one
registered with CMDBfServicesUtil. But how do I get
QueryOutputStreamHandler to use my IRecord implementation? Here is line
108 of QueryOutputStreamHandler.java:

currentRecord =
CommonArtifactFactory.getInstance().createRecord(currentResu ltEntity, null);

Why isn't it this?

currentRecord = getArtifactFactory().createRecord(currentResultEntity,
null);

If it were, then I could create different artifact factories for
different MDRs, although that might not be the easiest method.

Dave


David Whiteman wrote:
> Hi David,
>
> Is it possible to create a serializer aggregate that keys off something
> in the query response, perhaps a namespace, to delgate to model-specific
> serializers for the actual work? That might not be an elegant solution
> but might work for you in the near term. We would also like to hear
> your ideas on what sort of API you would like to see, maybe some sort of
> serializer/datatype mapping registry?
>
> It sounds like the last sentence of your post is regarding a separate
> problem. When you say it ignores it for most object types, can you give
> an example of one for which it does, or a place you think it is doing this?
>
> In general, I would encourage you to create defects and/or enhancements
> in our Bugzilla system where appopriate, including specific details and
> recreation scenarios, as this will make it more likely that we can get
> the work scheduled, and offer more timely solutions. Plus, being the
> creator of the defect, you will get notified as the status changes.
>
> Thanks,
> David
>
> David Loewenstern wrote:
>> I apologize if this question has been answered before, but I have not
>> seen it in the archives.
>>
>> Say I have an application that queries two _different_ MDRs with
>> _different_ underlying object models. The application can be
>> configured so it knows which model goes with which MDR. To make this
>> concrete, say we have one MDR that handles personnel and a different
>> MDR that handles IT assets, and we need to construct queries across
>> both (e.g., "list all assets that are assigned to managers with pay
>> grade above 10").
>>
>> I realize it isn't Cosmos's or CMDBf's job to tear apart the query and
>> route the subqueries to the appropriate MDR, and I'm not worried about
>> that. The problem I have is specific, and has to do with handling the
>> query responses.
>>
>> Both MDRs will send back query responses containing records. For one,
>> the records will be personnel, for the other, IT assets. As far as I
>> can tell, the CMDBf will handle both by way of the
>> QueryOutputStreamHandler, ultimately calling
>> CMDBfServicesUtil.getRecordSerializer(). There can be only one record
>> serializer -- it's a class variable, so even if I have two separate
>> instances of QueryOutputStreamHandler, they will both use the same
>> record serializer. But I need two serializers, one for the personnel
>> data model, one for the IT data model. How do I accomplish this?
>>
>> It seems that there is a problem in that QueryOutputStreamHandler
>> accepts a user-defined ArtifactFactory but ignores it in favor of
>> CommonArtifactFactory when creating most object types.
>>
>> David Loewenstern
>> davidloe@us.ibm.com
Re: parsing query results from multiple MDRs [message #571869 is a reply to message #11477] Mon, 13 October 2008 22:44 Go to previous message
David Whiteman is currently offline David Whiteman
Messages: 166
Registered: July 2009
Senior Member
Dave,

To help flesh out my idea a bit further, maybe you could post examples
of the incoming XML so we could together figure out something to key off
of. If you want to disguise your model types for confidentiality,
change them to fruits & vegetables or something.

Regarding your 2nd point, I would consider that a bug, at least without
looking at it in great detail. I authorize you to create a defect for
that. ;-) It should go against the DataCollection component. Don't
worry about picking the wrong choices, we can always fix it up. And
also don't worry if there might be a duplicate defect - we'd rather you
create it and not lose time or track of the details, and we can always
close it as a duplicate with minimal effort. In general, the Eclipse
way is to err on the side of creating the defect. For this one, please
be sure to put dlwhiteman @ us.ibm.com (without the spaces) on the cc:
line.

Regards,
David

David Loewenstern wrote:
> Thanks for the quick response, Dave.
>
> I'm afraid to create defects or enhancements until I understand COSMOS a
> bit better. I don't want to spam defects that have already been
> addressed or that really work as designed. If we establish that I have
> in fact uncovered either a defect or a feature that needs to be
> addressed, I'll submit a bug report/feature request.
>
> I thought of using a serializer aggregate but I couldn't figure out what
> it would key off of. The problem is that the metadata you'd need to
> know which serializer to use (e.g., a schema name or an MDR identifier)
> isn't available at the point that the serializer is called.
>
> Yes, a serializer registry could work.
>
> The idea for the last sentence came from the javadoc for
> IRecord.setValueFromString:
>
> * Set the value for this record from a string.
> * This API is needed because the value coming in
> * from XML will be raw text. Implementers are
> * responsible for converting this raw text to
> * their format of choice.
>
> So, if I created my own implementation of IRecord, I could have it parse
> the XML appropriately, using its own serializer instead of the one
> registered with CMDBfServicesUtil. But how do I get
> QueryOutputStreamHandler to use my IRecord implementation? Here is line
> 108 of QueryOutputStreamHandler.java:
>
> currentRecord =
> CommonArtifactFactory.getInstance().createRecord(currentResu ltEntity,
> null);
>
> Why isn't it this?
>
> currentRecord = getArtifactFactory().createRecord(currentResultEntity,
> null);
>
> If it were, then I could create different artifact factories for
> different MDRs, although that might not be the easiest method.
>
> Dave
Re: parsing query results from multiple MDRs [message #571894 is a reply to message #11514] Tue, 14 October 2008 03:35 Go to previous message
David Whiteman is currently offline David Whiteman
Messages: 166
Registered: July 2009
Senior Member
I just realized something. You are trying to convert the XML returned
from the MDRs into POJOs. The original intent for the consumer of the
transformations was the MDRs themselves. I.e. the MDR would create the
query response as POJOs, and those would be converted into XML to be
returned to the client. Since the client can be a UI that displays the
query result as XML, we had not designed the transformation code to
handle query results from multiple MDRs.

That said, I think this is a valid usage, and we can still likely make
things work via the approach we discussed previously of inspecting the
query result XMLs and routing the conversion to one of several
domain-specific handlers.

David
Previous Topic:parsing query results from multiple MDRs
Next Topic:How to run the example MDR plugin project ?
Goto Forum:
  


Current Time: Fri Oct 24 12:02:46 GMT 2014

Powered by FUDForum. Page generated in 0.06392 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software