Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Archived » OHF » submissionset.uniqueId generation problem
submissionset.uniqueId generation problem [message #36590] Thu, 08 November 2007 22:12 Go to next message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi,
I ran into a funny problem with the submission set's unique id generation.
I passed "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
submission set's sourceId, and the bridge generated
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
as the XDSSubmissionSet.uniqueId. This is evidenced by the following
snippet of the log entry's printout of the message transmitted:

<rim:ExternalIdentifier
identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
value="XDSSubmissionSet.sourceId"
/></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
value="XDSSubmissionSet.uniqueId" /></rim:Name></rim:ExternalIdentifier>

Unfortunately, the repository complained about the length of uniqueId in
this snippet from the response:

Schema Validation Errors:&#xa;Input did not validate against
schema:&#xa;Error: cvc-maxLength-valid: Value
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
with length = '80' is not facet-valid with respect to maxLength '64' for
type 'ShortName'

I just did a quick count of my characters and there are about 45. The
unique id generation brings the count to 80, so it adds about 35
characters. If this is typical, it means that the bridge user cannot pass
an OID for the submission set sourceId that is longer than about 29
characters. In my case, the fixed part of the OID is
"1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24 characters-
leaving me only 5 characters to generate all my OIDs for IHE testing.

I don't really know what the technical difference is between the sourceId
and the uniqueId, but does the uniqueId need to be that much longer? Can
it just be the same?

thanks,
Jesse
Re: submissionset.uniqueId generation problem [message #36624 is a reply to message #36590] Thu, 08 November 2007 22:35 Go to previous messageGo to next message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Oh, one more thing about this. The really bad part is that the bridge
responds with:
<ns4:success
xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>

It also shows a documentEntryUUID so it appears as if the document was
successfully submitted. The only way I knew there was an error was
because I was watching the log.

thanks,
Jesse

Jesse Pangburn wrote:

> Hi,
> I ran into a funny problem with the submission set's unique id generation.
> I passed "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
> submission set's sourceId, and the bridge generated
>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
> as the XDSSubmissionSet.uniqueId. This is evidenced by the following
> snippet of the log entry's printout of the message transmitted:

> <rim:ExternalIdentifier
> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
> value="XDSSubmissionSet.sourceId"
> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
> value="XDSSubmissionSet.uniqueId" /></rim:Name></rim:ExternalIdentifier>

> Unfortunately, the repository complained about the length of uniqueId in
> this snippet from the response:

> Schema Validation Errors:&#xa;Input did not validate against
> schema:&#xa;Error: cvc-maxLength-valid: Value
>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
> with length = '80' is not facet-valid with respect to maxLength '64' for
> type 'ShortName'

> I just did a quick count of my characters and there are about 45. The
> unique id generation brings the count to 80, so it adds about 35
> characters. If this is typical, it means that the bridge user cannot pass
> an OID for the submission set sourceId that is longer than about 29
> characters. In my case, the fixed part of the OID is
> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24 characters-
> leaving me only 5 characters to generate all my OIDs for IHE testing.

> I don't really know what the technical difference is between the sourceId
> and the uniqueId, but does the uniqueId need to be that much longer? Can
> it just be the same?

> thanks,
> Jesse
Re: submissionset.uniqueId generation problem [message #36692 is a reply to message #36624] Thu, 08 November 2007 23:22 Go to previous messageGo to next message
No real name is currently offline No real nameFriend
Messages: 292
Registered: July 2009
Senior Member
Hi Jesse -

For the repository error, as you noted, this is an ebXML 2.1 issue in
that it only allows 64 charater representaion of this attribute. EbXML
3.0 (as used in XDS.b) ... extends this to 256, much more reasonable.

Is there any particular reason for such a long sourceId?

Typically, what I recommend is to use a single OID that identifies the
organization. Off of that oid, one can build an oid for each system in
the organization (the sourceId) and uniqueIds for documents from that
organization. Since the metadata tracks both the sourceId and the
document uniqueId, there is no need to build the document uniqueId off
of the sourceId oid ... but maybe this is something our code is doing.

This is a known bug that there is no particularly good workaround:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
I still am undecided as to what to do. Enforcement of the 64 character
limit at every level in OHF code seems redundant as when it is submitted
to the repository, this error will surface anyway and the end result is
some exception for our users in both cases. So ... I just need to figure
out where to draw the line.

As for the miss-reported status, this is a bug. Please submit another
report regarding this issue. Submit a second if you are not building the
uniqueId from the sourceId yourself (this is something I'll have to
discuss with Matt) and could have API implications ... or we adjust our
OID generation algorithm and start throwing exceptions in OHF with this
issue.

- Sarah



Jesse Pangburn wrote:
> Oh, one more thing about this. The really bad part is that the bridge
> responds with:
> <ns4:success
> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>
> It also shows a documentEntryUUID so it appears as if the document was
> successfully submitted. The only way I knew there was an error was
> because I was watching the log.
>
> thanks,
> Jesse
>
> Jesse Pangburn wrote:
>
>> Hi,
>> I ran into a funny problem with the submission set's unique id
>> generation. I passed "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2"
>> as the submission set's sourceId, and the bridge generated
>
> " 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>
>
>> as the XDSSubmissionSet.uniqueId. This is evidenced by the following
>> snippet of the log entry's printout of the message transmitted:
>
>
>> <rim:ExternalIdentifier
>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>
> value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>
>
>> value="XDSSubmissionSet.sourceId"
>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>
> value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>
>
>> value="XDSSubmissionSet.uniqueId" /></rim:Name></rim:ExternalIdentifier>
>
>
>> Unfortunately, the repository complained about the length of uniqueId
>> in this snippet from the response:
>
>
>> Schema Validation Errors:&#xa;Input did not validate against
>> schema:&#xa;Error: cvc-maxLength-valid: Value
>
> '1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>
>
>> with length = '80' is not facet-valid with respect to maxLength '64'
>> for type 'ShortName'
>
>
>> I just did a quick count of my characters and there are about 45. The
>> unique id generation brings the count to 80, so it adds about 35
>> characters. If this is typical, it means that the bridge user cannot
>> pass an OID for the submission set sourceId that is longer than about
>> 29 characters. In my case, the fixed part of the OID is
>> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>> characters- leaving me only 5 characters to generate all my OIDs for
>> IHE testing.
>
>
>> I don't really know what the technical difference is between the
>> sourceId and the uniqueId, but does the uniqueId need to be that much
>> longer? Can it just be the same?
>
>
>> thanks,
>> Jesse
>
>
>
Re: submissionset.uniqueId generation problem [message #36796 is a reply to message #36692] Fri, 09 November 2007 20:13 Go to previous messageGo to next message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Sarah,
For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854" is my
company OID, ".1" adds R&D department, and I added ".100" for IHE testing.
So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24 characters already.
The bridge generates the uniqueId from the sourceId by appending 35
characters (seems consistent). So 24+35=59, that leaves me with 5
characters for all my IHE testing OIDs. I would think most people are in
a similar situation?

I don't think you (the bridge) need to be throwing errors for the 64
character limit, though it wouldn't hurt, because as you said the
repository will complain. That's fine, I agree you don't need to bother.

The thing I would suggest (er... request) is that you either don't build
the uniqueId from the sourceId, or if you do- use less characters? If the
sourceId is already unique, then just adding one more character would make
it unique and different from the sourceId.

As for the success response when there's an error, I submitted Bug 209382
https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382

thanks
Jesse

Sarah Knoop wrote:

> Hi Jesse -

> For the repository error, as you noted, this is an ebXML 2.1 issue in
> that it only allows 64 charater representaion of this attribute. EbXML
> 3.0 (as used in XDS.b) ... extends this to 256, much more reasonable.

> Is there any particular reason for such a long sourceId?

> Typically, what I recommend is to use a single OID that identifies the
> organization. Off of that oid, one can build an oid for each system in
> the organization (the sourceId) and uniqueIds for documents from that
> organization. Since the metadata tracks both the sourceId and the
> document uniqueId, there is no need to build the document uniqueId off
> of the sourceId oid ... but maybe this is something our code is doing.

> This is a known bug that there is no particularly good workaround:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
> I still am undecided as to what to do. Enforcement of the 64 character
> limit at every level in OHF code seems redundant as when it is submitted
> to the repository, this error will surface anyway and the end result is
> some exception for our users in both cases. So ... I just need to figure
> out where to draw the line.

> As for the miss-reported status, this is a bug. Please submit another
> report regarding this issue. Submit a second if you are not building the
> uniqueId from the sourceId yourself (this is something I'll have to
> discuss with Matt) and could have API implications ... or we adjust our
> OID generation algorithm and start throwing exceptions in OHF with this
> issue.

> - Sarah



> Jesse Pangburn wrote:
>> Oh, one more thing about this. The really bad part is that the bridge
>> responds with:
>> <ns4:success
>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>
>> It also shows a documentEntryUUID so it appears as if the document was
>> successfully submitted. The only way I knew there was an error was
>> because I was watching the log.
>>
>> thanks,
>> Jesse
>>
>> Jesse Pangburn wrote:
>>
>>> Hi,
>>> I ran into a funny problem with the submission set's unique id
>>> generation. I passed "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2"
>>> as the submission set's sourceId, and the bridge generated
>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>
>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the following
>>> snippet of the log entry's printout of the message transmitted:
>>
>>
>>> <rim:ExternalIdentifier
>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>
>>> value="XDSSubmissionSet.sourceId"
>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>
>>> value="XDSSubmissionSet.uniqueId" /></rim:Name></rim:ExternalIdentifier>
>>
>>
>>> Unfortunately, the repository complained about the length of uniqueId
>>> in this snippet from the response:
>>
>>
>>> Schema Validation Errors:&#xa;Input did not validate against
>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>
>>> with length = '80' is not facet-valid with respect to maxLength '64'
>>> for type 'ShortName'
>>
>>
>>> I just did a quick count of my characters and there are about 45. The
>>> unique id generation brings the count to 80, so it adds about 35
>>> characters. If this is typical, it means that the bridge user cannot
>>> pass an OID for the submission set sourceId that is longer than about
>>> 29 characters. In my case, the fixed part of the OID is
>>> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>> characters- leaving me only 5 characters to generate all my OIDs for
>>> IHE testing.
>>
>>
>>> I don't really know what the technical difference is between the
>>> sourceId and the uniqueId, but does the uniqueId need to be that much
>>> longer? Can it just be the same?
>>
>>
>>> thanks,
>>> Jesse
>>
>>
>>
Re: submissionset.uniqueId generation problem [message #36932 is a reply to message #36796] Mon, 12 November 2007 01:32 Go to previous messageGo to next message
No real name is currently offline No real nameFriend
Messages: 292
Registered: July 2009
Senior Member
Hi Jesse,
I have committed a fix for this bug (below) to OHF CVS. Please update
from there. This issue highlighted the fact that the bridge allows only
"success' and 'failure' status while IHE allows an additional 'partial
success' status. This will necessitate an API change in the bridge.
Another bug will be opened to address this: 209441.

For the OID problem, our difficulty lies in ensuring that the generated
uniqueID is truly unique (at least with high probability). For those 35
characters we use a combination of machine MAC address and the system
time in milli's, and then a sequence number, in case these are generated
in rapid succession. We use the sourceId as a base to further ensure
uniqueness. Currently the bridge is installed locally by each vendor,
but it's intent is to be a centralized service for many users. Hence the
precautions and why using the source ID will help with uniqueness.

In my note below, I was a bit confused (I read your email too fast). So
for this reason above, I'd like to keep the correlation between sourceId
and submission set unique id. But, IHE does not require it and (now
arguing with my self :-)) as I stated below the metadata preserves this
correlation in the XML, so there is no need to do so for that reason.

I'll discuss with Matt. It's a simple change, as we have a method to
generate OIDs using a default prefix (something like 1.2.8). But, I'm
still torn as to what is the best approach and what we will end up with
in terms of uniqueness, in the end.

Does the community have an opinion on this one??
- Sarah


Jesse Pangburn wrote:
> Hi Sarah,
> For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854" is
> my company OID, ".1" adds R&D department, and I added ".100" for IHE
> testing. So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24 characters
> already. The bridge generates the uniqueId from the sourceId by
> appending 35 characters (seems consistent). So 24+35=59, that leaves me
> with 5 characters for all my IHE testing OIDs. I would think most
> people are in a similar situation?
>
> I don't think you (the bridge) need to be throwing errors for the 64
> character limit, though it wouldn't hurt, because as you said the
> repository will complain. That's fine, I agree you don't need to bother.
>
> The thing I would suggest (er... request) is that you either don't build
> the uniqueId from the sourceId, or if you do- use less characters? If
> the sourceId is already unique, then just adding one more character
> would make it unique and different from the sourceId.
>
> As for the success response when there's an error, I submitted Bug
> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>
> thanks
> Jesse
>
> Sarah Knoop wrote:
>
>> Hi Jesse -
>
>
>> For the repository error, as you noted, this is an ebXML 2.1 issue in
>> that it only allows 64 charater representaion of this attribute. EbXML
>> 3.0 (as used in XDS.b) ... extends this to 256, much more reasonable.
>
>
>> Is there any particular reason for such a long sourceId?
>
>
>> Typically, what I recommend is to use a single OID that identifies
>> the organization. Off of that oid, one can build an oid for each
>> system in the organization (the sourceId) and uniqueIds for documents
>> from that organization. Since the metadata tracks both the sourceId
>> and the document uniqueId, there is no need to build the document
>> uniqueId off of the sourceId oid ... but maybe this is something our
>> code is doing.
>
>
>> This is a known bug that there is no particularly good workaround:
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>> I still am undecided as to what to do. Enforcement of the 64 character
>> limit at every level in OHF code seems redundant as when it is
>> submitted to the repository, this error will surface anyway and the
>> end result is some exception for our users in both cases. So ... I
>> just need to figure out where to draw the line.
>
>
>> As for the miss-reported status, this is a bug. Please submit another
>> report regarding this issue. Submit a second if you are not building
>> the uniqueId from the sourceId yourself (this is something I'll have
>> to discuss with Matt) and could have API implications ... or we adjust
>> our OID generation algorithm and start throwing exceptions in OHF with
>> this issue.
>
>
>> - Sarah
>
>
>
>
>> Jesse Pangburn wrote:
>>
>>> Oh, one more thing about this. The really bad part is that the
>>> bridge responds with:
>>> <ns4:success
>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>
>>> It also shows a documentEntryUUID so it appears as if the document
>>> was successfully submitted. The only way I knew there was an error
>>> was because I was watching the log.
>>>
>>> thanks,
>>> Jesse
>>>
>>> Jesse Pangburn wrote:
>>>
>>>> Hi,
>>>> I ran into a funny problem with the submission set's unique id
>>>> generation. I passed
>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the submission
>>>> set's sourceId, and the bridge generated
>>>
>>>
>>>
> " 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>
>
>>>
>>>
>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>> following snippet of the log entry's printout of the message
>>>> transmitted:
>>>
>>>
>>>
>>>> <rim:ExternalIdentifier
>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>
>>>
>>>
> value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>
>
>>>
>>>
>>>> value="XDSSubmissionSet.sourceId"
>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>
>>>
>>>
> value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>
>
>>>
>>>
>>>> value="XDSSubmissionSet.uniqueId"
>>>> /></rim:Name></rim:ExternalIdentifier>
>>>
>>>
>>>
>>>> Unfortunately, the repository complained about the length of
>>>> uniqueId in this snippet from the response:
>>>
>>>
>>>
>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>
>>>
>>>
> '1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>
>
>>>
>>>
>>>> with length = '80' is not facet-valid with respect to maxLength '64'
>>>> for type 'ShortName'
>>>
>>>
>>>
>>>> I just did a quick count of my characters and there are about 45.
>>>> The unique id generation brings the count to 80, so it adds about 35
>>>> characters. If this is typical, it means that the bridge user
>>>> cannot pass an OID for the submission set sourceId that is longer
>>>> than about 29 characters. In my case, the fixed part of the OID is
>>>> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>>> characters- leaving me only 5 characters to generate all my OIDs for
>>>> IHE testing.
>>>
>>>
>>>
>>>> I don't really know what the technical difference is between the
>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>> much longer? Can it just be the same?
>>>
>>>
>>>
>>>> thanks,
>>>> Jesse
>>>
>>>
>>>
>>>
>
>
Re: submissionset.uniqueId generation problem [message #37000 is a reply to message #36932] Mon, 12 November 2007 17:57 Go to previous messageGo to next message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Sarah,
This is probably a dumb question, but what would "partial success" mean in
this case? Why is it not just a failure?

As for the sourceId, I just looked that up in the ITI TF Vol 2 on page 122
in the Rev 3.0 final text version. Apparently it doesn't need to be
unique- it identifies the source system that produced a document. From
the PDF describing the submission set sourceId:
*********
OID identifying the instance of the Document Source that
contributed the Submission Set. When a "broker" is involved
in sending submission sets from a collection of client systems,
it should use a different source ID for submissions from each
separate system to allow for tracking.
*********

So, in light of this, it seems that the bridge should have another
parameter for sourceId. Otherwise there's no way for a client of the
bridge to identify the source ID of the system that created it. In fact,
perhaps you could remove the uniqueId parameter and replace it with the
sourceId. Then we could pass a sourceId indicating the system that
generated a request, and you could use your current algorithm to generate
a uniqueId from that sourceId instead of the other way around. In this
way, the sourceId would truly indicate a source system, and the uniqueId
would be unique. It should leave enough characters for us to uniquely
identify systems but keep the uniqueId under the 64 character limit.

Or you could just take sourceId and uniqueId as parameters and let the
clients worry about passing what they want. Either way is good for me.
Anyone else want to chime in?

thanks,
Jesse
Sarah Knoop wrote:

> Hi Jesse,
> I have committed a fix for this bug (below) to OHF CVS. Please update
> from there. This issue highlighted the fact that the bridge allows only
> "success' and 'failure' status while IHE allows an additional 'partial
> success' status. This will necessitate an API change in the bridge.
> Another bug will be opened to address this: 209441.

> For the OID problem, our difficulty lies in ensuring that the generated
> uniqueID is truly unique (at least with high probability). For those 35
> characters we use a combination of machine MAC address and the system
> time in milli's, and then a sequence number, in case these are generated
> in rapid succession. We use the sourceId as a base to further ensure
> uniqueness. Currently the bridge is installed locally by each vendor,
> but it's intent is to be a centralized service for many users. Hence the
> precautions and why using the source ID will help with uniqueness.

> In my note below, I was a bit confused (I read your email too fast). So
> for this reason above, I'd like to keep the correlation between sourceId
> and submission set unique id. But, IHE does not require it and (now
> arguing with my self :-)) as I stated below the metadata preserves this
> correlation in the XML, so there is no need to do so for that reason.

> I'll discuss with Matt. It's a simple change, as we have a method to
> generate OIDs using a default prefix (something like 1.2.8). But, I'm
> still torn as to what is the best approach and what we will end up with
> in terms of uniqueness, in the end.

> Does the community have an opinion on this one??
> - Sarah


> Jesse Pangburn wrote:
>> Hi Sarah,
>> For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854" is
>> my company OID, ".1" adds R&D department, and I added ".100" for IHE
>> testing. So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24 characters
>> already. The bridge generates the uniqueId from the sourceId by
>> appending 35 characters (seems consistent). So 24+35=59, that leaves me
>> with 5 characters for all my IHE testing OIDs. I would think most
>> people are in a similar situation?
>>
>> I don't think you (the bridge) need to be throwing errors for the 64
>> character limit, though it wouldn't hurt, because as you said the
>> repository will complain. That's fine, I agree you don't need to bother.
>>
>> The thing I would suggest (er... request) is that you either don't build
>> the uniqueId from the sourceId, or if you do- use less characters? If
>> the sourceId is already unique, then just adding one more character
>> would make it unique and different from the sourceId.
>>
>> As for the success response when there's an error, I submitted Bug
>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>
>> thanks
>> Jesse
>>
>> Sarah Knoop wrote:
>>
>>> Hi Jesse -
>>
>>
>>> For the repository error, as you noted, this is an ebXML 2.1 issue in
>>> that it only allows 64 charater representaion of this attribute. EbXML
>>> 3.0 (as used in XDS.b) ... extends this to 256, much more reasonable.
>>
>>
>>> Is there any particular reason for such a long sourceId?
>>
>>
>>> Typically, what I recommend is to use a single OID that identifies
>>> the organization. Off of that oid, one can build an oid for each
>>> system in the organization (the sourceId) and uniqueIds for documents
>>> from that organization. Since the metadata tracks both the sourceId
>>> and the document uniqueId, there is no need to build the document
>>> uniqueId off of the sourceId oid ... but maybe this is something our
>>> code is doing.
>>
>>
>>> This is a known bug that there is no particularly good workaround:
>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>> I still am undecided as to what to do. Enforcement of the 64 character
>>> limit at every level in OHF code seems redundant as when it is
>>> submitted to the repository, this error will surface anyway and the
>>> end result is some exception for our users in both cases. So ... I
>>> just need to figure out where to draw the line.
>>
>>
>>> As for the miss-reported status, this is a bug. Please submit another
>>> report regarding this issue. Submit a second if you are not building
>>> the uniqueId from the sourceId yourself (this is something I'll have
>>> to discuss with Matt) and could have API implications ... or we adjust
>>> our OID generation algorithm and start throwing exceptions in OHF with
>>> this issue.
>>
>>
>>> - Sarah
>>
>>
>>
>>
>>> Jesse Pangburn wrote:
>>>
>>>> Oh, one more thing about this. The really bad part is that the
>>>> bridge responds with:
>>>> <ns4:success
>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>
>>>> It also shows a documentEntryUUID so it appears as if the document
>>>> was successfully submitted. The only way I knew there was an error
>>>> was because I was watching the log.
>>>>
>>>> thanks,
>>>> Jesse
>>>>
>>>> Jesse Pangburn wrote:
>>>>
>>>>> Hi,
>>>>> I ran into a funny problem with the submission set's unique id
>>>>> generation. I passed
>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the submission
>>>>> set's sourceId, and the bridge generated
>>>>
>>>>
>>>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>
>>>>
>>>>
>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>> following snippet of the log entry's printout of the message
>>>>> transmitted:
>>>>
>>>>
>>>>
>>>>> <rim:ExternalIdentifier
>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>
>>>>
>>>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>
>>>>
>>>>
>>>>> value="XDSSubmissionSet.sourceId"
>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>
>>>>
>>>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>
>>>>
>>>>
>>>>> value="XDSSubmissionSet.uniqueId"
>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>
>>>>
>>>>
>>>>> Unfortunately, the repository complained about the length of
>>>>> uniqueId in this snippet from the response:
>>>>
>>>>
>>>>
>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>
>>>>
>>>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>
>>>>
>>>>
>>>>> with length = '80' is not facet-valid with respect to maxLength '64'
>>>>> for type 'ShortName'
>>>>
>>>>
>>>>
>>>>> I just did a quick count of my characters and there are about 45.
>>>>> The unique id generation brings the count to 80, so it adds about 35
>>>>> characters. If this is typical, it means that the bridge user
>>>>> cannot pass an OID for the submission set sourceId that is longer
>>>>> than about 29 characters. In my case, the fixed part of the OID is
>>>>> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>>>> characters- leaving me only 5 characters to generate all my OIDs for
>>>>> IHE testing.
>>>>
>>>>
>>>>
>>>>> I don't really know what the technical difference is between the
>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>> much longer? Can it just be the same?
>>>>
>>>>
>>>>
>>>>> thanks,
>>>>> Jesse
>>>>
>>>>
>>>>
>>>>
>>
>>
Re: submissionset.uniqueId generation problem [message #37069 is a reply to message #37000] Tue, 13 November 2007 19:55 Go to previous messageGo to next message
No real name is currently offline No real nameFriend
Messages: 292
Registered: July 2009
Senior Member
Hi Jesse,

Not a "stupid question" :-). The new profile XCA utilizes the same
transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
profile a query can be farmed out to many servers and the responses
aggregated. If one of the many servers is down, say, then this will
return a 'failure' while the others return 'success'. Hence the 'partial
success status' ... Now this is not going to be the case for submitting
the document, but to keep things consistent across the bridge api, I'd
like all three status to be available.

I think your suggestion below, to open the bridge api for both sourceId
and uniqueID is the best suggestion. We'll role that in with the next
set of API changes needed, resulting from your other bugs :-). We will
also need to make updates per the specification section you have quoted
below. We'll have to think how best to implement (document) this.

Thanks for being a great tester:-)
- Sarah




Jesse Pangburn wrote:
> Hi Sarah,
> This is probably a dumb question, but what would "partial success" mean
> in this case? Why is it not just a failure?
>
> As for the sourceId, I just looked that up in the ITI TF Vol 2 on page
> 122 in the Rev 3.0 final text version. Apparently it doesn't need to be
> unique- it identifies the source system that produced a document. From
> the PDF describing the submission set sourceId:
> *********
> OID identifying the instance of the Document Source that
> contributed the Submission Set. When a "broker" is involved
> in sending submission sets from a collection of client systems,
> it should use a different source ID for submissions from each
> separate system to allow for tracking.
> *********
>
> So, in light of this, it seems that the bridge should have another
> parameter for sourceId. Otherwise there's no way for a client of the
> bridge to identify the source ID of the system that created it. In
> fact, perhaps you could remove the uniqueId parameter and replace it
> with the sourceId. Then we could pass a sourceId indicating the system
> that generated a request, and you could use your current algorithm to
> generate a uniqueId from that sourceId instead of the other way around.
> In this way, the sourceId would truly indicate a source system, and the
> uniqueId would be unique. It should leave enough characters for us to
> uniquely identify systems but keep the uniqueId under the 64 character
> limit.
>
> Or you could just take sourceId and uniqueId as parameters and let the
> clients worry about passing what they want. Either way is good for me.
> Anyone else want to chime in?
>
> thanks,
> Jesse
> Sarah Knoop wrote:
>
>> Hi Jesse,
>> I have committed a fix for this bug (below) to OHF CVS. Please update
>> from there. This issue highlighted the fact that the bridge allows
>> only "success' and 'failure' status while IHE allows an additional
>> 'partial success' status. This will necessitate an API change in the
>> bridge. Another bug will be opened to address this: 209441.
>
>
>> For the OID problem, our difficulty lies in ensuring that the
>> generated uniqueID is truly unique (at least with high probability).
>> For those 35 characters we use a combination of machine MAC address
>> and the system time in milli's, and then a sequence number, in case
>> these are generated in rapid succession. We use the sourceId as a base
>> to further ensure uniqueness. Currently the bridge is installed
>> locally by each vendor, but it's intent is to be a centralized service
>> for many users. Hence the precautions and why using the source ID will
>> help with uniqueness.
>
>
>> In my note below, I was a bit confused (I read your email too fast).
>> So for this reason above, I'd like to keep the correlation between
>> sourceId and submission set unique id. But, IHE does not require it
>> and (now arguing with my self :-)) as I stated below the metadata
>> preserves this correlation in the XML, so there is no need to do so
>> for that reason.
>
>
>> I'll discuss with Matt. It's a simple change, as we have a method to
>> generate OIDs using a default prefix (something like 1.2.8). But, I'm
>> still torn as to what is the best approach and what we will end up
>> with in terms of uniqueness, in the end.
>
>
>> Does the community have an opinion on this one??
>> - Sarah
>
>
>
>> Jesse Pangburn wrote:
>>
>>> Hi Sarah,
>>> For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854"
>>> is my company OID, ".1" adds R&D department, and I added ".100" for
>>> IHE testing. So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24
>>> characters already. The bridge generates the uniqueId from the
>>> sourceId by appending 35 characters (seems consistent). So 24+35=59,
>>> that leaves me with 5 characters for all my IHE testing OIDs. I
>>> would think most people are in a similar situation?
>>>
>>> I don't think you (the bridge) need to be throwing errors for the 64
>>> character limit, though it wouldn't hurt, because as you said the
>>> repository will complain. That's fine, I agree you don't need to
>>> bother.
>>>
>>> The thing I would suggest (er... request) is that you either don't
>>> build the uniqueId from the sourceId, or if you do- use less
>>> characters? If the sourceId is already unique, then just adding one
>>> more character would make it unique and different from the sourceId.
>>>
>>> As for the success response when there's an error, I submitted Bug
>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>
>>> thanks
>>> Jesse
>>>
>>> Sarah Knoop wrote:
>>>
>>>> Hi Jesse -
>>>
>>>
>>>
>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>> in that it only allows 64 charater representaion of this attribute.
>>>> EbXML 3.0 (as used in XDS.b) ... extends this to 256, much more
>>>> reasonable.
>>>
>>>
>>>
>>>> Is there any particular reason for such a long sourceId?
>>>
>>>
>>>
>>>> Typically, what I recommend is to use a single OID that identifies
>>>> the organization. Off of that oid, one can build an oid for each
>>>> system in the organization (the sourceId) and uniqueIds for
>>>> documents from that organization. Since the metadata tracks both the
>>>> sourceId and the document uniqueId, there is no need to build the
>>>> document uniqueId off of the sourceId oid ... but maybe this is
>>>> something our code is doing.
>>>
>>>
>>>
>>>> This is a known bug that there is no particularly good workaround:
>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>> I still am undecided as to what to do. Enforcement of the 64
>>>> character limit at every level in OHF code seems redundant as when
>>>> it is submitted to the repository, this error will surface anyway
>>>> and the end result is some exception for our users in both cases. So
>>>> ... I just need to figure out where to draw the line.
>>>
>>>
>>>
>>>> As for the miss-reported status, this is a bug. Please submit
>>>> another report regarding this issue. Submit a second if you are not
>>>> building the uniqueId from the sourceId yourself (this is
>>>> something I'll have to discuss with Matt) and could have API
>>>> implications ... or we adjust our OID generation algorithm and start
>>>> throwing exceptions in OHF with this issue.
>>>
>>>
>>>
>>>> - Sarah
>>>
>>>
>>>
>>>
>>>
>>>> Jesse Pangburn wrote:
>>>>
>>>>> Oh, one more thing about this. The really bad part is that the
>>>>> bridge responds with:
>>>>> <ns4:success
>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>
>>>>> It also shows a documentEntryUUID so it appears as if the document
>>>>> was successfully submitted. The only way I knew there was an error
>>>>> was because I was watching the log.
>>>>>
>>>>> thanks,
>>>>> Jesse
>>>>>
>>>>> Jesse Pangburn wrote:
>>>>>
>>>>>> Hi,
>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>> generation. I passed
>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the submission
>>>>>> set's sourceId, and the bridge generated
>>>>>
>>>>>
>>>>>
>>>>>
>>>
> " 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>
>
>>>
>>>
>>>>>
>>>>>
>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>> following snippet of the log entry's printout of the message
>>>>>> transmitted:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> <rim:ExternalIdentifier
>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>
>>>>>
>>>>>
>>>>>
>>>
> value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>
>
>>>
>>>
>>>>>
>>>>>
>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>
>>>>>
>>>>>
>>>>>
>>>
> value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>
>
>>>
>>>
>>>>>
>>>>>
>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Unfortunately, the repository complained about the length of
>>>>>> uniqueId in this snippet from the response:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>
>>>>>
>>>>>
>>>>>
>>>
> '1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>
>
>>>
>>>
>>>>>
>>>>>
>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>> '64' for type 'ShortName'
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I just did a quick count of my characters and there are about 45.
>>>>>> The unique id generation brings the count to 80, so it adds about
>>>>>> 35 characters. If this is typical, it means that the bridge user
>>>>>> cannot pass an OID for the submission set sourceId that is longer
>>>>>> than about 29 characters. In my case, the fixed part of the OID
>>>>>> is "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>>>>> characters- leaving me only 5 characters to generate all my OIDs
>>>>>> for IHE testing.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I don't really know what the technical difference is between the
>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>> much longer? Can it just be the same?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> thanks,
>>>>>> Jesse
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>
>
Re: submissionset.uniqueId generation problem [message #37800 is a reply to message #37069] Fri, 30 November 2007 19:38 Go to previous messageGo to next message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Sarah,
I tried out the latest build of the bridge and it seems this bug is not
fixed. It occurred to me that maybe there was never a bug entered for
this in bugzilla. Do you know if this is the case? Or maybe it's on your
list still but you haven't gotten to it yet?

Otherwise the new build is working great for me. Just wanted to check if
this one is on the list :-)

thanks,
Jesse

Sarah Knoop wrote:

> Hi Jesse,

> Not a "stupid question" :-). The new profile XCA utilizes the same
> transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
> profile a query can be farmed out to many servers and the responses
> aggregated. If one of the many servers is down, say, then this will
> return a 'failure' while the others return 'success'. Hence the 'partial
> success status' ... Now this is not going to be the case for submitting
> the document, but to keep things consistent across the bridge api, I'd
> like all three status to be available.

> I think your suggestion below, to open the bridge api for both sourceId
> and uniqueID is the best suggestion. We'll role that in with the next
> set of API changes needed, resulting from your other bugs :-). We will
> also need to make updates per the specification section you have quoted
> below. We'll have to think how best to implement (document) this.

> Thanks for being a great tester:-)
> - Sarah




> Jesse Pangburn wrote:
>> Hi Sarah,
>> This is probably a dumb question, but what would "partial success" mean
>> in this case? Why is it not just a failure?
>>
>> As for the sourceId, I just looked that up in the ITI TF Vol 2 on page
>> 122 in the Rev 3.0 final text version. Apparently it doesn't need to be
>> unique- it identifies the source system that produced a document. From
>> the PDF describing the submission set sourceId:
>> *********
>> OID identifying the instance of the Document Source that
>> contributed the Submission Set. When a "broker" is involved
>> in sending submission sets from a collection of client systems,
>> it should use a different source ID for submissions from each
>> separate system to allow for tracking.
>> *********
>>
>> So, in light of this, it seems that the bridge should have another
>> parameter for sourceId. Otherwise there's no way for a client of the
>> bridge to identify the source ID of the system that created it. In
>> fact, perhaps you could remove the uniqueId parameter and replace it
>> with the sourceId. Then we could pass a sourceId indicating the system
>> that generated a request, and you could use your current algorithm to
>> generate a uniqueId from that sourceId instead of the other way around.
>> In this way, the sourceId would truly indicate a source system, and the
>> uniqueId would be unique. It should leave enough characters for us to
>> uniquely identify systems but keep the uniqueId under the 64 character
>> limit.
>>
>> Or you could just take sourceId and uniqueId as parameters and let the
>> clients worry about passing what they want. Either way is good for me.
>> Anyone else want to chime in?
>>
>> thanks,
>> Jesse
>> Sarah Knoop wrote:
>>
>>> Hi Jesse,
>>> I have committed a fix for this bug (below) to OHF CVS. Please update
>>> from there. This issue highlighted the fact that the bridge allows
>>> only "success' and 'failure' status while IHE allows an additional
>>> 'partial success' status. This will necessitate an API change in the
>>> bridge. Another bug will be opened to address this: 209441.
>>
>>
>>> For the OID problem, our difficulty lies in ensuring that the
>>> generated uniqueID is truly unique (at least with high probability).
>>> For those 35 characters we use a combination of machine MAC address
>>> and the system time in milli's, and then a sequence number, in case
>>> these are generated in rapid succession. We use the sourceId as a base
>>> to further ensure uniqueness. Currently the bridge is installed
>>> locally by each vendor, but it's intent is to be a centralized service
>>> for many users. Hence the precautions and why using the source ID will
>>> help with uniqueness.
>>
>>
>>> In my note below, I was a bit confused (I read your email too fast).
>>> So for this reason above, I'd like to keep the correlation between
>>> sourceId and submission set unique id. But, IHE does not require it
>>> and (now arguing with my self :-)) as I stated below the metadata
>>> preserves this correlation in the XML, so there is no need to do so
>>> for that reason.
>>
>>
>>> I'll discuss with Matt. It's a simple change, as we have a method to
>>> generate OIDs using a default prefix (something like 1.2.8). But, I'm
>>> still torn as to what is the best approach and what we will end up
>>> with in terms of uniqueness, in the end.
>>
>>
>>> Does the community have an opinion on this one??
>>> - Sarah
>>
>>
>>
>>> Jesse Pangburn wrote:
>>>
>>>> Hi Sarah,
>>>> For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854"
>>>> is my company OID, ".1" adds R&D department, and I added ".100" for
>>>> IHE testing. So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24
>>>> characters already. The bridge generates the uniqueId from the
>>>> sourceId by appending 35 characters (seems consistent). So 24+35=59,
>>>> that leaves me with 5 characters for all my IHE testing OIDs. I
>>>> would think most people are in a similar situation?
>>>>
>>>> I don't think you (the bridge) need to be throwing errors for the 64
>>>> character limit, though it wouldn't hurt, because as you said the
>>>> repository will complain. That's fine, I agree you don't need to
>>>> bother.
>>>>
>>>> The thing I would suggest (er... request) is that you either don't
>>>> build the uniqueId from the sourceId, or if you do- use less
>>>> characters? If the sourceId is already unique, then just adding one
>>>> more character would make it unique and different from the sourceId.
>>>>
>>>> As for the success response when there's an error, I submitted Bug
>>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>>
>>>> thanks
>>>> Jesse
>>>>
>>>> Sarah Knoop wrote:
>>>>
>>>>> Hi Jesse -
>>>>
>>>>
>>>>
>>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>>> in that it only allows 64 charater representaion of this attribute.
>>>>> EbXML 3.0 (as used in XDS.b) ... extends this to 256, much more
>>>>> reasonable.
>>>>
>>>>
>>>>
>>>>> Is there any particular reason for such a long sourceId?
>>>>
>>>>
>>>>
>>>>> Typically, what I recommend is to use a single OID that identifies
>>>>> the organization. Off of that oid, one can build an oid for each
>>>>> system in the organization (the sourceId) and uniqueIds for
>>>>> documents from that organization. Since the metadata tracks both the
>>>>> sourceId and the document uniqueId, there is no need to build the
>>>>> document uniqueId off of the sourceId oid ... but maybe this is
>>>>> something our code is doing.
>>>>
>>>>
>>>>
>>>>> This is a known bug that there is no particularly good workaround:
>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>>> I still am undecided as to what to do. Enforcement of the 64
>>>>> character limit at every level in OHF code seems redundant as when
>>>>> it is submitted to the repository, this error will surface anyway
>>>>> and the end result is some exception for our users in both cases. So
>>>>> ... I just need to figure out where to draw the line.
>>>>
>>>>
>>>>
>>>>> As for the miss-reported status, this is a bug. Please submit
>>>>> another report regarding this issue. Submit a second if you are not
>>>>> building the uniqueId from the sourceId yourself (this is
>>>>> something I'll have to discuss with Matt) and could have API
>>>>> implications ... or we adjust our OID generation algorithm and start
>>>>> throwing exceptions in OHF with this issue.
>>>>
>>>>
>>>>
>>>>> - Sarah
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Jesse Pangburn wrote:
>>>>>
>>>>>> Oh, one more thing about this. The really bad part is that the
>>>>>> bridge responds with:
>>>>>> <ns4:success
>>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>>
>>>>>> It also shows a documentEntryUUID so it appears as if the document
>>>>>> was successfully submitted. The only way I knew there was an error
>>>>>> was because I was watching the log.
>>>>>>
>>>>>> thanks,
>>>>>> Jesse
>>>>>>
>>>>>> Jesse Pangburn wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>>> generation. I passed
>>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the submission
>>>>>>> set's sourceId, and the bridge generated
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>>> following snippet of the log entry's printout of the message
>>>>>>> transmitted:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> <rim:ExternalIdentifier
>>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Unfortunately, the repository complained about the length of
>>>>>>> uniqueId in this snippet from the response:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>>> '64' for type 'ShortName'
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I just did a quick count of my characters and there are about 45.
>>>>>>> The unique id generation brings the count to 80, so it adds about
>>>>>>> 35 characters. If this is typical, it means that the bridge user
>>>>>>> cannot pass an OID for the submission set sourceId that is longer
>>>>>>> than about 29 characters. In my case, the fixed part of the OID
>>>>>>> is "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>>>>>> characters- leaving me only 5 characters to generate all my OIDs
>>>>>>> for IHE testing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I don't really know what the technical difference is between the
>>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>>> much longer? Can it just be the same?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> thanks,
>>>>>>> Jesse
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: submissionset.uniqueId generation problem [message #37869 is a reply to message #37800] Mon, 03 December 2007 18:27 Go to previous messageGo to next message
Matthew DavisFriend
Messages: 269
Registered: July 2009
Senior Member
Hi Jesse,

Nope... this was the one issue that fell through the cracks after I got
back. It is now fixed and a new build is generating right now - it'll
be available for download in a couple hours.

API change is as follows:

In the "DocumentSubmissionMetadata" type, there is a new value
'submissionSetUniqueId'. The Bridge will no longer auto-generate the
SubmissionSet UniqueId based on the SourceId.
DocumentSubmissionMetadata.submissionSetUniqueId is a required value and
the Bridge will throw an exception if empty.

Thanks,
-Matt


Jesse Pangburn wrote:
> Hi Sarah,
> I tried out the latest build of the bridge and it seems this bug is not
> fixed. It occurred to me that maybe there was never a bug entered for
> this in bugzilla. Do you know if this is the case? Or maybe it's on
> your list still but you haven't gotten to it yet?
>
> Otherwise the new build is working great for me. Just wanted to check
> if this one is on the list :-)
>
> thanks,
> Jesse
>
> Sarah Knoop wrote:
>
>> Hi Jesse,
>
>> Not a "stupid question" :-). The new profile XCA utilizes the same
>> transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
>> profile a query can be farmed out to many servers and the responses
>> aggregated. If one of the many servers is down, say, then this will
>> return a 'failure' while the others return 'success'. Hence the
>> 'partial success status' ... Now this is not going to be the case for
>> submitting the document, but to keep things consistent across the
>> bridge api, I'd like all three status to be available.
>
>> I think your suggestion below, to open the bridge api for both
>> sourceId and uniqueID is the best suggestion. We'll role that in with
>> the next set of API changes needed, resulting from your other bugs
>> :-). We will also need to make updates per the specification section
>> you have quoted below. We'll have to think how best to implement
>> (document) this.
>
>> Thanks for being a great tester:-)
>> - Sarah
>
>
>
>
>> Jesse Pangburn wrote:
>>> Hi Sarah,
>>> This is probably a dumb question, but what would "partial success"
>>> mean in this case? Why is it not just a failure?
>>>
>>> As for the sourceId, I just looked that up in the ITI TF Vol 2 on
>>> page 122 in the Rev 3.0 final text version. Apparently it doesn't
>>> need to be unique- it identifies the source system that produced a
>>> document. From the PDF describing the submission set sourceId:
>>> *********
>>> OID identifying the instance of the Document Source that
>>> contributed the Submission Set. When a "broker" is involved
>>> in sending submission sets from a collection of client systems,
>>> it should use a different source ID for submissions from each
>>> separate system to allow for tracking.
>>> *********
>>>
>>> So, in light of this, it seems that the bridge should have another
>>> parameter for sourceId. Otherwise there's no way for a client of the
>>> bridge to identify the source ID of the system that created it. In
>>> fact, perhaps you could remove the uniqueId parameter and replace it
>>> with the sourceId. Then we could pass a sourceId indicating the
>>> system that generated a request, and you could use your current
>>> algorithm to generate a uniqueId from that sourceId instead of the
>>> other way around. In this way, the sourceId would truly indicate a
>>> source system, and the uniqueId would be unique. It should leave
>>> enough characters for us to uniquely identify systems but keep the
>>> uniqueId under the 64 character limit.
>>>
>>> Or you could just take sourceId and uniqueId as parameters and let
>>> the clients worry about passing what they want. Either way is good
>>> for me. Anyone else want to chime in?
>>>
>>> thanks,
>>> Jesse
>>> Sarah Knoop wrote:
>>>
>>>> Hi Jesse,
>>>> I have committed a fix for this bug (below) to OHF CVS. Please
>>>> update from there. This issue highlighted the fact that the bridge
>>>> allows only "success' and 'failure' status while IHE allows an
>>>> additional 'partial success' status. This will necessitate an API
>>>> change in the bridge. Another bug will be opened to address this:
>>>> 209441.
>>>
>>>
>>>> For the OID problem, our difficulty lies in ensuring that the
>>>> generated uniqueID is truly unique (at least with high probability).
>>>> For those 35 characters we use a combination of machine MAC address
>>>> and the system time in milli's, and then a sequence number, in case
>>>> these are generated in rapid succession. We use the sourceId as a
>>>> base to further ensure uniqueness. Currently the bridge is installed
>>>> locally by each vendor, but it's intent is to be a centralized
>>>> service for many users. Hence the precautions and why using the
>>>> source ID will help with uniqueness.
>>>
>>>
>>>> In my note below, I was a bit confused (I read your email too fast).
>>>> So for this reason above, I'd like to keep the correlation between
>>>> sourceId and submission set unique id. But, IHE does not require it
>>>> and (now arguing with my self :-)) as I stated below the metadata
>>>> preserves this correlation in the XML, so there is no need to do so
>>>> for that reason.
>>>
>>>
>>>> I'll discuss with Matt. It's a simple change, as we have a method to
>>>> generate OIDs using a default prefix (something like 1.2.8). But,
>>>> I'm still torn as to what is the best approach and what we will end
>>>> up with in terms of uniqueness, in the end.
>>>
>>>
>>>> Does the community have an opinion on this one??
>>>> - Sarah
>>>
>>>
>>>
>>>> Jesse Pangburn wrote:
>>>>
>>>>> Hi Sarah,
>>>>> For the sourceId, I don't have too much choice.
>>>>> "1.3.6.1.4.1.11854" is my company OID, ".1" adds R&D department,
>>>>> and I added ".100" for IHE testing. So I'm stuck with the
>>>>> "1.3.6.1.4.1.11854.1.100." 24 characters already. The bridge
>>>>> generates the uniqueId from the sourceId by appending 35 characters
>>>>> (seems consistent). So 24+35=59, that leaves me with 5 characters
>>>>> for all my IHE testing OIDs. I would think most people are in a
>>>>> similar situation?
>>>>>
>>>>> I don't think you (the bridge) need to be throwing errors for the
>>>>> 64 character limit, though it wouldn't hurt, because as you said
>>>>> the repository will complain. That's fine, I agree you don't need
>>>>> to bother.
>>>>>
>>>>> The thing I would suggest (er... request) is that you either don't
>>>>> build the uniqueId from the sourceId, or if you do- use less
>>>>> characters? If the sourceId is already unique, then just adding
>>>>> one more character would make it unique and different from the
>>>>> sourceId.
>>>>>
>>>>> As for the success response when there's an error, I submitted Bug
>>>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>>>
>>>>> thanks
>>>>> Jesse
>>>>>
>>>>> Sarah Knoop wrote:
>>>>>
>>>>>> Hi Jesse -
>>>>>
>>>>>
>>>>>
>>>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>>>> in that it only allows 64 charater representaion of this
>>>>>> attribute. EbXML 3.0 (as used in XDS.b) ... extends this to 256,
>>>>>> much more reasonable.
>>>>>
>>>>>
>>>>>
>>>>>> Is there any particular reason for such a long sourceId?
>>>>>
>>>>>
>>>>>
>>>>>> Typically, what I recommend is to use a single OID that
>>>>>> identifies the organization. Off of that oid, one can build an oid
>>>>>> for each system in the organization (the sourceId) and uniqueIds
>>>>>> for documents from that organization. Since the metadata tracks
>>>>>> both the sourceId and the document uniqueId, there is no need to
>>>>>> build the document uniqueId off of the sourceId oid ... but maybe
>>>>>> this is something our code is doing.
>>>>>
>>>>>
>>>>>
>>>>>> This is a known bug that there is no particularly good workaround:
>>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>>>> I still am undecided as to what to do. Enforcement of the 64
>>>>>> character limit at every level in OHF code seems redundant as when
>>>>>> it is submitted to the repository, this error will surface anyway
>>>>>> and the end result is some exception for our users in both cases.
>>>>>> So ... I just need to figure out where to draw the line.
>>>>>
>>>>>
>>>>>
>>>>>> As for the miss-reported status, this is a bug. Please submit
>>>>>> another report regarding this issue. Submit a second if you are
>>>>>> not building the uniqueId from the sourceId yourself (this is
>>>>>> something I'll have to discuss with Matt) and could have API
>>>>>> implications ... or we adjust our OID generation algorithm and
>>>>>> start throwing exceptions in OHF with this issue.
>>>>>
>>>>>
>>>>>
>>>>>> - Sarah
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Jesse Pangburn wrote:
>>>>>>
>>>>>>> Oh, one more thing about this. The really bad part is that the
>>>>>>> bridge responds with:
>>>>>>> <ns4:success
>>>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>>>
>>>>>>> It also shows a documentEntryUUID so it appears as if the
>>>>>>> document was successfully submitted. The only way I knew there
>>>>>>> was an error was because I was watching the log.
>>>>>>>
>>>>>>> thanks,
>>>>>>> Jesse
>>>>>>>
>>>>>>> Jesse Pangburn wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>>>> generation. I passed
>>>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
>>>>>>>> submission set's sourceId, and the bridge generated
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> " 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>>>> following snippet of the log entry's printout of the message
>>>>>>>> transmitted:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> <rim:ExternalIdentifier
>>>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Unfortunately, the repository complained about the length of
>>>>>>>> uniqueId in this snippet from the response:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> '1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>>>> '64' for type 'ShortName'
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> I just did a quick count of my characters and there are about
>>>>>>>> 45. The unique id generation brings the count to 80, so it adds
>>>>>>>> about 35 characters. If this is typical, it means that the
>>>>>>>> bridge user cannot pass an OID for the submission set sourceId
>>>>>>>> that is longer than about 29 characters. In my case, the fixed
>>>>>>>> part of the OID is "1.3.6.1.4.1.11854.1.100." (my IHE testing
>>>>>>>> OID) which is 24 characters- leaving me only 5 characters to
>>>>>>>> generate all my OIDs for IHE testing.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> I don't really know what the technical difference is between the
>>>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>>>> much longer? Can it just be the same?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Jesse
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>
>
Re: submissionset.uniqueId generation problem [message #37902 is a reply to message #37869] Mon, 03 December 2007 20:38 Go to previous messageGo to next message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Matt,
Thank you very much for this and all the other fixes/improvements, you
guys are awesome!

thanks,
Jesse

Matthew Davis wrote:

> Hi Jesse,

> Nope... this was the one issue that fell through the cracks after I got
> back. It is now fixed and a new build is generating right now - it'll
> be available for download in a couple hours.

> API change is as follows:

> In the "DocumentSubmissionMetadata" type, there is a new value
> 'submissionSetUniqueId'. The Bridge will no longer auto-generate the
> SubmissionSet UniqueId based on the SourceId.
> DocumentSubmissionMetadata.submissionSetUniqueId is a required value and
> the Bridge will throw an exception if empty.

> Thanks,
> -Matt


> Jesse Pangburn wrote:
>> Hi Sarah,
>> I tried out the latest build of the bridge and it seems this bug is not
>> fixed. It occurred to me that maybe there was never a bug entered for
>> this in bugzilla. Do you know if this is the case? Or maybe it's on
>> your list still but you haven't gotten to it yet?
>>
>> Otherwise the new build is working great for me. Just wanted to check
>> if this one is on the list :-)
>>
>> thanks,
>> Jesse
>>
>> Sarah Knoop wrote:
>>
>>> Hi Jesse,
>>
>>> Not a "stupid question" :-). The new profile XCA utilizes the same
>>> transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
>>> profile a query can be farmed out to many servers and the responses
>>> aggregated. If one of the many servers is down, say, then this will
>>> return a 'failure' while the others return 'success'. Hence the
>>> 'partial success status' ... Now this is not going to be the case for
>>> submitting the document, but to keep things consistent across the
>>> bridge api, I'd like all three status to be available.
>>
>>> I think your suggestion below, to open the bridge api for both
>>> sourceId and uniqueID is the best suggestion. We'll role that in with
>>> the next set of API changes needed, resulting from your other bugs
>>> :-). We will also need to make updates per the specification section
>>> you have quoted below. We'll have to think how best to implement
>>> (document) this.
>>
>>> Thanks for being a great tester:-)
>>> - Sarah
>>
>>
>>
>>
>>> Jesse Pangburn wrote:
>>>> Hi Sarah,
>>>> This is probably a dumb question, but what would "partial success"
>>>> mean in this case? Why is it not just a failure?
>>>>
>>>> As for the sourceId, I just looked that up in the ITI TF Vol 2 on
>>>> page 122 in the Rev 3.0 final text version. Apparently it doesn't
>>>> need to be unique- it identifies the source system that produced a
>>>> document. From the PDF describing the submission set sourceId:
>>>> *********
>>>> OID identifying the instance of the Document Source that
>>>> contributed the Submission Set. When a "broker" is involved
>>>> in sending submission sets from a collection of client systems,
>>>> it should use a different source ID for submissions from each
>>>> separate system to allow for tracking.
>>>> *********
>>>>
>>>> So, in light of this, it seems that the bridge should have another
>>>> parameter for sourceId. Otherwise there's no way for a client of the
>>>> bridge to identify the source ID of the system that created it. In
>>>> fact, perhaps you could remove the uniqueId parameter and replace it
>>>> with the sourceId. Then we could pass a sourceId indicating the
>>>> system that generated a request, and you could use your current
>>>> algorithm to generate a uniqueId from that sourceId instead of the
>>>> other way around. In this way, the sourceId would truly indicate a
>>>> source system, and the uniqueId would be unique. It should leave
>>>> enough characters for us to uniquely identify systems but keep the
>>>> uniqueId under the 64 character limit.
>>>>
>>>> Or you could just take sourceId and uniqueId as parameters and let
>>>> the clients worry about passing what they want. Either way is good
>>>> for me. Anyone else want to chime in?
>>>>
>>>> thanks,
>>>> Jesse
>>>> Sarah Knoop wrote:
>>>>
>>>>> Hi Jesse,
>>>>> I have committed a fix for this bug (below) to OHF CVS. Please
>>>>> update from there. This issue highlighted the fact that the bridge
>>>>> allows only "success' and 'failure' status while IHE allows an
>>>>> additional 'partial success' status. This will necessitate an API
>>>>> change in the bridge. Another bug will be opened to address this:
>>>>> 209441.
>>>>
>>>>
>>>>> For the OID problem, our difficulty lies in ensuring that the
>>>>> generated uniqueID is truly unique (at least with high probability).
>>>>> For those 35 characters we use a combination of machine MAC address
>>>>> and the system time in milli's, and then a sequence number, in case
>>>>> these are generated in rapid succession. We use the sourceId as a
>>>>> base to further ensure uniqueness. Currently the bridge is installed
>>>>> locally by each vendor, but it's intent is to be a centralized
>>>>> service for many users. Hence the precautions and why using the
>>>>> source ID will help with uniqueness.
>>>>
>>>>
>>>>> In my note below, I was a bit confused (I read your email too fast).
>>>>> So for this reason above, I'd like to keep the correlation between
>>>>> sourceId and submission set unique id. But, IHE does not require it
>>>>> and (now arguing with my self :-)) as I stated below the metadata
>>>>> preserves this correlation in the XML, so there is no need to do so
>>>>> for that reason.
>>>>
>>>>
>>>>> I'll discuss with Matt. It's a simple change, as we have a method to
>>>>> generate OIDs using a default prefix (something like 1.2.8). But,
>>>>> I'm still torn as to what is the best approach and what we will end
>>>>> up with in terms of uniqueness, in the end.
>>>>
>>>>
>>>>> Does the community have an opinion on this one??
>>>>> - Sarah
>>>>
>>>>
>>>>
>>>>> Jesse Pangburn wrote:
>>>>>
>>>>>> Hi Sarah,
>>>>>> For the sourceId, I don't have too much choice.
>>>>>> "1.3.6.1.4.1.11854" is my company OID, ".1" adds R&D department,
>>>>>> and I added ".100" for IHE testing. So I'm stuck with the
>>>>>> "1.3.6.1.4.1.11854.1.100." 24 characters already. The bridge
>>>>>> generates the uniqueId from the sourceId by appending 35 characters
>>>>>> (seems consistent). So 24+35=59, that leaves me with 5 characters
>>>>>> for all my IHE testing OIDs. I would think most people are in a
>>>>>> similar situation?
>>>>>>
>>>>>> I don't think you (the bridge) need to be throwing errors for the
>>>>>> 64 character limit, though it wouldn't hurt, because as you said
>>>>>> the repository will complain. That's fine, I agree you don't need
>>>>>> to bother.
>>>>>>
>>>>>> The thing I would suggest (er... request) is that you either don't
>>>>>> build the uniqueId from the sourceId, or if you do- use less
>>>>>> characters? If the sourceId is already unique, then just adding
>>>>>> one more character would make it unique and different from the
>>>>>> sourceId.
>>>>>>
>>>>>> As for the success response when there's an error, I submitted Bug
>>>>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>>>>
>>>>>> thanks
>>>>>> Jesse
>>>>>>
>>>>>> Sarah Knoop wrote:
>>>>>>
>>>>>>> Hi Jesse -
>>>>>>
>>>>>>
>>>>>>
>>>>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>>>>> in that it only allows 64 charater representaion of this
>>>>>>> attribute. EbXML 3.0 (as used in XDS.b) ... extends this to 256,
>>>>>>> much more reasonable.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Is there any particular reason for such a long sourceId?
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Typically, what I recommend is to use a single OID that
>>>>>>> identifies the organization. Off of that oid, one can build an oid
>>>>>>> for each system in the organization (the sourceId) and uniqueIds
>>>>>>> for documents from that organization. Since the metadata tracks
>>>>>>> both the sourceId and the document uniqueId, there is no need to
>>>>>>> build the document uniqueId off of the sourceId oid ... but maybe
>>>>>>> this is something our code is doing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> This is a known bug that there is no particularly good workaround:
>>>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>>>>> I still am undecided as to what to do. Enforcement of the 64
>>>>>>> character limit at every level in OHF code seems redundant as when
>>>>>>> it is submitted to the repository, this error will surface anyway
>>>>>>> and the end result is some exception for our users in both cases.
>>>>>>> So ... I just need to figure out where to draw the line.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> As for the miss-reported status, this is a bug. Please submit
>>>>>>> another report regarding this issue. Submit a second if you are
>>>>>>> not building the uniqueId from the sourceId yourself (this is
>>>>>>> something I'll have to discuss with Matt) and could have API
>>>>>>> implications ... or we adjust our OID generation algorithm and
>>>>>>> start throwing exceptions in OHF with this issue.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> - Sarah
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Jesse Pangburn wrote:
>>>>>>>
>>>>>>>> Oh, one more thing about this. The really bad part is that the
>>>>>>>> bridge responds with:
>>>>>>>> <ns4:success
>>>>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>>>>
>>>>>>>> It also shows a documentEntryUUID so it appears as if the
>>>>>>>> document was successfully submitted. The only way I knew there
>>>>>>>> was an error was because I was watching the log.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Jesse
>>>>>>>>
>>>>>>>> Jesse Pangburn wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>>>>> generation. I passed
>>>>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
>>>>>>>>> submission set's sourceId, and the bridge generated
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>>>>> following snippet of the log entry's printout of the message
>>>>>>>>> transmitted:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> <rim:ExternalIdentifier
>>>>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Unfortunately, the repository complained about the length of
>>>>>>>>> uniqueId in this snippet from the response:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>>>>> '64' for type 'ShortName'
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I just did a quick count of my characters and there are about
>>>>>>>>> 45. The unique id generation brings the count to 80, so it adds
>>>>>>>>> about 35 characters. If this is typical, it means that the
>>>>>>>>> bridge user cannot pass an OID for the submission set sourceId
>>>>>>>>> that is longer than about 29 characters. In my case, the fixed
>>>>>>>>> part of the OID is "1.3.6.1.4.1.11854.1.100." (my IHE testing
>>>>>>>>> OID) which is 24 characters- leaving me only 5 characters to
>>>>>>>>> generate all my OIDs for IHE testing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I don't really know what the technical difference is between the
>>>>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>>>>> much longer? Can it just be the same?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> Jesse
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: submissionset.uniqueId generation problem [message #38073 is a reply to message #37869] Tue, 04 December 2007 21:56 Go to previous message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Matt,
I've tested this and it appears to be working fine. I verified the
XDSSubmissionSet.uniqueId and XDSSubmissionSet.sourceId values in the
SubmitObjectsRequest message.

thanks,
Jesse

Matthew Davis wrote:

> Hi Jesse,

> Nope... this was the one issue that fell through the cracks after I got
> back. It is now fixed and a new build is generating right now - it'll
> be available for download in a couple hours.

> API change is as follows:

> In the "DocumentSubmissionMetadata" type, there is a new value
> 'submissionSetUniqueId'. The Bridge will no longer auto-generate the
> SubmissionSet UniqueId based on the SourceId.
> DocumentSubmissionMetadata.submissionSetUniqueId is a required value and
> the Bridge will throw an exception if empty.

> Thanks,
> -Matt


> Jesse Pangburn wrote:
>> Hi Sarah,
>> I tried out the latest build of the bridge and it seems this bug is not
>> fixed. It occurred to me that maybe there was never a bug entered for
>> this in bugzilla. Do you know if this is the case? Or maybe it's on
>> your list still but you haven't gotten to it yet?
>>
>> Otherwise the new build is working great for me. Just wanted to check
>> if this one is on the list :-)
>>
>> thanks,
>> Jesse
>>
>> Sarah Knoop wrote:
>>
>>> Hi Jesse,
>>
>>> Not a "stupid question" :-). The new profile XCA utilizes the same
>>> transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
>>> profile a query can be farmed out to many servers and the responses
>>> aggregated. If one of the many servers is down, say, then this will
>>> return a 'failure' while the others return 'success'. Hence the
>>> 'partial success status' ... Now this is not going to be the case for
>>> submitting the document, but to keep things consistent across the
>>> bridge api, I'd like all three status to be available.
>>
>>> I think your suggestion below, to open the bridge api for both
>>> sourceId and uniqueID is the best suggestion. We'll role that in with
>>> the next set of API changes needed, resulting from your other bugs
>>> :-). We will also need to make updates per the specification section
>>> you have quoted below. We'll have to think how best to implement
>>> (document) this.
>>
>>> Thanks for being a great tester:-)
>>> - Sarah
>>
>>
>>
>>
>>> Jesse Pangburn wrote:
>>>> Hi Sarah,
>>>> This is probably a dumb question, but what would "partial success"
>>>> mean in this case? Why is it not just a failure?
>>>>
>>>> As for the sourceId, I just looked that up in the ITI TF Vol 2 on
>>>> page 122 in the Rev 3.0 final text version. Apparently it doesn't
>>>> need to be unique- it identifies the source system that produced a
>>>> document. From the PDF describing the submission set sourceId:
>>>> *********
>>>> OID identifying the instance of the Document Source that
>>>> contributed the Submission Set. When a "broker" is involved
>>>> in sending submission sets from a collection of client systems,
>>>> it should use a different source ID for submissions from each
>>>> separate system to allow for tracking.
>>>> *********
>>>>
>>>> So, in light of this, it seems that the bridge should have another
>>>> parameter for sourceId. Otherwise there's no way for a client of the
>>>> bridge to identify the source ID of the system that created it. In
>>>> fact, perhaps you could remove the uniqueId parameter and replace it
>>>> with the sourceId. Then we could pass a sourceId indicating the
>>>> system that generated a request, and you could use your current
>>>> algorithm to generate a uniqueId from that sourceId instead of the
>>>> other way around. In this way, the sourceId would truly indicate a
>>>> source system, and the uniqueId would be unique. It should leave
>>>> enough characters for us to uniquely identify systems but keep the
>>>> uniqueId under the 64 character limit.
>>>>
>>>> Or you could just take sourceId and uniqueId as parameters and let
>>>> the clients worry about passing what they want. Either way is good
>>>> for me. Anyone else want to chime in?
>>>>
>>>> thanks,
>>>> Jesse
>>>> Sarah Knoop wrote:
>>>>
>>>>> Hi Jesse,
>>>>> I have committed a fix for this bug (below) to OHF CVS. Please
>>>>> update from there. This issue highlighted the fact that the bridge
>>>>> allows only "success' and 'failure' status while IHE allows an
>>>>> additional 'partial success' status. This will necessitate an API
>>>>> change in the bridge. Another bug will be opened to address this:
>>>>> 209441.
>>>>
>>>>
>>>>> For the OID problem, our difficulty lies in ensuring that the
>>>>> generated uniqueID is truly unique (at least with high probability).
>>>>> For those 35 characters we use a combination of machine MAC address
>>>>> and the system time in milli's, and then a sequence number, in case
>>>>> these are generated in rapid succession. We use the sourceId as a
>>>>> base to further ensure uniqueness. Currently the bridge is installed
>>>>> locally by each vendor, but it's intent is to be a centralized
>>>>> service for many users. Hence the precautions and why using the
>>>>> source ID will help with uniqueness.
>>>>
>>>>
>>>>> In my note below, I was a bit confused (I read your email too fast).
>>>>> So for this reason above, I'd like to keep the correlation between
>>>>> sourceId and submission set unique id. But, IHE does not require it
>>>>> and (now arguing with my self :-)) as I stated below the metadata
>>>>> preserves this correlation in the XML, so there is no need to do so
>>>>> for that reason.
>>>>
>>>>
>>>>> I'll discuss with Matt. It's a simple change, as we have a method to
>>>>> generate OIDs using a default prefix (something like 1.2.8). But,
>>>>> I'm still torn as to what is the best approach and what we will end
>>>>> up with in terms of uniqueness, in the end.
>>>>
>>>>
>>>>> Does the community have an opinion on this one??
>>>>> - Sarah
>>>>
>>>>
>>>>
>>>>> Jesse Pangburn wrote:
>>>>>
>>>>>> Hi Sarah,
>>>>>> For the sourceId, I don't have too much choice.
>>>>>> "1.3.6.1.4.1.11854" is my company OID, ".1" adds R&D department,
>>>>>> and I added ".100" for IHE testing. So I'm stuck with the
>>>>>> "1.3.6.1.4.1.11854.1.100." 24 characters already. The bridge
>>>>>> generates the uniqueId from the sourceId by appending 35 characters
>>>>>> (seems consistent). So 24+35=59, that leaves me with 5 characters
>>>>>> for all my IHE testing OIDs. I would think most people are in a
>>>>>> similar situation?
>>>>>>
>>>>>> I don't think you (the bridge) need to be throwing errors for the
>>>>>> 64 character limit, though it wouldn't hurt, because as you said
>>>>>> the repository will complain. That's fine, I agree you don't need
>>>>>> to bother.
>>>>>>
>>>>>> The thing I would suggest (er... request) is that you either don't
>>>>>> build the uniqueId from the sourceId, or if you do- use less
>>>>>> characters? If the sourceId is already unique, then just adding
>>>>>> one more character would make it unique and different from the
>>>>>> sourceId.
>>>>>>
>>>>>> As for the success response when there's an error, I submitted Bug
>>>>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>>>>
>>>>>> thanks
>>>>>> Jesse
>>>>>>
>>>>>> Sarah Knoop wrote:
>>>>>>
>>>>>>> Hi Jesse -
>>>>>>
>>>>>>
>>>>>>
>>>>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>>>>> in that it only allows 64 charater representaion of this
>>>>>>> attribute. EbXML 3.0 (as used in XDS.b) ... extends this to 256,
>>>>>>> much more reasonable.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Is there any particular reason for such a long sourceId?
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Typically, what I recommend is to use a single OID that
>>>>>>> identifies the organization. Off of that oid, one can build an oid
>>>>>>> for each system in the organization (the sourceId) and uniqueIds
>>>>>>> for documents from that organization. Since the metadata tracks
>>>>>>> both the sourceId and the document uniqueId, there is no need to
>>>>>>> build the document uniqueId off of the sourceId oid ... but maybe
>>>>>>> this is something our code is doing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> This is a known bug that there is no particularly good workaround:
>>>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>>>>> I still am undecided as to what to do. Enforcement of the 64
>>>>>>> character limit at every level in OHF code seems redundant as when
>>>>>>> it is submitted to the repository, this error will surface anyway
>>>>>>> and the end result is some exception for our users in both cases.
>>>>>>> So ... I just need to figure out where to draw the line.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> As for the miss-reported status, this is a bug. Please submit
>>>>>>> another report regarding this issue. Submit a second if you are
>>>>>>> not building the uniqueId from the sourceId yourself (this is
>>>>>>> something I'll have to discuss with Matt) and could have API
>>>>>>> implications ... or we adjust our OID generation algorithm and
>>>>>>> start throwing exceptions in OHF with this issue.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> - Sarah
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Jesse Pangburn wrote:
>>>>>>>
>>>>>>>> Oh, one more thing about this. The really bad part is that the
>>>>>>>> bridge responds with:
>>>>>>>> <ns4:success
>>>>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>>>>
>>>>>>>> It also shows a documentEntryUUID so it appears as if the
>>>>>>>> document was successfully submitted. The only way I knew there
>>>>>>>> was an error was because I was watching the log.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Jesse
>>>>>>>>
>>>>>>>> Jesse Pangburn wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>>>>> generation. I passed
>>>>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
>>>>>>>>> submission set's sourceId, and the bridge generated
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>>>>> following snippet of the log entry's printout of the message
>>>>>>>>> transmitted:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> <rim:ExternalIdentifier
>>>>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Unfortunately, the repository complained about the length of
>>>>>>>>> uniqueId in this snippet from the response:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>>>>> '64' for type 'ShortName'
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I just did a quick count of my characters and there are about
>>>>>>>>> 45. The unique id generation brings the count to 80, so it adds
>>>>>>>>> about 35 characters. If this is typical, it means that the
>>>>>>>>> bridge user cannot pass an OID for the submission set sourceId
>>>>>>>>> that is longer than about 29 characters. In my case, the fixed
>>>>>>>>> part of the OID is "1.3.6.1.4.1.11854.1.100." (my IHE testing
>>>>>>>>> OID) which is 24 characters- leaving me only 5 characters to
>>>>>>>>> generate all my OIDs for IHE testing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I don't really know what the technical difference is between the
>>>>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>>>>> much longer? Can it just be the same?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> Jesse
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: submissionset.uniqueId generation problem [message #582934 is a reply to message #36590] Thu, 08 November 2007 22:35 Go to previous message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Oh, one more thing about this. The really bad part is that the bridge
responds with:
<ns4:success
xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>

It also shows a documentEntryUUID so it appears as if the document was
successfully submitted. The only way I knew there was an error was
because I was watching the log.

thanks,
Jesse

Jesse Pangburn wrote:

> Hi,
> I ran into a funny problem with the submission set's unique id generation.
> I passed "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
> submission set's sourceId, and the bridge generated
>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
> as the XDSSubmissionSet.uniqueId. This is evidenced by the following
> snippet of the log entry's printout of the message transmitted:

> <rim:ExternalIdentifier
> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
> value="XDSSubmissionSet.sourceId"
> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
> value="XDSSubmissionSet.uniqueId" /></rim:Name></rim:ExternalIdentifier>

> Unfortunately, the repository complained about the length of uniqueId in
> this snippet from the response:

> Schema Validation Errors:&#xa;Input did not validate against
> schema:&#xa;Error: cvc-maxLength-valid: Value
>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
> with length = '80' is not facet-valid with respect to maxLength '64' for
> type 'ShortName'

> I just did a quick count of my characters and there are about 45. The
> unique id generation brings the count to 80, so it adds about 35
> characters. If this is typical, it means that the bridge user cannot pass
> an OID for the submission set sourceId that is longer than about 29
> characters. In my case, the fixed part of the OID is
> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24 characters-
> leaving me only 5 characters to generate all my OIDs for IHE testing.

> I don't really know what the technical difference is between the sourceId
> and the uniqueId, but does the uniqueId need to be that much longer? Can
> it just be the same?

> thanks,
> Jesse
Re: submissionset.uniqueId generation problem [message #582948 is a reply to message #36624] Thu, 08 November 2007 23:22 Go to previous message
No real name is currently offline No real nameFriend
Messages: 292
Registered: July 2009
Senior Member
Hi Jesse -

For the repository error, as you noted, this is an ebXML 2.1 issue in
that it only allows 64 charater representaion of this attribute. EbXML
3.0 (as used in XDS.b) ... extends this to 256, much more reasonable.

Is there any particular reason for such a long sourceId?

Typically, what I recommend is to use a single OID that identifies the
organization. Off of that oid, one can build an oid for each system in
the organization (the sourceId) and uniqueIds for documents from that
organization. Since the metadata tracks both the sourceId and the
document uniqueId, there is no need to build the document uniqueId off
of the sourceId oid ... but maybe this is something our code is doing.

This is a known bug that there is no particularly good workaround:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
I still am undecided as to what to do. Enforcement of the 64 character
limit at every level in OHF code seems redundant as when it is submitted
to the repository, this error will surface anyway and the end result is
some exception for our users in both cases. So ... I just need to figure
out where to draw the line.

As for the miss-reported status, this is a bug. Please submit another
report regarding this issue. Submit a second if you are not building the
uniqueId from the sourceId yourself (this is something I'll have to
discuss with Matt) and could have API implications ... or we adjust our
OID generation algorithm and start throwing exceptions in OHF with this
issue.

- Sarah



Jesse Pangburn wrote:
> Oh, one more thing about this. The really bad part is that the bridge
> responds with:
> <ns4:success
> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>
> It also shows a documentEntryUUID so it appears as if the document was
> successfully submitted. The only way I knew there was an error was
> because I was watching the log.
>
> thanks,
> Jesse
>
> Jesse Pangburn wrote:
>
>> Hi,
>> I ran into a funny problem with the submission set's unique id
>> generation. I passed "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2"
>> as the submission set's sourceId, and the bridge generated
>
> " 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>
>
>> as the XDSSubmissionSet.uniqueId. This is evidenced by the following
>> snippet of the log entry's printout of the message transmitted:
>
>
>> <rim:ExternalIdentifier
>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>
> value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>
>
>> value="XDSSubmissionSet.sourceId"
>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>
> value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>
>
>> value="XDSSubmissionSet.uniqueId" /></rim:Name></rim:ExternalIdentifier>
>
>
>> Unfortunately, the repository complained about the length of uniqueId
>> in this snippet from the response:
>
>
>> Schema Validation Errors:&#xa;Input did not validate against
>> schema:&#xa;Error: cvc-maxLength-valid: Value
>
> '1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>
>
>> with length = '80' is not facet-valid with respect to maxLength '64'
>> for type 'ShortName'
>
>
>> I just did a quick count of my characters and there are about 45. The
>> unique id generation brings the count to 80, so it adds about 35
>> characters. If this is typical, it means that the bridge user cannot
>> pass an OID for the submission set sourceId that is longer than about
>> 29 characters. In my case, the fixed part of the OID is
>> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>> characters- leaving me only 5 characters to generate all my OIDs for
>> IHE testing.
>
>
>> I don't really know what the technical difference is between the
>> sourceId and the uniqueId, but does the uniqueId need to be that much
>> longer? Can it just be the same?
>
>
>> thanks,
>> Jesse
>
>
>
Re: submissionset.uniqueId generation problem [message #582979 is a reply to message #36692] Fri, 09 November 2007 20:13 Go to previous message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Sarah,
For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854" is my
company OID, ".1" adds R&D department, and I added ".100" for IHE testing.
So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24 characters already.
The bridge generates the uniqueId from the sourceId by appending 35
characters (seems consistent). So 24+35=59, that leaves me with 5
characters for all my IHE testing OIDs. I would think most people are in
a similar situation?

I don't think you (the bridge) need to be throwing errors for the 64
character limit, though it wouldn't hurt, because as you said the
repository will complain. That's fine, I agree you don't need to bother.

The thing I would suggest (er... request) is that you either don't build
the uniqueId from the sourceId, or if you do- use less characters? If the
sourceId is already unique, then just adding one more character would make
it unique and different from the sourceId.

As for the success response when there's an error, I submitted Bug 209382
https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382

thanks
Jesse

Sarah Knoop wrote:

> Hi Jesse -

> For the repository error, as you noted, this is an ebXML 2.1 issue in
> that it only allows 64 charater representaion of this attribute. EbXML
> 3.0 (as used in XDS.b) ... extends this to 256, much more reasonable.

> Is there any particular reason for such a long sourceId?

> Typically, what I recommend is to use a single OID that identifies the
> organization. Off of that oid, one can build an oid for each system in
> the organization (the sourceId) and uniqueIds for documents from that
> organization. Since the metadata tracks both the sourceId and the
> document uniqueId, there is no need to build the document uniqueId off
> of the sourceId oid ... but maybe this is something our code is doing.

> This is a known bug that there is no particularly good workaround:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
> I still am undecided as to what to do. Enforcement of the 64 character
> limit at every level in OHF code seems redundant as when it is submitted
> to the repository, this error will surface anyway and the end result is
> some exception for our users in both cases. So ... I just need to figure
> out where to draw the line.

> As for the miss-reported status, this is a bug. Please submit another
> report regarding this issue. Submit a second if you are not building the
> uniqueId from the sourceId yourself (this is something I'll have to
> discuss with Matt) and could have API implications ... or we adjust our
> OID generation algorithm and start throwing exceptions in OHF with this
> issue.

> - Sarah



> Jesse Pangburn wrote:
>> Oh, one more thing about this. The really bad part is that the bridge
>> responds with:
>> <ns4:success
>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>
>> It also shows a documentEntryUUID so it appears as if the document was
>> successfully submitted. The only way I knew there was an error was
>> because I was watching the log.
>>
>> thanks,
>> Jesse
>>
>> Jesse Pangburn wrote:
>>
>>> Hi,
>>> I ran into a funny problem with the submission set's unique id
>>> generation. I passed "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2"
>>> as the submission set's sourceId, and the bridge generated
>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>
>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the following
>>> snippet of the log entry's printout of the message transmitted:
>>
>>
>>> <rim:ExternalIdentifier
>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>
>>> value="XDSSubmissionSet.sourceId"
>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>
>>> value="XDSSubmissionSet.uniqueId" /></rim:Name></rim:ExternalIdentifier>
>>
>>
>>> Unfortunately, the repository complained about the length of uniqueId
>>> in this snippet from the response:
>>
>>
>>> Schema Validation Errors:&#xa;Input did not validate against
>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>
>>> with length = '80' is not facet-valid with respect to maxLength '64'
>>> for type 'ShortName'
>>
>>
>>> I just did a quick count of my characters and there are about 45. The
>>> unique id generation brings the count to 80, so it adds about 35
>>> characters. If this is typical, it means that the bridge user cannot
>>> pass an OID for the submission set sourceId that is longer than about
>>> 29 characters. In my case, the fixed part of the OID is
>>> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>> characters- leaving me only 5 characters to generate all my OIDs for
>>> IHE testing.
>>
>>
>>> I don't really know what the technical difference is between the
>>> sourceId and the uniqueId, but does the uniqueId need to be that much
>>> longer? Can it just be the same?
>>
>>
>>> thanks,
>>> Jesse
>>
>>
>>
Re: submissionset.uniqueId generation problem [message #583033 is a reply to message #36796] Mon, 12 November 2007 01:32 Go to previous message
No real name is currently offline No real nameFriend
Messages: 292
Registered: July 2009
Senior Member
Hi Jesse,
I have committed a fix for this bug (below) to OHF CVS. Please update
from there. This issue highlighted the fact that the bridge allows only
"success' and 'failure' status while IHE allows an additional 'partial
success' status. This will necessitate an API change in the bridge.
Another bug will be opened to address this: 209441.

For the OID problem, our difficulty lies in ensuring that the generated
uniqueID is truly unique (at least with high probability). For those 35
characters we use a combination of machine MAC address and the system
time in milli's, and then a sequence number, in case these are generated
in rapid succession. We use the sourceId as a base to further ensure
uniqueness. Currently the bridge is installed locally by each vendor,
but it's intent is to be a centralized service for many users. Hence the
precautions and why using the source ID will help with uniqueness.

In my note below, I was a bit confused (I read your email too fast). So
for this reason above, I'd like to keep the correlation between sourceId
and submission set unique id. But, IHE does not require it and (now
arguing with my self :-)) as I stated below the metadata preserves this
correlation in the XML, so there is no need to do so for that reason.

I'll discuss with Matt. It's a simple change, as we have a method to
generate OIDs using a default prefix (something like 1.2.8). But, I'm
still torn as to what is the best approach and what we will end up with
in terms of uniqueness, in the end.

Does the community have an opinion on this one??
- Sarah


Jesse Pangburn wrote:
> Hi Sarah,
> For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854" is
> my company OID, ".1" adds R&D department, and I added ".100" for IHE
> testing. So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24 characters
> already. The bridge generates the uniqueId from the sourceId by
> appending 35 characters (seems consistent). So 24+35=59, that leaves me
> with 5 characters for all my IHE testing OIDs. I would think most
> people are in a similar situation?
>
> I don't think you (the bridge) need to be throwing errors for the 64
> character limit, though it wouldn't hurt, because as you said the
> repository will complain. That's fine, I agree you don't need to bother.
>
> The thing I would suggest (er... request) is that you either don't build
> the uniqueId from the sourceId, or if you do- use less characters? If
> the sourceId is already unique, then just adding one more character
> would make it unique and different from the sourceId.
>
> As for the success response when there's an error, I submitted Bug
> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>
> thanks
> Jesse
>
> Sarah Knoop wrote:
>
>> Hi Jesse -
>
>
>> For the repository error, as you noted, this is an ebXML 2.1 issue in
>> that it only allows 64 charater representaion of this attribute. EbXML
>> 3.0 (as used in XDS.b) ... extends this to 256, much more reasonable.
>
>
>> Is there any particular reason for such a long sourceId?
>
>
>> Typically, what I recommend is to use a single OID that identifies
>> the organization. Off of that oid, one can build an oid for each
>> system in the organization (the sourceId) and uniqueIds for documents
>> from that organization. Since the metadata tracks both the sourceId
>> and the document uniqueId, there is no need to build the document
>> uniqueId off of the sourceId oid ... but maybe this is something our
>> code is doing.
>
>
>> This is a known bug that there is no particularly good workaround:
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>> I still am undecided as to what to do. Enforcement of the 64 character
>> limit at every level in OHF code seems redundant as when it is
>> submitted to the repository, this error will surface anyway and the
>> end result is some exception for our users in both cases. So ... I
>> just need to figure out where to draw the line.
>
>
>> As for the miss-reported status, this is a bug. Please submit another
>> report regarding this issue. Submit a second if you are not building
>> the uniqueId from the sourceId yourself (this is something I'll have
>> to discuss with Matt) and could have API implications ... or we adjust
>> our OID generation algorithm and start throwing exceptions in OHF with
>> this issue.
>
>
>> - Sarah
>
>
>
>
>> Jesse Pangburn wrote:
>>
>>> Oh, one more thing about this. The really bad part is that the
>>> bridge responds with:
>>> <ns4:success
>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>
>>> It also shows a documentEntryUUID so it appears as if the document
>>> was successfully submitted. The only way I knew there was an error
>>> was because I was watching the log.
>>>
>>> thanks,
>>> Jesse
>>>
>>> Jesse Pangburn wrote:
>>>
>>>> Hi,
>>>> I ran into a funny problem with the submission set's unique id
>>>> generation. I passed
>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the submission
>>>> set's sourceId, and the bridge generated
>>>
>>>
>>>
> " 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>
>
>>>
>>>
>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>> following snippet of the log entry's printout of the message
>>>> transmitted:
>>>
>>>
>>>
>>>> <rim:ExternalIdentifier
>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>
>>>
>>>
> value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>
>
>>>
>>>
>>>> value="XDSSubmissionSet.sourceId"
>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>
>>>
>>>
> value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>
>
>>>
>>>
>>>> value="XDSSubmissionSet.uniqueId"
>>>> /></rim:Name></rim:ExternalIdentifier>
>>>
>>>
>>>
>>>> Unfortunately, the repository complained about the length of
>>>> uniqueId in this snippet from the response:
>>>
>>>
>>>
>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>
>>>
>>>
> '1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>
>
>>>
>>>
>>>> with length = '80' is not facet-valid with respect to maxLength '64'
>>>> for type 'ShortName'
>>>
>>>
>>>
>>>> I just did a quick count of my characters and there are about 45.
>>>> The unique id generation brings the count to 80, so it adds about 35
>>>> characters. If this is typical, it means that the bridge user
>>>> cannot pass an OID for the submission set sourceId that is longer
>>>> than about 29 characters. In my case, the fixed part of the OID is
>>>> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>>> characters- leaving me only 5 characters to generate all my OIDs for
>>>> IHE testing.
>>>
>>>
>>>
>>>> I don't really know what the technical difference is between the
>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>> much longer? Can it just be the same?
>>>
>>>
>>>
>>>> thanks,
>>>> Jesse
>>>
>>>
>>>
>>>
>
>
Re: submissionset.uniqueId generation problem [message #583061 is a reply to message #36932] Mon, 12 November 2007 17:57 Go to previous message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Sarah,
This is probably a dumb question, but what would "partial success" mean in
this case? Why is it not just a failure?

As for the sourceId, I just looked that up in the ITI TF Vol 2 on page 122
in the Rev 3.0 final text version. Apparently it doesn't need to be
unique- it identifies the source system that produced a document. From
the PDF describing the submission set sourceId:
*********
OID identifying the instance of the Document Source that
contributed the Submission Set. When a "broker" is involved
in sending submission sets from a collection of client systems,
it should use a different source ID for submissions from each
separate system to allow for tracking.
*********

So, in light of this, it seems that the bridge should have another
parameter for sourceId. Otherwise there's no way for a client of the
bridge to identify the source ID of the system that created it. In fact,
perhaps you could remove the uniqueId parameter and replace it with the
sourceId. Then we could pass a sourceId indicating the system that
generated a request, and you could use your current algorithm to generate
a uniqueId from that sourceId instead of the other way around. In this
way, the sourceId would truly indicate a source system, and the uniqueId
would be unique. It should leave enough characters for us to uniquely
identify systems but keep the uniqueId under the 64 character limit.

Or you could just take sourceId and uniqueId as parameters and let the
clients worry about passing what they want. Either way is good for me.
Anyone else want to chime in?

thanks,
Jesse
Sarah Knoop wrote:

> Hi Jesse,
> I have committed a fix for this bug (below) to OHF CVS. Please update
> from there. This issue highlighted the fact that the bridge allows only
> "success' and 'failure' status while IHE allows an additional 'partial
> success' status. This will necessitate an API change in the bridge.
> Another bug will be opened to address this: 209441.

> For the OID problem, our difficulty lies in ensuring that the generated
> uniqueID is truly unique (at least with high probability). For those 35
> characters we use a combination of machine MAC address and the system
> time in milli's, and then a sequence number, in case these are generated
> in rapid succession. We use the sourceId as a base to further ensure
> uniqueness. Currently the bridge is installed locally by each vendor,
> but it's intent is to be a centralized service for many users. Hence the
> precautions and why using the source ID will help with uniqueness.

> In my note below, I was a bit confused (I read your email too fast). So
> for this reason above, I'd like to keep the correlation between sourceId
> and submission set unique id. But, IHE does not require it and (now
> arguing with my self :-)) as I stated below the metadata preserves this
> correlation in the XML, so there is no need to do so for that reason.

> I'll discuss with Matt. It's a simple change, as we have a method to
> generate OIDs using a default prefix (something like 1.2.8). But, I'm
> still torn as to what is the best approach and what we will end up with
> in terms of uniqueness, in the end.

> Does the community have an opinion on this one??
> - Sarah


> Jesse Pangburn wrote:
>> Hi Sarah,
>> For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854" is
>> my company OID, ".1" adds R&D department, and I added ".100" for IHE
>> testing. So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24 characters
>> already. The bridge generates the uniqueId from the sourceId by
>> appending 35 characters (seems consistent). So 24+35=59, that leaves me
>> with 5 characters for all my IHE testing OIDs. I would think most
>> people are in a similar situation?
>>
>> I don't think you (the bridge) need to be throwing errors for the 64
>> character limit, though it wouldn't hurt, because as you said the
>> repository will complain. That's fine, I agree you don't need to bother.
>>
>> The thing I would suggest (er... request) is that you either don't build
>> the uniqueId from the sourceId, or if you do- use less characters? If
>> the sourceId is already unique, then just adding one more character
>> would make it unique and different from the sourceId.
>>
>> As for the success response when there's an error, I submitted Bug
>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>
>> thanks
>> Jesse
>>
>> Sarah Knoop wrote:
>>
>>> Hi Jesse -
>>
>>
>>> For the repository error, as you noted, this is an ebXML 2.1 issue in
>>> that it only allows 64 charater representaion of this attribute. EbXML
>>> 3.0 (as used in XDS.b) ... extends this to 256, much more reasonable.
>>
>>
>>> Is there any particular reason for such a long sourceId?
>>
>>
>>> Typically, what I recommend is to use a single OID that identifies
>>> the organization. Off of that oid, one can build an oid for each
>>> system in the organization (the sourceId) and uniqueIds for documents
>>> from that organization. Since the metadata tracks both the sourceId
>>> and the document uniqueId, there is no need to build the document
>>> uniqueId off of the sourceId oid ... but maybe this is something our
>>> code is doing.
>>
>>
>>> This is a known bug that there is no particularly good workaround:
>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>> I still am undecided as to what to do. Enforcement of the 64 character
>>> limit at every level in OHF code seems redundant as when it is
>>> submitted to the repository, this error will surface anyway and the
>>> end result is some exception for our users in both cases. So ... I
>>> just need to figure out where to draw the line.
>>
>>
>>> As for the miss-reported status, this is a bug. Please submit another
>>> report regarding this issue. Submit a second if you are not building
>>> the uniqueId from the sourceId yourself (this is something I'll have
>>> to discuss with Matt) and could have API implications ... or we adjust
>>> our OID generation algorithm and start throwing exceptions in OHF with
>>> this issue.
>>
>>
>>> - Sarah
>>
>>
>>
>>
>>> Jesse Pangburn wrote:
>>>
>>>> Oh, one more thing about this. The really bad part is that the
>>>> bridge responds with:
>>>> <ns4:success
>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>
>>>> It also shows a documentEntryUUID so it appears as if the document
>>>> was successfully submitted. The only way I knew there was an error
>>>> was because I was watching the log.
>>>>
>>>> thanks,
>>>> Jesse
>>>>
>>>> Jesse Pangburn wrote:
>>>>
>>>>> Hi,
>>>>> I ran into a funny problem with the submission set's unique id
>>>>> generation. I passed
>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the submission
>>>>> set's sourceId, and the bridge generated
>>>>
>>>>
>>>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>
>>>>
>>>>
>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>> following snippet of the log entry's printout of the message
>>>>> transmitted:
>>>>
>>>>
>>>>
>>>>> <rim:ExternalIdentifier
>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>
>>>>
>>>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>
>>>>
>>>>
>>>>> value="XDSSubmissionSet.sourceId"
>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>
>>>>
>>>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>
>>>>
>>>>
>>>>> value="XDSSubmissionSet.uniqueId"
>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>
>>>>
>>>>
>>>>> Unfortunately, the repository complained about the length of
>>>>> uniqueId in this snippet from the response:
>>>>
>>>>
>>>>
>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>
>>>>
>>>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>
>>>>
>>>>
>>>>> with length = '80' is not facet-valid with respect to maxLength '64'
>>>>> for type 'ShortName'
>>>>
>>>>
>>>>
>>>>> I just did a quick count of my characters and there are about 45.
>>>>> The unique id generation brings the count to 80, so it adds about 35
>>>>> characters. If this is typical, it means that the bridge user
>>>>> cannot pass an OID for the submission set sourceId that is longer
>>>>> than about 29 characters. In my case, the fixed part of the OID is
>>>>> "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>>>> characters- leaving me only 5 characters to generate all my OIDs for
>>>>> IHE testing.
>>>>
>>>>
>>>>
>>>>> I don't really know what the technical difference is between the
>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>> much longer? Can it just be the same?
>>>>
>>>>
>>>>
>>>>> thanks,
>>>>> Jesse
>>>>
>>>>
>>>>
>>>>
>>
>>
Re: submissionset.uniqueId generation problem [message #583093 is a reply to message #37000] Tue, 13 November 2007 19:55 Go to previous message
No real name is currently offline No real nameFriend
Messages: 292
Registered: July 2009
Senior Member
Hi Jesse,

Not a "stupid question" :-). The new profile XCA utilizes the same
transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
profile a query can be farmed out to many servers and the responses
aggregated. If one of the many servers is down, say, then this will
return a 'failure' while the others return 'success'. Hence the 'partial
success status' ... Now this is not going to be the case for submitting
the document, but to keep things consistent across the bridge api, I'd
like all three status to be available.

I think your suggestion below, to open the bridge api for both sourceId
and uniqueID is the best suggestion. We'll role that in with the next
set of API changes needed, resulting from your other bugs :-). We will
also need to make updates per the specification section you have quoted
below. We'll have to think how best to implement (document) this.

Thanks for being a great tester:-)
- Sarah




Jesse Pangburn wrote:
> Hi Sarah,
> This is probably a dumb question, but what would "partial success" mean
> in this case? Why is it not just a failure?
>
> As for the sourceId, I just looked that up in the ITI TF Vol 2 on page
> 122 in the Rev 3.0 final text version. Apparently it doesn't need to be
> unique- it identifies the source system that produced a document. From
> the PDF describing the submission set sourceId:
> *********
> OID identifying the instance of the Document Source that
> contributed the Submission Set. When a "broker" is involved
> in sending submission sets from a collection of client systems,
> it should use a different source ID for submissions from each
> separate system to allow for tracking.
> *********
>
> So, in light of this, it seems that the bridge should have another
> parameter for sourceId. Otherwise there's no way for a client of the
> bridge to identify the source ID of the system that created it. In
> fact, perhaps you could remove the uniqueId parameter and replace it
> with the sourceId. Then we could pass a sourceId indicating the system
> that generated a request, and you could use your current algorithm to
> generate a uniqueId from that sourceId instead of the other way around.
> In this way, the sourceId would truly indicate a source system, and the
> uniqueId would be unique. It should leave enough characters for us to
> uniquely identify systems but keep the uniqueId under the 64 character
> limit.
>
> Or you could just take sourceId and uniqueId as parameters and let the
> clients worry about passing what they want. Either way is good for me.
> Anyone else want to chime in?
>
> thanks,
> Jesse
> Sarah Knoop wrote:
>
>> Hi Jesse,
>> I have committed a fix for this bug (below) to OHF CVS. Please update
>> from there. This issue highlighted the fact that the bridge allows
>> only "success' and 'failure' status while IHE allows an additional
>> 'partial success' status. This will necessitate an API change in the
>> bridge. Another bug will be opened to address this: 209441.
>
>
>> For the OID problem, our difficulty lies in ensuring that the
>> generated uniqueID is truly unique (at least with high probability).
>> For those 35 characters we use a combination of machine MAC address
>> and the system time in milli's, and then a sequence number, in case
>> these are generated in rapid succession. We use the sourceId as a base
>> to further ensure uniqueness. Currently the bridge is installed
>> locally by each vendor, but it's intent is to be a centralized service
>> for many users. Hence the precautions and why using the source ID will
>> help with uniqueness.
>
>
>> In my note below, I was a bit confused (I read your email too fast).
>> So for this reason above, I'd like to keep the correlation between
>> sourceId and submission set unique id. But, IHE does not require it
>> and (now arguing with my self :-)) as I stated below the metadata
>> preserves this correlation in the XML, so there is no need to do so
>> for that reason.
>
>
>> I'll discuss with Matt. It's a simple change, as we have a method to
>> generate OIDs using a default prefix (something like 1.2.8). But, I'm
>> still torn as to what is the best approach and what we will end up
>> with in terms of uniqueness, in the end.
>
>
>> Does the community have an opinion on this one??
>> - Sarah
>
>
>
>> Jesse Pangburn wrote:
>>
>>> Hi Sarah,
>>> For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854"
>>> is my company OID, ".1" adds R&D department, and I added ".100" for
>>> IHE testing. So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24
>>> characters already. The bridge generates the uniqueId from the
>>> sourceId by appending 35 characters (seems consistent). So 24+35=59,
>>> that leaves me with 5 characters for all my IHE testing OIDs. I
>>> would think most people are in a similar situation?
>>>
>>> I don't think you (the bridge) need to be throwing errors for the 64
>>> character limit, though it wouldn't hurt, because as you said the
>>> repository will complain. That's fine, I agree you don't need to
>>> bother.
>>>
>>> The thing I would suggest (er... request) is that you either don't
>>> build the uniqueId from the sourceId, or if you do- use less
>>> characters? If the sourceId is already unique, then just adding one
>>> more character would make it unique and different from the sourceId.
>>>
>>> As for the success response when there's an error, I submitted Bug
>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>
>>> thanks
>>> Jesse
>>>
>>> Sarah Knoop wrote:
>>>
>>>> Hi Jesse -
>>>
>>>
>>>
>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>> in that it only allows 64 charater representaion of this attribute.
>>>> EbXML 3.0 (as used in XDS.b) ... extends this to 256, much more
>>>> reasonable.
>>>
>>>
>>>
>>>> Is there any particular reason for such a long sourceId?
>>>
>>>
>>>
>>>> Typically, what I recommend is to use a single OID that identifies
>>>> the organization. Off of that oid, one can build an oid for each
>>>> system in the organization (the sourceId) and uniqueIds for
>>>> documents from that organization. Since the metadata tracks both the
>>>> sourceId and the document uniqueId, there is no need to build the
>>>> document uniqueId off of the sourceId oid ... but maybe this is
>>>> something our code is doing.
>>>
>>>
>>>
>>>> This is a known bug that there is no particularly good workaround:
>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>> I still am undecided as to what to do. Enforcement of the 64
>>>> character limit at every level in OHF code seems redundant as when
>>>> it is submitted to the repository, this error will surface anyway
>>>> and the end result is some exception for our users in both cases. So
>>>> ... I just need to figure out where to draw the line.
>>>
>>>
>>>
>>>> As for the miss-reported status, this is a bug. Please submit
>>>> another report regarding this issue. Submit a second if you are not
>>>> building the uniqueId from the sourceId yourself (this is
>>>> something I'll have to discuss with Matt) and could have API
>>>> implications ... or we adjust our OID generation algorithm and start
>>>> throwing exceptions in OHF with this issue.
>>>
>>>
>>>
>>>> - Sarah
>>>
>>>
>>>
>>>
>>>
>>>> Jesse Pangburn wrote:
>>>>
>>>>> Oh, one more thing about this. The really bad part is that the
>>>>> bridge responds with:
>>>>> <ns4:success
>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>
>>>>> It also shows a documentEntryUUID so it appears as if the document
>>>>> was successfully submitted. The only way I knew there was an error
>>>>> was because I was watching the log.
>>>>>
>>>>> thanks,
>>>>> Jesse
>>>>>
>>>>> Jesse Pangburn wrote:
>>>>>
>>>>>> Hi,
>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>> generation. I passed
>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the submission
>>>>>> set's sourceId, and the bridge generated
>>>>>
>>>>>
>>>>>
>>>>>
>>>
> " 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>
>
>>>
>>>
>>>>>
>>>>>
>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>> following snippet of the log entry's printout of the message
>>>>>> transmitted:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> <rim:ExternalIdentifier
>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>
>>>>>
>>>>>
>>>>>
>>>
> value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>
>
>>>
>>>
>>>>>
>>>>>
>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>
>>>>>
>>>>>
>>>>>
>>>
> value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>
>
>>>
>>>
>>>>>
>>>>>
>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Unfortunately, the repository complained about the length of
>>>>>> uniqueId in this snippet from the response:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>
>>>>>
>>>>>
>>>>>
>>>
> '1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>
>
>>>
>>>
>>>>>
>>>>>
>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>> '64' for type 'ShortName'
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I just did a quick count of my characters and there are about 45.
>>>>>> The unique id generation brings the count to 80, so it adds about
>>>>>> 35 characters. If this is typical, it means that the bridge user
>>>>>> cannot pass an OID for the submission set sourceId that is longer
>>>>>> than about 29 characters. In my case, the fixed part of the OID
>>>>>> is "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>>>>> characters- leaving me only 5 characters to generate all my OIDs
>>>>>> for IHE testing.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I don't really know what the technical difference is between the
>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>> much longer? Can it just be the same?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> thanks,
>>>>>> Jesse
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>
>
Re: submissionset.uniqueId generation problem [message #583408 is a reply to message #37069] Fri, 30 November 2007 19:38 Go to previous message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Sarah,
I tried out the latest build of the bridge and it seems this bug is not
fixed. It occurred to me that maybe there was never a bug entered for
this in bugzilla. Do you know if this is the case? Or maybe it's on your
list still but you haven't gotten to it yet?

Otherwise the new build is working great for me. Just wanted to check if
this one is on the list :-)

thanks,
Jesse

Sarah Knoop wrote:

> Hi Jesse,

> Not a "stupid question" :-). The new profile XCA utilizes the same
> transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
> profile a query can be farmed out to many servers and the responses
> aggregated. If one of the many servers is down, say, then this will
> return a 'failure' while the others return 'success'. Hence the 'partial
> success status' ... Now this is not going to be the case for submitting
> the document, but to keep things consistent across the bridge api, I'd
> like all three status to be available.

> I think your suggestion below, to open the bridge api for both sourceId
> and uniqueID is the best suggestion. We'll role that in with the next
> set of API changes needed, resulting from your other bugs :-). We will
> also need to make updates per the specification section you have quoted
> below. We'll have to think how best to implement (document) this.

> Thanks for being a great tester:-)
> - Sarah




> Jesse Pangburn wrote:
>> Hi Sarah,
>> This is probably a dumb question, but what would "partial success" mean
>> in this case? Why is it not just a failure?
>>
>> As for the sourceId, I just looked that up in the ITI TF Vol 2 on page
>> 122 in the Rev 3.0 final text version. Apparently it doesn't need to be
>> unique- it identifies the source system that produced a document. From
>> the PDF describing the submission set sourceId:
>> *********
>> OID identifying the instance of the Document Source that
>> contributed the Submission Set. When a "broker" is involved
>> in sending submission sets from a collection of client systems,
>> it should use a different source ID for submissions from each
>> separate system to allow for tracking.
>> *********
>>
>> So, in light of this, it seems that the bridge should have another
>> parameter for sourceId. Otherwise there's no way for a client of the
>> bridge to identify the source ID of the system that created it. In
>> fact, perhaps you could remove the uniqueId parameter and replace it
>> with the sourceId. Then we could pass a sourceId indicating the system
>> that generated a request, and you could use your current algorithm to
>> generate a uniqueId from that sourceId instead of the other way around.
>> In this way, the sourceId would truly indicate a source system, and the
>> uniqueId would be unique. It should leave enough characters for us to
>> uniquely identify systems but keep the uniqueId under the 64 character
>> limit.
>>
>> Or you could just take sourceId and uniqueId as parameters and let the
>> clients worry about passing what they want. Either way is good for me.
>> Anyone else want to chime in?
>>
>> thanks,
>> Jesse
>> Sarah Knoop wrote:
>>
>>> Hi Jesse,
>>> I have committed a fix for this bug (below) to OHF CVS. Please update
>>> from there. This issue highlighted the fact that the bridge allows
>>> only "success' and 'failure' status while IHE allows an additional
>>> 'partial success' status. This will necessitate an API change in the
>>> bridge. Another bug will be opened to address this: 209441.
>>
>>
>>> For the OID problem, our difficulty lies in ensuring that the
>>> generated uniqueID is truly unique (at least with high probability).
>>> For those 35 characters we use a combination of machine MAC address
>>> and the system time in milli's, and then a sequence number, in case
>>> these are generated in rapid succession. We use the sourceId as a base
>>> to further ensure uniqueness. Currently the bridge is installed
>>> locally by each vendor, but it's intent is to be a centralized service
>>> for many users. Hence the precautions and why using the source ID will
>>> help with uniqueness.
>>
>>
>>> In my note below, I was a bit confused (I read your email too fast).
>>> So for this reason above, I'd like to keep the correlation between
>>> sourceId and submission set unique id. But, IHE does not require it
>>> and (now arguing with my self :-)) as I stated below the metadata
>>> preserves this correlation in the XML, so there is no need to do so
>>> for that reason.
>>
>>
>>> I'll discuss with Matt. It's a simple change, as we have a method to
>>> generate OIDs using a default prefix (something like 1.2.8). But, I'm
>>> still torn as to what is the best approach and what we will end up
>>> with in terms of uniqueness, in the end.
>>
>>
>>> Does the community have an opinion on this one??
>>> - Sarah
>>
>>
>>
>>> Jesse Pangburn wrote:
>>>
>>>> Hi Sarah,
>>>> For the sourceId, I don't have too much choice. "1.3.6.1.4.1.11854"
>>>> is my company OID, ".1" adds R&D department, and I added ".100" for
>>>> IHE testing. So I'm stuck with the "1.3.6.1.4.1.11854.1.100." 24
>>>> characters already. The bridge generates the uniqueId from the
>>>> sourceId by appending 35 characters (seems consistent). So 24+35=59,
>>>> that leaves me with 5 characters for all my IHE testing OIDs. I
>>>> would think most people are in a similar situation?
>>>>
>>>> I don't think you (the bridge) need to be throwing errors for the 64
>>>> character limit, though it wouldn't hurt, because as you said the
>>>> repository will complain. That's fine, I agree you don't need to
>>>> bother.
>>>>
>>>> The thing I would suggest (er... request) is that you either don't
>>>> build the uniqueId from the sourceId, or if you do- use less
>>>> characters? If the sourceId is already unique, then just adding one
>>>> more character would make it unique and different from the sourceId.
>>>>
>>>> As for the success response when there's an error, I submitted Bug
>>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>>
>>>> thanks
>>>> Jesse
>>>>
>>>> Sarah Knoop wrote:
>>>>
>>>>> Hi Jesse -
>>>>
>>>>
>>>>
>>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>>> in that it only allows 64 charater representaion of this attribute.
>>>>> EbXML 3.0 (as used in XDS.b) ... extends this to 256, much more
>>>>> reasonable.
>>>>
>>>>
>>>>
>>>>> Is there any particular reason for such a long sourceId?
>>>>
>>>>
>>>>
>>>>> Typically, what I recommend is to use a single OID that identifies
>>>>> the organization. Off of that oid, one can build an oid for each
>>>>> system in the organization (the sourceId) and uniqueIds for
>>>>> documents from that organization. Since the metadata tracks both the
>>>>> sourceId and the document uniqueId, there is no need to build the
>>>>> document uniqueId off of the sourceId oid ... but maybe this is
>>>>> something our code is doing.
>>>>
>>>>
>>>>
>>>>> This is a known bug that there is no particularly good workaround:
>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>>> I still am undecided as to what to do. Enforcement of the 64
>>>>> character limit at every level in OHF code seems redundant as when
>>>>> it is submitted to the repository, this error will surface anyway
>>>>> and the end result is some exception for our users in both cases. So
>>>>> ... I just need to figure out where to draw the line.
>>>>
>>>>
>>>>
>>>>> As for the miss-reported status, this is a bug. Please submit
>>>>> another report regarding this issue. Submit a second if you are not
>>>>> building the uniqueId from the sourceId yourself (this is
>>>>> something I'll have to discuss with Matt) and could have API
>>>>> implications ... or we adjust our OID generation algorithm and start
>>>>> throwing exceptions in OHF with this issue.
>>>>
>>>>
>>>>
>>>>> - Sarah
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Jesse Pangburn wrote:
>>>>>
>>>>>> Oh, one more thing about this. The really bad part is that the
>>>>>> bridge responds with:
>>>>>> <ns4:success
>>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>>
>>>>>> It also shows a documentEntryUUID so it appears as if the document
>>>>>> was successfully submitted. The only way I knew there was an error
>>>>>> was because I was watching the log.
>>>>>>
>>>>>> thanks,
>>>>>> Jesse
>>>>>>
>>>>>> Jesse Pangburn wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>>> generation. I passed
>>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the submission
>>>>>>> set's sourceId, and the bridge generated
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>>> following snippet of the log entry's printout of the message
>>>>>>> transmitted:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> <rim:ExternalIdentifier
>>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Unfortunately, the repository complained about the length of
>>>>>>> uniqueId in this snippet from the response:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>>> '64' for type 'ShortName'
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I just did a quick count of my characters and there are about 45.
>>>>>>> The unique id generation brings the count to 80, so it adds about
>>>>>>> 35 characters. If this is typical, it means that the bridge user
>>>>>>> cannot pass an OID for the submission set sourceId that is longer
>>>>>>> than about 29 characters. In my case, the fixed part of the OID
>>>>>>> is "1.3.6.1.4.1.11854.1.100." (my IHE testing OID) which is 24
>>>>>>> characters- leaving me only 5 characters to generate all my OIDs
>>>>>>> for IHE testing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I don't really know what the technical difference is between the
>>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>>> much longer? Can it just be the same?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> thanks,
>>>>>>> Jesse
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: submissionset.uniqueId generation problem [message #583438 is a reply to message #37800] Mon, 03 December 2007 18:27 Go to previous message
Matthew DavisFriend
Messages: 269
Registered: July 2009
Senior Member
Hi Jesse,

Nope... this was the one issue that fell through the cracks after I got
back. It is now fixed and a new build is generating right now - it'll
be available for download in a couple hours.

API change is as follows:

In the "DocumentSubmissionMetadata" type, there is a new value
'submissionSetUniqueId'. The Bridge will no longer auto-generate the
SubmissionSet UniqueId based on the SourceId.
DocumentSubmissionMetadata.submissionSetUniqueId is a required value and
the Bridge will throw an exception if empty.

Thanks,
-Matt


Jesse Pangburn wrote:
> Hi Sarah,
> I tried out the latest build of the bridge and it seems this bug is not
> fixed. It occurred to me that maybe there was never a bug entered for
> this in bugzilla. Do you know if this is the case? Or maybe it's on
> your list still but you haven't gotten to it yet?
>
> Otherwise the new build is working great for me. Just wanted to check
> if this one is on the list :-)
>
> thanks,
> Jesse
>
> Sarah Knoop wrote:
>
>> Hi Jesse,
>
>> Not a "stupid question" :-). The new profile XCA utilizes the same
>> transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
>> profile a query can be farmed out to many servers and the responses
>> aggregated. If one of the many servers is down, say, then this will
>> return a 'failure' while the others return 'success'. Hence the
>> 'partial success status' ... Now this is not going to be the case for
>> submitting the document, but to keep things consistent across the
>> bridge api, I'd like all three status to be available.
>
>> I think your suggestion below, to open the bridge api for both
>> sourceId and uniqueID is the best suggestion. We'll role that in with
>> the next set of API changes needed, resulting from your other bugs
>> :-). We will also need to make updates per the specification section
>> you have quoted below. We'll have to think how best to implement
>> (document) this.
>
>> Thanks for being a great tester:-)
>> - Sarah
>
>
>
>
>> Jesse Pangburn wrote:
>>> Hi Sarah,
>>> This is probably a dumb question, but what would "partial success"
>>> mean in this case? Why is it not just a failure?
>>>
>>> As for the sourceId, I just looked that up in the ITI TF Vol 2 on
>>> page 122 in the Rev 3.0 final text version. Apparently it doesn't
>>> need to be unique- it identifies the source system that produced a
>>> document. From the PDF describing the submission set sourceId:
>>> *********
>>> OID identifying the instance of the Document Source that
>>> contributed the Submission Set. When a "broker" is involved
>>> in sending submission sets from a collection of client systems,
>>> it should use a different source ID for submissions from each
>>> separate system to allow for tracking.
>>> *********
>>>
>>> So, in light of this, it seems that the bridge should have another
>>> parameter for sourceId. Otherwise there's no way for a client of the
>>> bridge to identify the source ID of the system that created it. In
>>> fact, perhaps you could remove the uniqueId parameter and replace it
>>> with the sourceId. Then we could pass a sourceId indicating the
>>> system that generated a request, and you could use your current
>>> algorithm to generate a uniqueId from that sourceId instead of the
>>> other way around. In this way, the sourceId would truly indicate a
>>> source system, and the uniqueId would be unique. It should leave
>>> enough characters for us to uniquely identify systems but keep the
>>> uniqueId under the 64 character limit.
>>>
>>> Or you could just take sourceId and uniqueId as parameters and let
>>> the clients worry about passing what they want. Either way is good
>>> for me. Anyone else want to chime in?
>>>
>>> thanks,
>>> Jesse
>>> Sarah Knoop wrote:
>>>
>>>> Hi Jesse,
>>>> I have committed a fix for this bug (below) to OHF CVS. Please
>>>> update from there. This issue highlighted the fact that the bridge
>>>> allows only "success' and 'failure' status while IHE allows an
>>>> additional 'partial success' status. This will necessitate an API
>>>> change in the bridge. Another bug will be opened to address this:
>>>> 209441.
>>>
>>>
>>>> For the OID problem, our difficulty lies in ensuring that the
>>>> generated uniqueID is truly unique (at least with high probability).
>>>> For those 35 characters we use a combination of machine MAC address
>>>> and the system time in milli's, and then a sequence number, in case
>>>> these are generated in rapid succession. We use the sourceId as a
>>>> base to further ensure uniqueness. Currently the bridge is installed
>>>> locally by each vendor, but it's intent is to be a centralized
>>>> service for many users. Hence the precautions and why using the
>>>> source ID will help with uniqueness.
>>>
>>>
>>>> In my note below, I was a bit confused (I read your email too fast).
>>>> So for this reason above, I'd like to keep the correlation between
>>>> sourceId and submission set unique id. But, IHE does not require it
>>>> and (now arguing with my self :-)) as I stated below the metadata
>>>> preserves this correlation in the XML, so there is no need to do so
>>>> for that reason.
>>>
>>>
>>>> I'll discuss with Matt. It's a simple change, as we have a method to
>>>> generate OIDs using a default prefix (something like 1.2.8). But,
>>>> I'm still torn as to what is the best approach and what we will end
>>>> up with in terms of uniqueness, in the end.
>>>
>>>
>>>> Does the community have an opinion on this one??
>>>> - Sarah
>>>
>>>
>>>
>>>> Jesse Pangburn wrote:
>>>>
>>>>> Hi Sarah,
>>>>> For the sourceId, I don't have too much choice.
>>>>> "1.3.6.1.4.1.11854" is my company OID, ".1" adds R&D department,
>>>>> and I added ".100" for IHE testing. So I'm stuck with the
>>>>> "1.3.6.1.4.1.11854.1.100." 24 characters already. The bridge
>>>>> generates the uniqueId from the sourceId by appending 35 characters
>>>>> (seems consistent). So 24+35=59, that leaves me with 5 characters
>>>>> for all my IHE testing OIDs. I would think most people are in a
>>>>> similar situation?
>>>>>
>>>>> I don't think you (the bridge) need to be throwing errors for the
>>>>> 64 character limit, though it wouldn't hurt, because as you said
>>>>> the repository will complain. That's fine, I agree you don't need
>>>>> to bother.
>>>>>
>>>>> The thing I would suggest (er... request) is that you either don't
>>>>> build the uniqueId from the sourceId, or if you do- use less
>>>>> characters? If the sourceId is already unique, then just adding
>>>>> one more character would make it unique and different from the
>>>>> sourceId.
>>>>>
>>>>> As for the success response when there's an error, I submitted Bug
>>>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>>>
>>>>> thanks
>>>>> Jesse
>>>>>
>>>>> Sarah Knoop wrote:
>>>>>
>>>>>> Hi Jesse -
>>>>>
>>>>>
>>>>>
>>>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>>>> in that it only allows 64 charater representaion of this
>>>>>> attribute. EbXML 3.0 (as used in XDS.b) ... extends this to 256,
>>>>>> much more reasonable.
>>>>>
>>>>>
>>>>>
>>>>>> Is there any particular reason for such a long sourceId?
>>>>>
>>>>>
>>>>>
>>>>>> Typically, what I recommend is to use a single OID that
>>>>>> identifies the organization. Off of that oid, one can build an oid
>>>>>> for each system in the organization (the sourceId) and uniqueIds
>>>>>> for documents from that organization. Since the metadata tracks
>>>>>> both the sourceId and the document uniqueId, there is no need to
>>>>>> build the document uniqueId off of the sourceId oid ... but maybe
>>>>>> this is something our code is doing.
>>>>>
>>>>>
>>>>>
>>>>>> This is a known bug that there is no particularly good workaround:
>>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>>>> I still am undecided as to what to do. Enforcement of the 64
>>>>>> character limit at every level in OHF code seems redundant as when
>>>>>> it is submitted to the repository, this error will surface anyway
>>>>>> and the end result is some exception for our users in both cases.
>>>>>> So ... I just need to figure out where to draw the line.
>>>>>
>>>>>
>>>>>
>>>>>> As for the miss-reported status, this is a bug. Please submit
>>>>>> another report regarding this issue. Submit a second if you are
>>>>>> not building the uniqueId from the sourceId yourself (this is
>>>>>> something I'll have to discuss with Matt) and could have API
>>>>>> implications ... or we adjust our OID generation algorithm and
>>>>>> start throwing exceptions in OHF with this issue.
>>>>>
>>>>>
>>>>>
>>>>>> - Sarah
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Jesse Pangburn wrote:
>>>>>>
>>>>>>> Oh, one more thing about this. The really bad part is that the
>>>>>>> bridge responds with:
>>>>>>> <ns4:success
>>>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>>>
>>>>>>> It also shows a documentEntryUUID so it appears as if the
>>>>>>> document was successfully submitted. The only way I knew there
>>>>>>> was an error was because I was watching the log.
>>>>>>>
>>>>>>> thanks,
>>>>>>> Jesse
>>>>>>>
>>>>>>> Jesse Pangburn wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>>>> generation. I passed
>>>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
>>>>>>>> submission set's sourceId, and the bridge generated
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> " 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>>>> following snippet of the log entry's printout of the message
>>>>>>>> transmitted:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> <rim:ExternalIdentifier
>>>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Unfortunately, the repository complained about the length of
>>>>>>>> uniqueId in this snippet from the response:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
> '1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>
>>>
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>>>> '64' for type 'ShortName'
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> I just did a quick count of my characters and there are about
>>>>>>>> 45. The unique id generation brings the count to 80, so it adds
>>>>>>>> about 35 characters. If this is typical, it means that the
>>>>>>>> bridge user cannot pass an OID for the submission set sourceId
>>>>>>>> that is longer than about 29 characters. In my case, the fixed
>>>>>>>> part of the OID is "1.3.6.1.4.1.11854.1.100." (my IHE testing
>>>>>>>> OID) which is 24 characters- leaving me only 5 characters to
>>>>>>>> generate all my OIDs for IHE testing.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> I don't really know what the technical difference is between the
>>>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>>>> much longer? Can it just be the same?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Jesse
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>
>
Re: submissionset.uniqueId generation problem [message #583453 is a reply to message #37869] Mon, 03 December 2007 20:38 Go to previous message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Matt,
Thank you very much for this and all the other fixes/improvements, you
guys are awesome!

thanks,
Jesse

Matthew Davis wrote:

> Hi Jesse,

> Nope... this was the one issue that fell through the cracks after I got
> back. It is now fixed and a new build is generating right now - it'll
> be available for download in a couple hours.

> API change is as follows:

> In the "DocumentSubmissionMetadata" type, there is a new value
> 'submissionSetUniqueId'. The Bridge will no longer auto-generate the
> SubmissionSet UniqueId based on the SourceId.
> DocumentSubmissionMetadata.submissionSetUniqueId is a required value and
> the Bridge will throw an exception if empty.

> Thanks,
> -Matt


> Jesse Pangburn wrote:
>> Hi Sarah,
>> I tried out the latest build of the bridge and it seems this bug is not
>> fixed. It occurred to me that maybe there was never a bug entered for
>> this in bugzilla. Do you know if this is the case? Or maybe it's on
>> your list still but you haven't gotten to it yet?
>>
>> Otherwise the new build is working great for me. Just wanted to check
>> if this one is on the list :-)
>>
>> thanks,
>> Jesse
>>
>> Sarah Knoop wrote:
>>
>>> Hi Jesse,
>>
>>> Not a "stupid question" :-). The new profile XCA utilizes the same
>>> transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
>>> profile a query can be farmed out to many servers and the responses
>>> aggregated. If one of the many servers is down, say, then this will
>>> return a 'failure' while the others return 'success'. Hence the
>>> 'partial success status' ... Now this is not going to be the case for
>>> submitting the document, but to keep things consistent across the
>>> bridge api, I'd like all three status to be available.
>>
>>> I think your suggestion below, to open the bridge api for both
>>> sourceId and uniqueID is the best suggestion. We'll role that in with
>>> the next set of API changes needed, resulting from your other bugs
>>> :-). We will also need to make updates per the specification section
>>> you have quoted below. We'll have to think how best to implement
>>> (document) this.
>>
>>> Thanks for being a great tester:-)
>>> - Sarah
>>
>>
>>
>>
>>> Jesse Pangburn wrote:
>>>> Hi Sarah,
>>>> This is probably a dumb question, but what would "partial success"
>>>> mean in this case? Why is it not just a failure?
>>>>
>>>> As for the sourceId, I just looked that up in the ITI TF Vol 2 on
>>>> page 122 in the Rev 3.0 final text version. Apparently it doesn't
>>>> need to be unique- it identifies the source system that produced a
>>>> document. From the PDF describing the submission set sourceId:
>>>> *********
>>>> OID identifying the instance of the Document Source that
>>>> contributed the Submission Set. When a "broker" is involved
>>>> in sending submission sets from a collection of client systems,
>>>> it should use a different source ID for submissions from each
>>>> separate system to allow for tracking.
>>>> *********
>>>>
>>>> So, in light of this, it seems that the bridge should have another
>>>> parameter for sourceId. Otherwise there's no way for a client of the
>>>> bridge to identify the source ID of the system that created it. In
>>>> fact, perhaps you could remove the uniqueId parameter and replace it
>>>> with the sourceId. Then we could pass a sourceId indicating the
>>>> system that generated a request, and you could use your current
>>>> algorithm to generate a uniqueId from that sourceId instead of the
>>>> other way around. In this way, the sourceId would truly indicate a
>>>> source system, and the uniqueId would be unique. It should leave
>>>> enough characters for us to uniquely identify systems but keep the
>>>> uniqueId under the 64 character limit.
>>>>
>>>> Or you could just take sourceId and uniqueId as parameters and let
>>>> the clients worry about passing what they want. Either way is good
>>>> for me. Anyone else want to chime in?
>>>>
>>>> thanks,
>>>> Jesse
>>>> Sarah Knoop wrote:
>>>>
>>>>> Hi Jesse,
>>>>> I have committed a fix for this bug (below) to OHF CVS. Please
>>>>> update from there. This issue highlighted the fact that the bridge
>>>>> allows only "success' and 'failure' status while IHE allows an
>>>>> additional 'partial success' status. This will necessitate an API
>>>>> change in the bridge. Another bug will be opened to address this:
>>>>> 209441.
>>>>
>>>>
>>>>> For the OID problem, our difficulty lies in ensuring that the
>>>>> generated uniqueID is truly unique (at least with high probability).
>>>>> For those 35 characters we use a combination of machine MAC address
>>>>> and the system time in milli's, and then a sequence number, in case
>>>>> these are generated in rapid succession. We use the sourceId as a
>>>>> base to further ensure uniqueness. Currently the bridge is installed
>>>>> locally by each vendor, but it's intent is to be a centralized
>>>>> service for many users. Hence the precautions and why using the
>>>>> source ID will help with uniqueness.
>>>>
>>>>
>>>>> In my note below, I was a bit confused (I read your email too fast).
>>>>> So for this reason above, I'd like to keep the correlation between
>>>>> sourceId and submission set unique id. But, IHE does not require it
>>>>> and (now arguing with my self :-)) as I stated below the metadata
>>>>> preserves this correlation in the XML, so there is no need to do so
>>>>> for that reason.
>>>>
>>>>
>>>>> I'll discuss with Matt. It's a simple change, as we have a method to
>>>>> generate OIDs using a default prefix (something like 1.2.8). But,
>>>>> I'm still torn as to what is the best approach and what we will end
>>>>> up with in terms of uniqueness, in the end.
>>>>
>>>>
>>>>> Does the community have an opinion on this one??
>>>>> - Sarah
>>>>
>>>>
>>>>
>>>>> Jesse Pangburn wrote:
>>>>>
>>>>>> Hi Sarah,
>>>>>> For the sourceId, I don't have too much choice.
>>>>>> "1.3.6.1.4.1.11854" is my company OID, ".1" adds R&D department,
>>>>>> and I added ".100" for IHE testing. So I'm stuck with the
>>>>>> "1.3.6.1.4.1.11854.1.100." 24 characters already. The bridge
>>>>>> generates the uniqueId from the sourceId by appending 35 characters
>>>>>> (seems consistent). So 24+35=59, that leaves me with 5 characters
>>>>>> for all my IHE testing OIDs. I would think most people are in a
>>>>>> similar situation?
>>>>>>
>>>>>> I don't think you (the bridge) need to be throwing errors for the
>>>>>> 64 character limit, though it wouldn't hurt, because as you said
>>>>>> the repository will complain. That's fine, I agree you don't need
>>>>>> to bother.
>>>>>>
>>>>>> The thing I would suggest (er... request) is that you either don't
>>>>>> build the uniqueId from the sourceId, or if you do- use less
>>>>>> characters? If the sourceId is already unique, then just adding
>>>>>> one more character would make it unique and different from the
>>>>>> sourceId.
>>>>>>
>>>>>> As for the success response when there's an error, I submitted Bug
>>>>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>>>>
>>>>>> thanks
>>>>>> Jesse
>>>>>>
>>>>>> Sarah Knoop wrote:
>>>>>>
>>>>>>> Hi Jesse -
>>>>>>
>>>>>>
>>>>>>
>>>>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>>>>> in that it only allows 64 charater representaion of this
>>>>>>> attribute. EbXML 3.0 (as used in XDS.b) ... extends this to 256,
>>>>>>> much more reasonable.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Is there any particular reason for such a long sourceId?
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Typically, what I recommend is to use a single OID that
>>>>>>> identifies the organization. Off of that oid, one can build an oid
>>>>>>> for each system in the organization (the sourceId) and uniqueIds
>>>>>>> for documents from that organization. Since the metadata tracks
>>>>>>> both the sourceId and the document uniqueId, there is no need to
>>>>>>> build the document uniqueId off of the sourceId oid ... but maybe
>>>>>>> this is something our code is doing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> This is a known bug that there is no particularly good workaround:
>>>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>>>>> I still am undecided as to what to do. Enforcement of the 64
>>>>>>> character limit at every level in OHF code seems redundant as when
>>>>>>> it is submitted to the repository, this error will surface anyway
>>>>>>> and the end result is some exception for our users in both cases.
>>>>>>> So ... I just need to figure out where to draw the line.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> As for the miss-reported status, this is a bug. Please submit
>>>>>>> another report regarding this issue. Submit a second if you are
>>>>>>> not building the uniqueId from the sourceId yourself (this is
>>>>>>> something I'll have to discuss with Matt) and could have API
>>>>>>> implications ... or we adjust our OID generation algorithm and
>>>>>>> start throwing exceptions in OHF with this issue.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> - Sarah
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Jesse Pangburn wrote:
>>>>>>>
>>>>>>>> Oh, one more thing about this. The really bad part is that the
>>>>>>>> bridge responds with:
>>>>>>>> <ns4:success
>>>>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>>>>
>>>>>>>> It also shows a documentEntryUUID so it appears as if the
>>>>>>>> document was successfully submitted. The only way I knew there
>>>>>>>> was an error was because I was watching the log.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Jesse
>>>>>>>>
>>>>>>>> Jesse Pangburn wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>>>>> generation. I passed
>>>>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
>>>>>>>>> submission set's sourceId, and the bridge generated
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>>>>> following snippet of the log entry's printout of the message
>>>>>>>>> transmitted:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> <rim:ExternalIdentifier
>>>>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Unfortunately, the repository complained about the length of
>>>>>>>>> uniqueId in this snippet from the response:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>>>>> '64' for type 'ShortName'
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I just did a quick count of my characters and there are about
>>>>>>>>> 45. The unique id generation brings the count to 80, so it adds
>>>>>>>>> about 35 characters. If this is typical, it means that the
>>>>>>>>> bridge user cannot pass an OID for the submission set sourceId
>>>>>>>>> that is longer than about 29 characters. In my case, the fixed
>>>>>>>>> part of the OID is "1.3.6.1.4.1.11854.1.100." (my IHE testing
>>>>>>>>> OID) which is 24 characters- leaving me only 5 characters to
>>>>>>>>> generate all my OIDs for IHE testing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I don't really know what the technical difference is between the
>>>>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>>>>> much longer? Can it just be the same?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> Jesse
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: submissionset.uniqueId generation problem [message #583514 is a reply to message #37869] Tue, 04 December 2007 21:56 Go to previous message
Jesse Pangburn is currently offline Jesse PangburnFriend
Messages: 166
Registered: July 2009
Senior Member
Hi Matt,
I've tested this and it appears to be working fine. I verified the
XDSSubmissionSet.uniqueId and XDSSubmissionSet.sourceId values in the
SubmitObjectsRequest message.

thanks,
Jesse

Matthew Davis wrote:

> Hi Jesse,

> Nope... this was the one issue that fell through the cracks after I got
> back. It is now fixed and a new build is generating right now - it'll
> be available for download in a couple hours.

> API change is as follows:

> In the "DocumentSubmissionMetadata" type, there is a new value
> 'submissionSetUniqueId'. The Bridge will no longer auto-generate the
> SubmissionSet UniqueId based on the SourceId.
> DocumentSubmissionMetadata.submissionSetUniqueId is a required value and
> the Bridge will throw an exception if empty.

> Thanks,
> -Matt


> Jesse Pangburn wrote:
>> Hi Sarah,
>> I tried out the latest build of the bridge and it seems this bug is not
>> fixed. It occurred to me that maybe there was never a bug entered for
>> this in bugzilla. Do you know if this is the case? Or maybe it's on
>> your list still but you haven't gotten to it yet?
>>
>> Otherwise the new build is working great for me. Just wanted to check
>> if this one is on the list :-)
>>
>> thanks,
>> Jesse
>>
>> Sarah Knoop wrote:
>>
>>> Hi Jesse,
>>
>>> Not a "stupid question" :-). The new profile XCA utilizes the same
>>> transactions as XDS.b (and in the case of StoredQuery, XDS.a). In this
>>> profile a query can be farmed out to many servers and the responses
>>> aggregated. If one of the many servers is down, say, then this will
>>> return a 'failure' while the others return 'success'. Hence the
>>> 'partial success status' ... Now this is not going to be the case for
>>> submitting the document, but to keep things consistent across the
>>> bridge api, I'd like all three status to be available.
>>
>>> I think your suggestion below, to open the bridge api for both
>>> sourceId and uniqueID is the best suggestion. We'll role that in with
>>> the next set of API changes needed, resulting from your other bugs
>>> :-). We will also need to make updates per the specification section
>>> you have quoted below. We'll have to think how best to implement
>>> (document) this.
>>
>>> Thanks for being a great tester:-)
>>> - Sarah
>>
>>
>>
>>
>>> Jesse Pangburn wrote:
>>>> Hi Sarah,
>>>> This is probably a dumb question, but what would "partial success"
>>>> mean in this case? Why is it not just a failure?
>>>>
>>>> As for the sourceId, I just looked that up in the ITI TF Vol 2 on
>>>> page 122 in the Rev 3.0 final text version. Apparently it doesn't
>>>> need to be unique- it identifies the source system that produced a
>>>> document. From the PDF describing the submission set sourceId:
>>>> *********
>>>> OID identifying the instance of the Document Source that
>>>> contributed the Submission Set. When a "broker" is involved
>>>> in sending submission sets from a collection of client systems,
>>>> it should use a different source ID for submissions from each
>>>> separate system to allow for tracking.
>>>> *********
>>>>
>>>> So, in light of this, it seems that the bridge should have another
>>>> parameter for sourceId. Otherwise there's no way for a client of the
>>>> bridge to identify the source ID of the system that created it. In
>>>> fact, perhaps you could remove the uniqueId parameter and replace it
>>>> with the sourceId. Then we could pass a sourceId indicating the
>>>> system that generated a request, and you could use your current
>>>> algorithm to generate a uniqueId from that sourceId instead of the
>>>> other way around. In this way, the sourceId would truly indicate a
>>>> source system, and the uniqueId would be unique. It should leave
>>>> enough characters for us to uniquely identify systems but keep the
>>>> uniqueId under the 64 character limit.
>>>>
>>>> Or you could just take sourceId and uniqueId as parameters and let
>>>> the clients worry about passing what they want. Either way is good
>>>> for me. Anyone else want to chime in?
>>>>
>>>> thanks,
>>>> Jesse
>>>> Sarah Knoop wrote:
>>>>
>>>>> Hi Jesse,
>>>>> I have committed a fix for this bug (below) to OHF CVS. Please
>>>>> update from there. This issue highlighted the fact that the bridge
>>>>> allows only "success' and 'failure' status while IHE allows an
>>>>> additional 'partial success' status. This will necessitate an API
>>>>> change in the bridge. Another bug will be opened to address this:
>>>>> 209441.
>>>>
>>>>
>>>>> For the OID problem, our difficulty lies in ensuring that the
>>>>> generated uniqueID is truly unique (at least with high probability).
>>>>> For those 35 characters we use a combination of machine MAC address
>>>>> and the system time in milli's, and then a sequence number, in case
>>>>> these are generated in rapid succession. We use the sourceId as a
>>>>> base to further ensure uniqueness. Currently the bridge is installed
>>>>> locally by each vendor, but it's intent is to be a centralized
>>>>> service for many users. Hence the precautions and why using the
>>>>> source ID will help with uniqueness.
>>>>
>>>>
>>>>> In my note below, I was a bit confused (I read your email too fast).
>>>>> So for this reason above, I'd like to keep the correlation between
>>>>> sourceId and submission set unique id. But, IHE does not require it
>>>>> and (now arguing with my self :-)) as I stated below the metadata
>>>>> preserves this correlation in the XML, so there is no need to do so
>>>>> for that reason.
>>>>
>>>>
>>>>> I'll discuss with Matt. It's a simple change, as we have a method to
>>>>> generate OIDs using a default prefix (something like 1.2.8). But,
>>>>> I'm still torn as to what is the best approach and what we will end
>>>>> up with in terms of uniqueness, in the end.
>>>>
>>>>
>>>>> Does the community have an opinion on this one??
>>>>> - Sarah
>>>>
>>>>
>>>>
>>>>> Jesse Pangburn wrote:
>>>>>
>>>>>> Hi Sarah,
>>>>>> For the sourceId, I don't have too much choice.
>>>>>> "1.3.6.1.4.1.11854" is my company OID, ".1" adds R&D department,
>>>>>> and I added ".100" for IHE testing. So I'm stuck with the
>>>>>> "1.3.6.1.4.1.11854.1.100." 24 characters already. The bridge
>>>>>> generates the uniqueId from the sourceId by appending 35 characters
>>>>>> (seems consistent). So 24+35=59, that leaves me with 5 characters
>>>>>> for all my IHE testing OIDs. I would think most people are in a
>>>>>> similar situation?
>>>>>>
>>>>>> I don't think you (the bridge) need to be throwing errors for the
>>>>>> 64 character limit, though it wouldn't hurt, because as you said
>>>>>> the repository will complain. That's fine, I agree you don't need
>>>>>> to bother.
>>>>>>
>>>>>> The thing I would suggest (er... request) is that you either don't
>>>>>> build the uniqueId from the sourceId, or if you do- use less
>>>>>> characters? If the sourceId is already unique, then just adding
>>>>>> one more character would make it unique and different from the
>>>>>> sourceId.
>>>>>>
>>>>>> As for the success response when there's an error, I submitted Bug
>>>>>> 209382 https://bugs.eclipse.org/bugs/show_bug.cgi?id=209382
>>>>>>
>>>>>> thanks
>>>>>> Jesse
>>>>>>
>>>>>> Sarah Knoop wrote:
>>>>>>
>>>>>>> Hi Jesse -
>>>>>>
>>>>>>
>>>>>>
>>>>>>> For the repository error, as you noted, this is an ebXML 2.1 issue
>>>>>>> in that it only allows 64 charater representaion of this
>>>>>>> attribute. EbXML 3.0 (as used in XDS.b) ... extends this to 256,
>>>>>>> much more reasonable.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Is there any particular reason for such a long sourceId?
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Typically, what I recommend is to use a single OID that
>>>>>>> identifies the organization. Off of that oid, one can build an oid
>>>>>>> for each system in the organization (the sourceId) and uniqueIds
>>>>>>> for documents from that organization. Since the metadata tracks
>>>>>>> both the sourceId and the document uniqueId, there is no need to
>>>>>>> build the document uniqueId off of the sourceId oid ... but maybe
>>>>>>> this is something our code is doing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> This is a known bug that there is no particularly good workaround:
>>>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=182913
>>>>>>> I still am undecided as to what to do. Enforcement of the 64
>>>>>>> character limit at every level in OHF code seems redundant as when
>>>>>>> it is submitted to the repository, this error will surface anyway
>>>>>>> and the end result is some exception for our users in both cases.
>>>>>>> So ... I just need to figure out where to draw the line.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> As for the miss-reported status, this is a bug. Please submit
>>>>>>> another report regarding this issue. Submit a second if you are
>>>>>>> not building the uniqueId from the sourceId yourself (this is
>>>>>>> something I'll have to discuss with Matt) and could have API
>>>>>>> implications ... or we adjust our OID generation algorithm and
>>>>>>> start throwing exceptions in OHF with this issue.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> - Sarah
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Jesse Pangburn wrote:
>>>>>>>
>>>>>>>> Oh, one more thing about this. The really bad part is that the
>>>>>>>> bridge responds with:
>>>>>>>> <ns4:success
>>>>>>>> xmlns:ns4="http://type.bridge.ohf.eclipse.org">true</ns4:success>
>>>>>>>>
>>>>>>>> It also shows a documentEntryUUID so it appears as if the
>>>>>>>> document was successfully submitted. The only way I knew there
>>>>>>>> was an error was because I was watching the log.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> Jesse
>>>>>>>>
>>>>>>>> Jesse Pangburn wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I ran into a funny problem with the submission set's unique id
>>>>>>>>> generation. I passed
>>>>>>>>> "1.3.6.1.4.1.11854.1.100.2007.11.08.12.15.36.2" as the
>>>>>>>>> submission set's sourceId, and the bridge generated
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
" 1.3.6.1.4.1.11854.1.100.2007.11.18.12.15.36.2.11501602901714 6133.1194552938546.1 "
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> as the XDSSubmissionSet.uniqueId. This is evidenced by the
>>>>>>>>> following snippet of the log entry's printout of the message
>>>>>>>>> transmitted:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> <rim:ExternalIdentifier
>>>>>>>>> identificationScheme="urn:uuid:554ac39e-e3fe-47fe-b233-965d2a147832 "
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
value="1.3.6.1.4.1.11854.1.100.2007.11.08.13.56.21.2"><rim:Name ><rim:LocalizedString
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> value="XDSSubmissionSet.sourceId"
>>>>>>>>> /></rim:Name></rim:ExternalIdentifier><rim:ExternalIdentifier
>>>>>>>>> identificationScheme="urn:uuid:96fdda7c-d067-4183-912e-bf5ee74998a8 "
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
value=" 1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.12405701300202 3108.1194558984650.1 "><rim:Name><rim:LocalizedString
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> value="XDSSubmissionSet.uniqueId"
>>>>>>>>> /></rim:Name></rim:ExternalIdentifier>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Unfortunately, the repository complained about the length of
>>>>>>>>> uniqueId in this snippet from the response:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Schema Validation Errors:&#xa;Input did not validate against
>>>>>>>>> schema:&#xa;Error: cvc-maxLength-valid: Value
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
'1.3.6.1.4.1.11854.1.100.2007.11.18.13.56.21.2.1240570130020 23108.1194558984650.1'
>>
>>>>
>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> with length = '80' is not facet-valid with respect to maxLength
>>>>>>>>> '64' for type 'ShortName'
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I just did a quick count of my characters and there are about
>>>>>>>>> 45. The unique id generation brings the count to 80, so it adds
>>>>>>>>> about 35 characters. If this is typical, it means that the
>>>>>>>>> bridge user cannot pass an OID for the submission set sourceId
>>>>>>>>> that is longer than about 29 characters. In my case, the fixed
>>>>>>>>> part of the OID is "1.3.6.1.4.1.11854.1.100." (my IHE testing
>>>>>>>>> OID) which is 24 characters- leaving me only 5 characters to
>>>>>>>>> generate all my OIDs for IHE testing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I don't really know what the technical difference is between the
>>>>>>>>> sourceId and the uniqueId, but does the uniqueId need to be that
>>>>>>>>> much longer? Can it just be the same?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> Jesse
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Previous Topic:OutOfMemoryError from Bridge
Next Topic:non stored query error against NIST server
Goto Forum:
  


Current Time: Thu Mar 28 13:26:16 GMT 2024

Powered by FUDForum. Page generated in 0.04649 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top