Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » xsd2ttcn / bug handling substitution group?(xsd2ttcn or the xsds in TS 33.179 sem to have a bug)
xsd2ttcn / bug handling substitution group? [message #1798749] Fri, 23 November 2018 13:46 Go to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Hi XSD-Titan experts,

We at ETSI/STF160 are working developing test suites for 3GPP which require complex XSDs like the ones in
ftp://ftp.3gpp.org/specs/2018-09/Rel-13/33_series/33179-d70.zip
which use the substitution group attribute.

When working with this TS 33.179 we found following problem:
We cannot use the XSDs to generate the correct TTCN type defs. The encryption keys in the XML document should be encoded, for example as
<UserDecryptKey >
<EncryptedKey >
<EncryptionMethod Algorithm="aes256"/>
<ds:KeyInfo> <ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo> <CipherData> <CipherValue>DEADBEEF</CipherValue> </CipherData>
</EncryptedKey>
</UserDecryptKey>

The xsd2ttcn converter produces correctly
type union UserDecryptKey_group
{
KeyContentType userDecryptKey,
EncUserDecryptKey encUserDecryptKey
}

But EncUserDecryptKey resolves to simply KeyContentType instead of EncryptedKey which should be imported from module http_www_w3_org_2001_04_xmlenc.

My question: Is this an xsd2ttcn conversion issue, or the XSDs in TS 33.179 are faulty?

Attached the XSDs and the generated TTCN code.

Any hints will be appreciated.

Regards,
Olaf@ETSI-ST160

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1798937 is a reply to message #1798749] Tue, 27 November 2018 15:57 Go to previous messageGo to next message
Botond Baranyi is currently offline Botond BaranyiFriend
Messages: 34
Registered: February 2016
Member
Hi Olaf,

there are definitely issues in the XSDs. Though I'm not really sure what you're trying to achieve.

In type definition EncKeyContentType:

<xsd:complexType name="EncKeyContentType">
  <xsd:complexContent>
    <xsd:restriction base="ikms:KeyContentType">
      <xsd:sequence>
      <xsd:element ref="xenc:EncryptedKey" maxOccurs="1" minOccurs="1"/>
    </xsd:sequence>
    </xsd:restriction>
  </xsd:complexContent>
</xsd:complexType>


The keyword xsd:restriction is used incorrectly on type ikms:KeyContentType. The xsd:restriction restricts the set of values validated by the type. In this case, since ikms:KeyContentType is a hexBinary, it could be used to validate a smaller set of hexBinaries (instead of all of them). The element xenc:EncryptedKey cannot be used in this restriction, as it's not of the same type as ikms:KeyContentType.

If you want EncKeyContentType to resolve to EncryptedKey's type in the generated TTCN-3 code, then you should change it to something like this:

<xsd:complexType name="EncKeyContentType">
  <xsd:complexContent>
    <xsd:extension base="xenc:EncryptedKeyType">
    </xsd:extension>
  </xsd:complexContent>
</xsd:complexType>


Hope this helps.

Best regards,
Botond Baranyi

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1798944 is a reply to message #1798937] Tue, 27 November 2018 18:48 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Thanks, Botond,
I will try your proposal this week (I am not an XSD expert). The goal is to get the right TTCN type defs and templates , such that when encoded we will get message as in the example in TS 33.179 section D.3.3.2, see below.

Regards,
Olaf
---
TS 33.179 section D.3.3.2:


<?xml version="1.0" encoding="UTF-8"?>
<SignedKmsResponse xmlns= "urn:3gpp:ns:mcsecKMSSecExt:1.0" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"
xmlns:ds = "http://www.w3.org/2000/09/xmldsig#" xmlns:se = "urn:3gpp:ns:mcsecKMSSecExt:1.0"
xsi:schemaLocation = "urn:3gpp:ns:mcsecKMSSecExt:1.0 SE_KmsInterface_XMLSchema.xsd" Id = "xmldoc">
<KmsResponse xmlns= "urn:3gpp:ns:mcsecKMSInterface:1.0" Version = "1.0.0">
<KmsUri>kms.example.org</KmsUri>
<UserUri>example:user@example.org</UserUri>
<Time>2014-01-26T10:07:14</Time>
<KmsId>KMSProvider12345</KmsId>
<ClientReqUrl>http://kms.example.org/keymanagement/identity/v1/keyprov</ClientReqUrl>
<KmsMessage>
<KmsKeyProv Version = "1.0.0" xsi:type = "se:KmsKeyProvTkType">
<KmsKeySet Version = "1.1.0">
<KmsUri>kms.example.org</KmsUri>
<CertUri>cert1.kms.example.org</CertUri>
<Issuer>www.example.org</Issuer>
<UserUri>example:user@example.org</UserUri>
<UserID>0123456789ABCDEF0123456789ABCDEF</UserID>
<ValidFrom>2015-12-30T00:00:00</ValidFrom>
<ValidTo>2016-03-29T23:59:59</ValidTo>
<KeyPeriodNo>1514</KeyPeriodNo>
<Revoked>false</Revoked>
<UserDecryptKey xsi:type = "se:EncKeyContentType">
<EncryptedKey xmlns = "http://www.w3.org/2001/04/xmlenc#">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#kw-aes256"/>
<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo>
<CipherData>
<CipherValue>DEADBEEF</CipherValue>
</CipherData>
</EncryptedKey>
</UserDecryptKey>
<UserSigningKeySSK xsi:type = "se:EncKeyContentType">
<EncryptedKey xmlns = "http://www.w3.org/2001/04/xmlenc#">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#kw-aes256"/>
<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo>
<CipherData>
<CipherValue>DEADBEEF</CipherValue>
</CipherData>
</EncryptedKey>
</UserSigningKeySSK>
<UserPubTokenPVT xsi:type = "se:EncKeyContentType">
<EncryptedKey xmlns = "http://www.w3.org/2001/04/xmlenc#">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#kw-aes256"/>
<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo>
<CipherData>
<CipherValue>DEADBEEF</CipherValue>
</CipherData>
</EncryptedKey>
</UserPubTokenPVT>
</KmsKeySet>
<KmsKeySet Version = "1.1.0">
<KmsUri>kms.example.org</KmsUri>
<CertUri>cert1.kms.example.org</CertUri>
<Issuer>www.example.org</Issuer>
<UserUri>example:user.pseudonym@example.org</UserUri>
<UserID>0011223344556677889900AABBCCDDEEFF</UserID>
<ValidFrom>2015-12-30T00:00:00</ValidFrom>
<ValidTo>2016-03-29T23:59:59</ValidTo>
<ValidTo>2016-03-29T23:59:59</ValidTo>
<KeyPeriodNo>1514</KeyPeriodNo>
<Revoked>false</Revoked>
<UserDecryptKey xsi:type = "se:EncKeyContentType">
<EncryptedKey xmlns = "http://www.w3.org/2001/04/xmlenc#">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#kw-aes256"/>
<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo>
<CipherData>
<CipherValue>DEADBEEF</CipherValue>
</CipherData>
</EncryptedKey>
</UserDecryptKey>
<UserSigningKeySSK xsi:type = "se:EncKeyContentType">
<EncryptedKey xmlns = "http://www.w3.org/2001/04/xmlenc#">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#kw-aes256"/>
<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo>
<CipherData>
<CipherValue>DEADBEEF</CipherValue>
</CipherData>
</EncryptedKey>
</UserSigningKeySSK>
<UserPubTokenPVT xsi:type = "se:EncKeyContentType">
<EncryptedKey xmlns = "http://www.w3.org/2001/04/xmlenc#">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#kw-aes256"/>
<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo>
<CipherData>
<CipherValue>DEADBEEF</CipherValue>
</CipherData>
</EncryptedKey>
</UserPubTokenPVT>
</KmsKeySet>
<NewTransportKey xmlns= "urn:3gpp:ns:mcsecKMSInterface:1.0">
<EncryptedKey xmlns="http://www.w3.org/2001/04/xmlenc#" Type="http://www.w3.org/2001/04/xmlenc#EncryptedKey">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#kw-aes256"/>
<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo>
<CipherData>
<CipherValue>DEADBEEF</CipherValue>
</CipherData>
<CarriedKeyName>tk.13.user@example.org</CarriedKeyName>
</EncryptedKey>
</NewTransportKey>
</KmsKeyProv>
</KmsMessage>
</KmsResponse>
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
<CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
<SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#hmac-sha256">
<HMACOutputLength>128</HMACOutputLength>
</SignatureMethod>
<Reference URI="#xmldoc">
<DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
<DigestValue>nnnn</DigestValue>
</Reference>
</SignedInfo>
<SignatureValue>DEADBEEF</SignatureValue>
<KeyInfo>
<KeyName>tk.12.user@example.org</KeyName>
</KeyInfo>
</Signature>
</SignedKmsResponse>

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1798991 is a reply to message #1798944] Wed, 28 November 2018 11:47 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi OLaf,

when verified with xmllint, the above XSD presents several inconsistencies:


all
<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</KeyName>
</ds:KeyInfo>



should be changed to 

<ds:KeyInfo>
<ds:KeyName>tk.12.user@example.org</ds:KeyName>
</ds:KeyInfo>


and teh final 


<KeyInfo>
<KeyName>tk.12.user@example.org</KeyName>
</KeyInfo>


is maybe in the wrong namespace too.

Just my two cents

BR

Elemer

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799033 is a reply to message #1798991] Thu, 29 November 2018 07:48 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Hi Botond, Elemer,

I tried the proposal from Botond, which resulted in the correct resolution of EncKeyContentType.
But unfortunately the TTCN code does not compile, e.g. some import is missing in moduleurn_3gpp_ns_mcsecKMSSecExt_1_0.ttcn.

Attached the XSDs+TTCN files.

And Elemer, I will report your two cents inconsistencies in the example in the spec (I assume you did not validate against the XSDs, is just an XML syntax inconsistency).

Regards,
Olaf@ETSI-urn_3gpp_ns_mcsecKMSSecExt_1_0.ttcnTF160

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799099 is a reply to message #1798749] Thu, 29 November 2018 17:18 Go to previous messageGo to next message
Botond Baranyi is currently offline Botond BaranyiFriend
Messages: 34
Registered: February 2016
Member
Hi Olaf,

This is a known limitation of our xsd2ttcn tool. This occurs, because members of a substitution group are defined in several XSDs.

A solution to this problem is to generate all definitions into one TTCN file using the xsd2ttcn command line option '-o'.

However, this seems to have some issues as well (duplicate definitions with the same name or not handled, and certain references are still prefixed with module names). I'll look into these issues.

Best regards,
Botond Baranyi

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799156 is a reply to message #1799099] Fri, 30 November 2018 15:42 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Hi Botond,

I am new to Titan (and very excited on trying it) and would like to experiment with encoding /decoding simple SIP with XSD contents.
Do you know if someone has a simple Titan project with some simple SIP and XSD types and without external ports which I could download and compile and actually execute encval/decval?

Regards and have a nice weekend,
Olaf

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799183 is a reply to message #1799156] Sat, 01 December 2018 07:54 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Olaf,


the first place you should look into are conformance tests and regression tests included in the Titan source code repo :

https://github.com/eclipse/titan.core

there are large amount of tests for XML

Also browse this forum for relevant posts, e.g.:

https://www.eclipse.org/forums/index.php/t/1072874/
Testing SIPmsg demo defined in titan.TestPorts.SIPmsg

for SIP

( there's a simple SIP demo in https://github.com/eclipse/titan.TestPorts.SIPmsg)


and

Executing ETSI Diameter Rx test cases ( contains some SIP explanations)

and



https://www.eclipse.org/forums/index.php/t/1077272/
XML encoding in TTCN-3 and Titan part 1: basics

https://www.eclipse.org/forums/index.php/t/1077502/
XML encoding in TTCN-3 and Titan part II: example


for XML



I hope this helps

Best regards
Elemer

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799204 is a reply to message #1799183] Sat, 01 December 2018 21:55 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Wow, Elemer, this is impressive!
Thanks!
Olaf

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799677 is a reply to message #1798749] Wed, 12 December 2018 09:17 Go to previous messageGo to next message
Botond Baranyi is currently offline Botond BaranyiFriend
Messages: 34
Registered: February 2016
Member
Hi Olaf,

I've fixed the mentioned issues with xsd2ttcn's single module generation.
See https://bugs.eclipse.org/bugs/show_bug.cgi?id=541748 for details.

You should be able to compile the TTCN-3 generated from your XSDs if you use the command line option '-o'.

Best regards,
Botond Baranyi

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799683 is a reply to message #1799677] Wed, 12 December 2018 10:44 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Olaf,

of course for that you need to build Titan for your platform from the latest source in github.

BR
Elemer

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799690 is a reply to message #1799677] Wed, 12 December 2018 12:28 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Thanks, Botond, Elemer!

The problem with the -o option is that, we at ETSI STF160, write TTCN code which all other TTCN compilers should understand, and it will not work if we import modules which are not generated in the standard way, i.e. TTCN standard part 9.
The ATS we deliver does not contain any converted TTCN modules but the original XSDs only.

But the important thing is that it works, and users using Titan have to 'touch' the ATS which is not quite legal when using the ATS for conformance testing.

I will invetigate whether the converted code could be manipulated so that the ATS is not 'touched'.

Regards and thanks,
Olaf@ETSI-STF160

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799736 is a reply to message #1799690] Thu, 13 December 2018 08:13 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Olaf,

the way Titan supports XSD to TTCN-3 conversion is aligned with the standard part 9:

-the standard allows for explicit conversion , that is to generate an intermediate TTCN-3 code (instead of implicit where you can import directly the XSD)
-the only difference the -o option makes is that instead of generating a TTCN-3 file per namespace , Titan will generate a single TTCN-3 file , which is not in contradiction with the standard, which states:

"The mapping shall start on a set of valid XSD schema-s and shall result in a set of valid TTCN-3 modules. "

That being said, you will have to edit you ATS as follows:

-replace all

import from aaa.xsd all;

with

import from filegeneratedfromxsds all; //ttcn-3 file


Best regards
Elemer

[Updated on: Thu, 13 December 2018 08:55]

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799757 is a reply to message #1799736] Thu, 13 December 2018 10:03 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
Hi Elemer and Botond,

I just checked the new xsd2ttcn, but it doesn't work yet.

Attached the modules which would be the official delivery.
The main module is MCX_XML_Templates.ttcn.
There you need to uncomment the import of XSD_Definitions and remove all following imports.

Regards,
Olaf@ETSI-STF160
  • Attachment: MCX-XML.tgz
    (Size: 16.75KB, Downloaded 27 times)

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799822 is a reply to message #1799757] Fri, 14 December 2018 09:05 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Olaf,

Botond committed a fix that should address this issue, could you please try it?



BR

Elemer

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799830 is a reply to message #1799822] Fri, 14 December 2018 11:17 Go to previous messageGo to next message
Olaf Bergengruen is currently offline Olaf BergengruenFriend
Messages: 59
Registered: November 2018
Member
YES, it works now with the titan.core sources of today!!
Thanks, Elemer, Botond!

Again, the only issue now is how to make our delivery such that all TTCN compilers supporting XSDs can compile the code.
We handle many ATSs with different set of XSDs.
Currently I would go for our ETSI-STF160 delivery as follows:

module MCX_XML_Templates {

  /* 
   * Workaround for Titan users
   *  - xsd2ttcn -o *.xsd
   *  - Uncomment the import of XSD_Definitions
   *  - Commentout all other imports
   */
   // import from XSD_Definitions all;
 
  import from http_www_w3_org_XML_1998_namespace language "XSD" all;
  import from urn_3gpp_ns_mcsecKMSInterface_1_0  language "XSD" all;
  import from http_www_w3_org_2001_04_xmlenc  language "XSD" all;
  import from urn_3gpp_ns_mcpttInfo_1_0 language "XSD" all;
  import from urn_3gpp_ns_mcpttGKTP_1_0 language "XSD" all;
  import from urn_ietf_params_xml_ns_resource_lists language "XSD" all;
  import from urn_3gpp_ns_mcpttLocationInfo_1_0  language "XSD" all;
  import from urn_3gpp_mcptt_mcpttUEConfig_1_0 language "XSD" all;
  import from urn_3gpp_mcptt_user_profile_1_0 language "XSD" all;
  import from urn_3gpp_ns_mcpttServiceConfig_1_0 language "XSD" all;



Regards,
Olaf@ETSI-STF160

Report message to a moderator

Re: xsd2ttcn / bug handling substitution group? [message #1799831 is a reply to message #1799830] Fri, 14 December 2018 12:10 Go to previous message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1063
Registered: January 2015
Senior Member
Hi Olaf,

I'd probably write a simple script (awk maybe ?) to convert between the Titan and non-Titan version and add it to the delivery.

Let me restate though that the form accepted by Titan is perfectly aligned with the standard.


Best regards
Elemer

Report message to a moderator

Previous Topic: XML namespace "..." is in the excluded namespace list.
Next Topic:Project management tool for TTCN-3 project
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Thu Jul 09 21:45:33 GMT 2020

Powered by FUDForum. Page generated in 0.02705 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly