Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » XSD to ECore import help
XSD to ECore import help [message #421313] Mon, 04 August 2008 08:14 Go to next message
Andrew H is currently offline Andrew HFriend
Messages: 117
Registered: July 2009
Senior Member
I'm trying to import some industry standard XSD's (fpml.org) and have a
few questions. [Note: I am doing this with Ganymede (fully updated) on
Suse 10.1 with sun jdk 1.5.0_06]

1/ The import seems to take a very long time. If I pick up one of the
smaller leaf XSD's it works OK, but when I try one of the larger ones that
import many others (which in turn import others etc) then it is taking a
very long time. So far its been at least 2 hours and still going.
I'm running eclipse with -XX:+PrintGCDetails. Initially I did this to
check whether it was spending its time on full gc's but its not. It seems
to be creating a huge amount of short lived objects. It seems to be
running many short (eden) GC's every second. i.e. I get several of the
following per second
e.g [GC [DefNew: 5704K->7K(6400K), 0.0035280 secs] 88404K->82708K(90792K),
0.0036100 secs]
The step that is taking the time is the Load schema step. i.e. after I
point it at a schema.

Is this normal for large schemas or is something sick?

2/ One way around this would be if I could break the import into chunks.
i.e. import the leaf ones first, then work my way up. For this to work
each subsequent import would need to first look at the previous imports
and not reimport that schema.

e.g. a.xsd imports b.xsd

So I map b.xsd first, then map a.xsd telling it to use the mapping for
b.xsd that was done previously.

Is there a way to do this sort of thing?

Note I tried this but:
- its actually the load schema step that is taking the time and I don't
get asked about other models till the next step
- it doesn't work if everything is in the same ecore:package (see 3 below)

3/ The fpml specs use ecore tags (which is why I decided to evaluate EMF
in the first place). Unfortunately every single part of the schema is put
into ecore:package="org.fpml". This gets rather unwieldy to look at.

Is there any way I can break the resulting ecore model into separate
packages?

4/ Regardless of the answer to 3 is there a way I can at least break the
generated Java models into packages?
Re: XSD to ECore import help [message #421317 is a reply to message #421313] Mon, 04 August 2008 11:59 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 33113
Registered: July 2009
Senior Member
This is a multi-part message in MIME format.
--------------000509030208040204090101
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit

Andrew,

Comments below.

Andrew H wrote:
> I'm trying to import some industry standard XSD's (fpml.org) and have
> a few questions. [Note: I am doing this with Ganymede (fully updated)
> on Suse 10.1 with sun jdk 1.5.0_06]
>
> 1/ The import seems to take a very long time. If I pick up one of the
> smaller leaf XSD's it works OK, but when I try one of the larger ones
> that import many others (which in turn import others etc) then it is
> taking a very long time. So far its been at least 2 hours and still
> going. I'm running eclipse with -XX:+PrintGCDetails. Initially I did
> this to check whether it was spending its time on full gc's but its
> not. It seems to be creating a huge amount of short lived objects. It
> seems to be running many short (eden) GC's every second. i.e. I get
> several of the following per second
> e.g [GC [DefNew: 5704K->7K(6400K), 0.0035280 secs]
> 88404K->82708K(90792K), 0.0036100 secs]
> The step that is taking the time is the Load schema step. i.e. after I
> point it at a schema.
>
> Is this normal for large schemas or is something sick?
Some schemas are just grossly big.
>
> 2/ One way around this would be if I could break the import into
> chunks. i.e. import the leaf ones first, then work my way up. For this
> to work each subsequent import would need to first look at the
> previous imports and not reimport that schema.
The imports need to be resolved to hook up the dependencies, e.g,
between an element and its defined typd.
>
> e.g. a.xsd imports b.xsd
>
> So I map b.xsd first, then map a.xsd telling it to use the mapping for
> b.xsd that was done previously.
>
> Is there a way to do this sort of thing?
Yes, but it still needs to process the whole schema and also does the
full mapping.
>
> Note I tried this but:
> - its actually the load schema step that is taking the time and I
> don't get asked about other models till the next step
I'd really need to see this in action. I do seem to recall someone
working with this before:

http://www.google.com/cse?cx=017941334893793413703%3Asqfrdtd 112s&q=fpml+news.eclipse.tools.emf+news.eclipse.tools.em f&sa=Search
< http://www.google.com/cse?cx=017941334893793413703%3Asqfrdtd 112s&q=fpml+news.eclipse.tools.emf+news.eclipse.tools.em f&sa=Search>

> - it doesn't work if everything is in the same ecore:package (see 3
> below)
Each namespace maps to a separate package.
>
> 3/ The fpml specs use ecore tags (which is why I decided to evaluate
> EMF in the first place). Unfortunately every single part of the schema
> is put into ecore:package="org.fpml". This gets rather unwieldy to
> look at.
> Is there any way I can break the resulting ecore model into separate
> packages?
I seem to recall working with someone involved in the spec...
>
> 4/ Regardless of the answer to 3 is there a way I can at least break
> the generated Java models into packages?
I don't think that's the cause of your problem. You didn't check the
"Create XML Schema to Ecore Map" did you? You don't have problems with
firewall access or something like that? I'm pretty sure this has worked
well for others so there must be something unique to your situation.
How might I try to reproduce this?
>
>
>

--------------000509030208040204090101
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Andrew,<br>
<br>
Comments below.<br>
<br>
Andrew H wrote:
<blockquote
cite="mid:47a882879c7da422d84746d43abbe9da$1@www.eclipse.org"
type="cite">I'm trying to import some industry standard XSD's
(fpml.org) and have a few questions. [Note: I am doing this with
Ganymede (fully updated) on Suse 10.1 with sun jdk 1.5.0_06]
<br>
<br>
1/ The import seems to take a very long time. If I pick up one of the
smaller leaf XSD's it works OK, but when I try one of the larger ones
that import many others (which in turn import others etc) then it is
taking a very long time. So far its been at least 2 hours and still
going. I'm running eclipse with -XX:+PrintGCDetails. Initially I did
this to check whether it was spending its time on full gc's but its
not. It seems to be creating a huge amount of short lived objects. It
seems to be running many short (eden) GC's every second. i.e. I get
several of the following per second
<br>
e.g [GC [DefNew: 5704K-&gt;7K(6400K), 0.0035280 secs]
88404K-&gt;82708K(90792K), 0.0036100 secs]
<br>
The step that is taking the time is the Load schema step. i.e. after I
point it at a schema.
<br>
<br>
Is this normal for large schemas or is something sick?
<br>
</blockquote>
Some schemas are just grossly big.<br>
<blockquote
cite="mid:47a882879c7da422d84746d43abbe9da$1@www.eclipse.org"
type="cite"><br>
2/ One way around this would be if I could break the import into
chunks. i.e. import the leaf ones first, then work my way up. For this
to work each subsequent import would need to first look at the previous
imports and not reimport that schema.
<br>
</blockquote>
The imports need to be resolved to hook up the dependencies, e.g,
between an element and its defined typd.<br>
<blockquote
cite="mid:47a882879c7da422d84746d43abbe9da$1@www.eclipse.org"
type="cite"><br>
e.g. a.xsd imports b.xsd
<br>
<br>
So I map b.xsd first, then map a.xsd telling it to use the mapping for
b.xsd that was done previously.
<br>
<br>
Is there a way to do this sort of thing?
<br>
</blockquote>
Yes, but it still needs to process the whole schema and also does the
full mapping.<br>
<blockquote
cite="mid:47a882879c7da422d84746d43abbe9da$1@www.eclipse.org"
type="cite"><br>
Note I tried this but:
<br>
- its actually the load schema step that is taking the time and I don't
get asked about other models till the next step
<br>
</blockquote>
I'd really need to see this in action.


Ed Merks
Professional Support: https://www.macromodeling.com/
Re: XSD to ECore import help [message #421324 is a reply to message #421313] Mon, 04 August 2008 12:52 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: joerg.von.frantzius.artnology.com

Maybe they have the .ecore models ready for download somewhere, since
they have the ecore tags already in the XSD?

Andrew H wrote:
> I'm trying to import some industry standard XSD's (fpml.org) and have
> a few questions. [Note: I am doing this with Ganymede (fully updated)
> on Suse 10.1 with sun jdk 1.5.0_06]
>
> 1/ The import seems to take a very long time. If I pick up one of the
> smaller leaf XSD's it works OK, but when I try one of the larger ones
> that import many others (which in turn import others etc) then it is
> taking a very long time. So far its been at least 2 hours and still
> going. I'm running eclipse with -XX:+PrintGCDetails. Initially I did
> this to check whether it was spending its time on full gc's but its
> not. It seems to be creating a huge amount of short lived objects. It
> seems to be running many short (eden) GC's every second. i.e. I get
> several of the following per second
> e.g [GC [DefNew: 5704K->7K(6400K), 0.0035280 secs]
> 88404K->82708K(90792K), 0.0036100 secs]
> The step that is taking the time is the Load schema step. i.e. after I
> point it at a schema.
>
> Is this normal for large schemas or is something sick?
>
> 2/ One way around this would be if I could break the import into
> chunks. i.e. import the leaf ones first, then work my way up. For this
> to work each subsequent import would need to first look at the
> previous imports and not reimport that schema.
>
> e.g. a.xsd imports b.xsd
>
> So I map b.xsd first, then map a.xsd telling it to use the mapping for
> b.xsd that was done previously.
>
> Is there a way to do this sort of thing?
>
> Note I tried this but:
> - its actually the load schema step that is taking the time and I
> don't get asked about other models till the next step
> - it doesn't work if everything is in the same ecore:package (see 3
> below)
>
> 3/ The fpml specs use ecore tags (which is why I decided to evaluate
> EMF in the first place). Unfortunately every single part of the schema
> is put into ecore:package="org.fpml". This gets rather unwieldy to
> look at.
> Is there any way I can break the resulting ecore model into separate
> packages?
>
> 4/ Regardless of the answer to 3 is there a way I can at least break
> the generated Java models into packages?
>
>
>
Re: XSD to ECore import help [message #421326 is a reply to message #421317] Mon, 04 August 2008 13:03 Go to previous messageGo to next message
Andrew H is currently offline Andrew HFriend
Messages: 117
Registered: July 2009
Senior Member
Thanks Ed.

Further questions inline.

Ed Merks wrote:

> I'd really need to see this in action. I do seem to recall someone
> working with this before:

>
http://www.google.com/cse?cx=017941334893793413703%3Asqfrdtd 112s&q=fpml+news.eclipse.tools.emf+news.eclipse.tools.em f&sa=Search
>
< http://www.google.com/cse?cx=017941334893793413703%3Asqfrdtd 112s&q=fpml+news.eclipse.tools.emf+news.eclipse.tools.em f&sa=Search>

Thanks. Looks like at least 2 others (Matthew Rawlings & Dan Pollitt) have
tried EMF & FpML. They look like they had gotten further than me, but
unfortunately nothing in those threads helps.

I'm pretty sure it should be possible as the fpml schemas have made
extensive use of ecore attributes.

>> - it doesn't work if everything is in the same ecore:package (see 3
>> below)
> Each namespace maps to a separate package.

Right that would explain it. Since the whole schema is in a single
namespace it must therefore be in a single ecore package?

I take it there is no way to break it down into subpackages unless I
doctored the schema's to change namespaces (which would then mean I
couldn't process compliant documents)?


>> 4/ Regardless of the answer to 3 is there a way I can at least break
>> the generated Java models into packages?
> I don't think that's the cause of your problem. You didn't check the
> "Create XML Schema to Ecore Map" did you? You don't have problems with
> firewall access or something like that? I'm pretty sure this has worked
> well for others so there must be something unique to your situation.
> How might I try to reproduce this?

Yes I did check "Create XML Schema to Ecore Map". I take it that I
shouldn't have?
To be honest I didn't really know what it would do but thought it could do
no harm.

There could be firewall issues, but I doubt it cause I was able to import
some of the smaller schemas and I have all the schemas downloaded and in
the same directory.

Does it need to access the internet for the import?

I'll try without the "Create XML Schema to Ecore Map" tomorrow.
If I still have problems is there any diagnositcs I can try that will help
provide more info to you?
Unfortunately, the schemas are not publicly available so I can't post them
here.
Re: XSD to ECore import help [message #421327 is a reply to message #421326] Mon, 04 August 2008 13:23 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 33113
Registered: July 2009
Senior Member
Andrew,

Comments below.


Andrew H wrote:
> Thanks Ed.
> Further questions inline.
>
> Ed Merks wrote:
>
>> I'd really need to see this in action. I do seem to recall someone
>> working with this before:
>
>>
> http://www.google.com/cse?cx=017941334893793413703%3Asqfrdtd 112s&q=fpml+news.eclipse.tools.emf+news.eclipse.tools.em f&sa=Search
>
>>
> < http://www.google.com/cse?cx=017941334893793413703%3Asqfrdtd 112s&q=fpml+news.eclipse.tools.emf+news.eclipse.tools.em f&sa=Search>
>
>
> Thanks. Looks like at least 2 others (Matthew Rawlings & Dan Pollitt)
> have tried EMF & FpML. They look like they had gotten further than me,
> but unfortunately nothing in those threads helps.
Yes, I couldn't find links to the schemas to try it out.
>
> I'm pretty sure it should be possible as the fpml schemas have made
> extensive use of ecore attributes.
Exactly. I'm sure I've helped folks make it work.
>
>>> - it doesn't work if everything is in the same ecore:package (see 3
>>> below)
>> Each namespace maps to a separate package.
>
> Right that would explain it. Since the whole schema is in a single
> namespace it must therefore be in a single ecore package?
> I take it there is no way to break it down into subpackages unless I
> doctored the schema's to change namespaces (which would then mean I
> couldn't process compliant documents)?
Nope, there isn't a way.
>
>
>>> 4/ Regardless of the answer to 3 is there a way I can at least break
>>> the generated Java models into packages?
>> I don't think that's the cause of your problem. You didn't check the
>> "Create XML Schema to Ecore Map" did you? You don't have problems
>> with firewall access or something like that? I'm pretty sure this
>> has worked well for others so there must be something unique to your
>> situation. How might I try to reproduce this?
>
> Yes I did check "Create XML Schema to Ecore Map". I take it that I
> shouldn't have?
It has some very gross polynomial behavior so while interesting for
small models, it's worse than useless for big ones.
> To be honest I didn't really know what it would do but thought it
> could do no harm.
It kills performance, which is fine for a small or toy example, it's not
fine for an industrially large schema.
> There could be firewall issues, but I doubt it cause I was able to
> import some of the smaller schemas and I have all the schemas
> downloaded and in the same directory.
> Does it need to access the internet for the import?
To resolve imports whose location is based on an http URL, yes. The
import dialog itself lets you enter any URI, including a URL for
something on the internet.
> I'll try without the "Create XML Schema to Ecore Map" tomorrow.
That should do the trick.
> If I still have problems is there any diagnositcs I can try that will
> help provide more info to you?
I'm sure this is the issue.
> Unfortunately, the schemas are not publicly available so I can't post
> them here.
That explains the inability to find them.
>
>


Ed Merks
Professional Support: https://www.macromodeling.com/
Re: XSD to ECore import help [message #421337 is a reply to message #421327] Tue, 05 August 2008 00:20 Go to previous messageGo to next message
Andrew H is currently offline Andrew HFriend
Messages: 117
Registered: July 2009
Senior Member
Thanks Ed. This made a spectacular difference. Now I could import all the
schemas in a few minutes.

You may want to put a warning on that check box or otherwise rename it to
something like "Whatever you do don't check this checkbox".

So that leaves one question that got lost the first time.

Given that I now have an ecore model with 900 classes in one package, when
I generate the java model code I will get 2 packages with roughly 900
classes in each.

I take it that there are no controls in the mapping to Java that allows me
to split the java classes into subpackages?
Re: XSD to ECore import help [message #421338 is a reply to message #421337] Tue, 05 August 2008 00:47 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 33113
Registered: July 2009
Senior Member
Andrew,

Comments below.

Andrew H wrote:
> Thanks Ed. This made a spectacular difference. Now I could import all
> the schemas in a few minutes.
>
> You may want to put a warning on that check box or otherwise rename it
> to something like "Whatever you do don't check this checkbox".
Hehehehe.
>
> So that leaves one question that got lost the first time.
>
> Given that I now have an ecore model with 900 classes in one package,
> when I generate the java model code I will get 2 packages with roughly
> 900 classes in each.
That's quite a big model.
>
> I take it that there are no controls in the mapping to Java that
> allows me to split the java classes into subpackages?
Each namespace maps to an EPackage and all the classes for a given
EPackage are directed to just one Java package. It's really up to the
schema authors to provide a sensible decomposition in the first place.
For some reason folks see nothing wrong with creating namespaces so
overpopulated they surely exceed the human capacity to cope.


Ed Merks
Professional Support: https://www.macromodeling.com/
Re: XSD to ECore import help [message #421339 is a reply to message #421338] Tue, 05 August 2008 04:11 Go to previous messageGo to next message
Andrew H is currently offline Andrew HFriend
Messages: 117
Registered: July 2009
Senior Member
Fair enough. I suspect they do it cause they want to make it easy to
author documents. If there are lots of namespaces then the documents need
lots of namespace prefixes from memory.

Thanks for your help. I now have ecore diagram issues but am asking them
on the eclipse.technology.emft list. I assume thats' the right place?


Ed Merks wrote:
>>
>> I take it that there are no controls in the mapping to Java that
>> allows me to split the java classes into subpackages?
> Each namespace maps to an EPackage and all the classes for a given
> EPackage are directed to just one Java package. It's really up to the
> schema authors to provide a sensible decomposition in the first place.
> For some reason folks see nothing wrong with creating namespaces so
> overpopulated they surely exceed the human capacity to cope.
Re: XSD to ECore import help [message #421348 is a reply to message #421339] Tue, 05 August 2008 12:23 Go to previous message
Ed Merks is currently offline Ed MerksFriend
Messages: 33113
Registered: July 2009
Senior Member
Andrew,

Comments below.

Andrew H wrote:
> Fair enough. I suspect they do it cause they want to make it easy to
> author documents. If there are lots of namespaces then the documents
> need lots of namespace prefixes from memory.
The names of types will appear only if there need to be xsi:type
instances in the serialization. Is that common to see in the serialized
instances? In any case, it's a moot point since it's not going to change.
>
> Thanks for your help. I now have ecore diagram issues but am asking
> them on the eclipse.technology.emft list. I assume thats' the right
> place?
Yes, that's the right place.
>
>
> Ed Merks wrote:
>>>
>>> I take it that there are no controls in the mapping to Java that
>>> allows me to split the java classes into subpackages?
>> Each namespace maps to an EPackage and all the classes for a given
>> EPackage are directed to just one Java package. It's really up to
>> the schema authors to provide a sensible decomposition in the first
>> place. For some reason folks see nothing wrong with creating
>> namespaces so overpopulated they surely exceed the human capacity to
>> cope.
>
>


Ed Merks
Professional Support: https://www.macromodeling.com/
Previous Topic:Description of Eclipse XSD editor, .ecore, .genmodel and how to keep them all in sync?
Next Topic:Exception launching the Eclipse Platform:...
Goto Forum:
  


Current Time: Fri Mar 29 01:03:20 GMT 2024

Powered by FUDForum. Page generated in 0.04464 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top