Home » Modeling » EMF » [EMF Compare] Accounting for changes in XML schema
| [EMF Compare] Accounting for changes in XML schema [message #972519] |
Mon, 05 November 2012 17:40  |
 Stephen Slaboda
Messages: 8
Registered: November 2012 |
Junior Member |
|
|
Hi all,
I am trying to come up with a way to compare similar XML files that mostly have differences in the node values, but may also have some changes to the schema. For example, an enumeration value in the schema is changed. For ease of understanding, I've constructed a very simple example (my current XML files are 50K+ lines long):
File enum1.xsd:
<?xml version="1.0" encoding="utf-8"?>
<xsd:schema xmlns:xsd="http;//www,w3,org/2001/XMLSchema" elementFormDefault="qualified" attributeFormDefault="unqualified">
<xsd:element name="Test">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="TestEnum" type="MyEnum"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:simpleType name="MyEnum">
<xsd:restriction base="xsd:token">
<xsd:enumeration value="VAL1"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:schema>
File enum1.xml:
<?xml version="1.0" encoding="utf-8"?>
<Test xsi:noNamespaceSchemaLocation="enum1.xsd" xmlns:xsi="http;//www,w3,org/2001/XMLSchema-instance">
<TestEnum>VAL1</TestEnum>
</Test>
and equivalent enum2.xsd and enum2.xml with VAL1 replaced with VAL2 (and the schema location updated to enum2.xsd)
Note: You'll need to replace the commas in the website address with periods and the semicolon with a colon...since this is my first post, the forums deemed those code snippits as websites, which I'm not allowed to link to yet...
I then use the following code to convert these to dynamic Ecore models and perform a comparison on them:
XSDEcoreBuilder xsdEcoreBuilder = new XSDEcoreBuilder();
ResourceSet resourceSet = new ResourceSetImpl();
Collection eCorePackages = xsdEcoreBuilder.generate(URI.createFileURI("C:/temp/enum1.xsd"));
for (Iterator iter = eCorePackages.iterator(); iter.hasNext();) {
EPackage element = (EPackage) iter.next();
resourceSet.getPackageRegistry().put(element.getNsURI(), element);
}
resourceSet.getResourceFactoryRegistry().getExtensionToFactoryMap().put("xml", new GenericXMLResourceFactoryImpl());
HashMap options = new HashMap();
options.put(XMLResource.OPTION_EXTENDED_META_DATA, Boolean.TRUE);
Resource resource = resourceSet.createResource(URI.createFileURI("C:/temp/enum2.xml"));
resource.load(options);
resource = resourceSet.createResource(URI.createFileURI("C:/temp/enum1.xml"));
resource.load(options);
I get the following error:
org.eclipse.emf.ecore.resource.Resource$IOWrappedException: Value 'VAL1' is not legal. (file:/C:/temp/enum1.xml, 3, 29)
at org.eclipse.emf.ecore.xmi.impl.XMLLoadImpl.handleErrors(XMLLoadImpl.java:77)
at org.eclipse.emf.ecore.xmi.impl.XMLLoadImpl.load(XMLLoadImpl.java:185)
at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doLoad(XMLResourceImpl.java:240)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1505)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1284)
at gps.DynamicModelTest.execute(DynamicModelTest.java:56)
at org.eclipse.ui.internal.handlers.HandlerProxy.execute(HandlerProxy.java:290)
at org.eclipse.ui.internal.handlers.E4HandlerProxy.execute(E4HandlerProxy.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:56)
at org.eclipse.e4.core.internal.di.InjectorImpl.invokeUsingClass(InjectorImpl.java:229)
at org.eclipse.e4.core.internal.di.InjectorImpl.invoke(InjectorImpl.java:210)
at org.eclipse.e4.core.contexts.ContextInjectionFactory.invoke(ContextInjectionFactory.java:131)
at org.eclipse.e4.core.commands.internal.HandlerServiceImpl.executeHandler(HandlerServiceImpl.java:171)
at org.eclipse.e4.ui.workbench.renderers.swt.HandledContributionItem.executeItem(HandledContributionItem.java:814)
at org.eclipse.e4.ui.workbench.renderers.swt.HandledContributionItem.handleWidgetSelection(HandledContributionItem.java:707)
at org.eclipse.e4.ui.workbench.renderers.swt.HandledContributionItem.access$7(HandledContributionItem.java:691)
at org.eclipse.e4.ui.workbench.renderers.swt.HandledContributionItem$4.handleEvent(HandledContributionItem.java:630)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1053)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4169)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3758)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1029)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:923)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:86)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:588)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:543)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:124)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584)
at org.eclipse.equinox.launcher.Main.run(Main.java:1438)
at org.eclipse.equinox.launcher.Main.main(Main.java:1414)
So this leaves me with a few questions/problems:
1. Is it possible to skip these validation checks (I really just want to be able to see the differences between the files that aren't impacted by schema changes)
2. I have tried adding the enum2.xsd file to the resourceSet and it didn't work (same error).
3. I tried using separate resource sets for the two schema files and their xml files, but then the comparison using MatchResourceSet and DiffResourceSet simply infers that the entire top-level element has been removed and added, which isn't very useful in comparing the file contents.
My current means of doing this is to develop an ecore metamodel of one of the schemas myself and compare the schemas without the XML files, then convert the XML files into Ecore models and do a whole lot of calculations to determine the differences that were caused by schema change. Ultimately, I'd like to filter out the differences caused by schema changes while still allowing a dynamic model (so that I don't have to update the ecore models and redeploy to my users with every schema change).
I'm using the Modeling package of Eclipse Juno, with 1.3.2 EMF Compare (I've tried to install 2.0, but keep running into dependency problems)
Any help or thoughts would be greatly appreciated.
Thanks,
-Stephen
|
|
|
| Re: [EMF Compare] Accounting for changes in XML schema [message #976019 is a reply to message #972519] |
Thu, 08 November 2012 08:50  |
|
Stephen,
'EMF Compare' compares... EMF models  . This unfortunately means that the models need be valid for EMF before we can do anything with them.
However, if what your truly want is simply 'ignore' the differences on invalid parts of the model (because of schema changes), you might be able to do so through the load options EMF provides. The one you might be interested in is "RECORD_UNKNOWN_FEATURES" that will have the parser put the unkwown values in a separate map instead of throwing exceptions. Change your load options for the following :
HashMap options = new HashMap();
options.put(XMLResource.OPTION_EXTENDED_META_DATA, Boolean.TRUE);
options.put(XMLResource.OPTION_RECORD_UNKNOWN_FEATURE, Boolean.TRUE);
That being said, it will probably not be sufficient as the "feature" itself is valid, only its value is not. Ed Merks might be able to help more on that.
Finally, the load option you already have is suspicious. OPTION_EXTENDED_META_DATA is not something that your set to "true" or "false", it explicitely requires an instance of org.eclipse.emf.ecore.util.ExtendedMetaData. Since you give it an invalid value, your option is most likely ignored. This is probably the way to go though : you will probably need to provide an implementation of ExtendedMetaData that will allow you to "swap" the value of the enum at load time to a valid value. Again, Ed will probably know better whether this is even possible.
Laurent Goubet
Obeo
|
|
|
| Re: [EMF Compare] Accounting for changes in XML schema [message #976081 is a reply to message #976019] |
Thu, 08 November 2012 09:49 |
Ed Merks
Messages: 33140
Registered: July 2009 |
Senior Member |
|
|
Laurent,
It's a common assumption that when resourceSet.getResource(uri, true)
throws an exception you're dead in the water. But if you look at a
generated editor, you see it does this:
public void createModel()
{
URI resourceURI = EditUIUtil.getURI(getEditorInput());
Exception exception = null;
Resource resource = null;
try
{
// Load the resource through the editing domain.
//
resource =
editingDomain.getResourceSet().getResource(resourceURI, true);
}
catch (Exception e)
{
exception = e;
resource =
editingDomain.getResourceSet().getResource(resourceURI, false);
}
I.e., it tries a second time to find the resource in the resource set.
In general, the resource loads as much content as possible so while it
throws exceptions for things like bad values, it does in fact proceed
with processing all of the XML.
So for the example code shown, one should catch the exception but
continue to use the loaded resource because it will contain as much
content as could possibly be processed from the XML...
On 08/11/2012 9:50 AM, Laurent Goubet wrote:
> Stephen,
>
> 'EMF Compare' compares... EMF models :p. This unfortunately means that
> the models need be valid for EMF before we can do anything with them.
>
> However, if what your truly want is simply 'ignore' the differences on
> invalid parts of the model (because of schema changes), you might be
> able to do so through the load options EMF provides. The one you might
> be interested in is "RECORD_UNKNOWN_FEATURES" that will have the
> parser put the unkwown values in a separate map instead of throwing
> exceptions. Change your load options for the following :
>
>
> HashMap options = new HashMap();
> options.put(XMLResource.OPTION_EXTENDED_META_DATA, Boolean.TRUE);
> options.put(XMLResource.OPTION_RECORD_UNKNOWN_FEATURE, Boolean.TRUE);
>
>
> That being said, it will probably not be sufficient as the "feature"
> itself is valid, only its value is not. Ed Merks might be able to help
> more on that.
>
> Finally, the load option you already have is suspicious.
> OPTION_EXTENDED_META_DATA is not something that your set to "true" or
> "false", it explicitely requires an instance of
> org.eclipse.emf.ecore.util.ExtendedMetaData. Since you give it an
> invalid value, your option is most likely ignored. This is probably
> the way to go though : you will probably need to provide an
> implementation of ExtendedMetaData that will allow you to "swap" the
> value of the enum at load time to a valid value. Again, Ed will
> probably know better whether this is even possible.
>
> Laurent Goubet
> Obeo
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
| Re: [EMF Compare] Accounting for changes in XML schema [message #976590 is a reply to message #972519] |
Thu, 08 November 2012 17:52 |
Stephen Slaboda
Messages: 8
Registered: November 2012 |
Junior Member |
|
|
Thanks Laurent and Ed, that definitely gave me some more things to look into. Ed's suggestion of catching the exception on the resource.load(options) call looks like it worked (based on looking at the resource in the debugger), but now I have added code to load the files into the Ecore model objects:
EObject model1 = null;
try
{
model1 = ModelUtils.load(new File("C:/temp/enum1.xml"), resourceSet);
}
catch(Exception e)
{
model1 = ModelUtils.load(new File("C:/temp/enum1.xml"), resourceSet);
}
and I'm getting a similar exception, except that it is coming from the ModelUtils.load now, but it is complaining that VAL1 is not valid for C:\temp\enum1.xml, even though enum1.xml is still pointing at enum1.xsd, which has VAL1 as its enumerated value. The same strategy of double-loading it does not work like it did for the Resource.load call. It almost seems to me that it is somehow finding the enum data type for enum2.xsd instead of enum1.xsd because it is the first "match" it finds for the data type name in the registry. Does that sound right?
I'm still investigating how to use ExtendedMetaData to combine the enumeration values - can you provide an example or website showing how to create custom ExtendedMetaData for this case, or do I just need to subclass BasicExtendedMetaData and add the annotation in? Would I be using the EPackage contents generated for the different XSD files to combine the enumerations into the ExtendedMetaData?
|
|
|
| Re: [EMF Compare] Accounting for changes in XML schema [message #977371 is a reply to message #976590] |
Fri, 09 November 2012 08:27 |
|
Stephen,
Ed's suggestion was more like this :
EObject model1 = null;
try
{
model1 = ModelUtils.load(new File("C:/temp/enum1.xml"), resourceSet);
}
catch(Exception e)
{
// swallow
}
That is to say : "catch the exception, but continue to use the loaded resource"  . If you try to 'load' again in the catch ... you'll end up with the same exception thrown again.
Laurent Goubet
Obeo
|
|
|
| Re: [EMF Compare] Accounting for changes in XML schema [message #977396 is a reply to message #977371] |
Fri, 09 November 2012 08:51 |
Ed Merks
Messages: 33140
Registered: July 2009 |
Senior Member |
|
|
Laurent,
Questions below.
On 09/11/2012 9:27 AM, Laurent Goubet wrote:
> Stephen,
>
> Ed's suggestion was more like this :
>
>
> EObject model1 = null;
> try
> {
> model1 = ModelUtils.load(new File("C:/temp/enum1.xml"), resourceSet);
What does this utility do? It's returning an EObject and it's
presumably important to populate model1's value. Given that a
resource's contents can contain multiple objects, I'm not sure how this
utility returns just one EObject.
> }
> catch(Exception e)
> {
> // swallow
So how does one get model1 now? The resource set contains the resource
now, so there must be some way to get the one object. Presumably
resourceSet.getResource(URI.createFileURI(""C:/temp/enum1.xml"), false)
will find that resource and then there's some way of getting the
appropriate object from that, assume the resource didn't fail because it
doesn't exist or some similar type of failure that leaves the resource
completely empty.
> }
>
>
> That is to say : "catch the exception, but continue to use the loaded
> resource" :). If you try to 'load' again in the catch ... you'll end
> up with the same exception thrown again.
Yes? Will it not find the resource in the resource set and then
continue finding the model object?
>
> Laurent Goubet
> Obeo
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
| Re: [EMF Compare] Accounting for changes in XML schema [message #977604 is a reply to message #977396] |
Fri, 09 November 2012 12:18 |
|
Stephen,
ModelUtil was an utility we developped at the beginning of EMF Compare back in 2007, and that is a little dangerous (we removed this from the source code of 2.0).
ModelUtil.load(URI, ResourceSet) will try and load the given URI as an EMF model in the given resource set, registering a default resource factory if needed. If it manages to load the resource it will return the first of the EObjects it contains. (And that's already two reasons that make it dangerous .)
From an EObject, you can 'go up' to the containing resource with 'eObject.eResource()'. This is also the level whence you can find the other "roots" of your resource. All in all, you code should look something like this :
EObject root1 = null;
try
{
root1 = ModelUtils.load(new File("C:/temp/enum1.xml"), resourceSet);
}
catch(Exception e)
{
// swallow
}
if (root1 == null)
{
// failed to load the resource as anything useable.
// Throw exception, return or whatever, but the rest of this method will fail in NullPointerException if used
}
Resource model1 = root1.eResource();
...
The "Resource" is what holds the actual model. In the example you gave here, it will contain a single root of type "Test", which itself will contain your "TestEnum".
This is also what you need to compare with the other side.
Laurent Goubet
Obeo
[edit : Whoops, the question were from Ed not Stephen... I'm leaving the answers as-is as they should help anyway :)/]
[Updated on: Fri, 09 November 2012 12:20] Report message to a moderator |
|
|
Goto Forum:
Current Time: Thu Apr 25 07:49:47 GMT 2024
Powered by FUDForum. Page generated in 0.03383 seconds |
] 
Search
Help
Register
Login
Home
