Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » QVT-OML » QVTo standalone: EMF package registries w/o maven/testng?
QVTo standalone: EMF package registries w/o maven/testng? [message #1721591] Thu, 28 January 2016 14:06 Go to next message
Eclipse UserFriend
Dear all,

I have a small Maven/Tycho setup to create standalone QVTo apps (based
on TransformationExecutor & friends):

https://github.com/mrcalvin/qvto-cli

I was reported the following issue [*]:

When running a testng test suite, to challenge the QVTo app, the QVTo
compiler cannot resolve dynamically registered metamodels/ EPackages
anymore, as in:

https://github.com/mrcalvin/qvto-cli/blob/master/qvto-app/src/main/java/at/ac/wu/nm/qvto/App.java#L78

When called as-is (java -jar ...) the app and transformation (incl. MM
and M files specified at the command line) work as expected,

I have the suspicion that the maven/testng (surefire) infrastructure
have unwanted side effects, such as reset or emptying the EMF package
registries (tested global and local ones).

Is there prior experience out there? Any hints would be appreciated!

Stefan

[*] https://github.com/mrcalvin/qvto-cli/issues/1
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1721594 is a reply to message #1721591] Thu, 28 January 2016 14:40 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 5625
Registered: July 2009
Senior Member
Hi

NEVER EVER reset the global registries. They only initialize once
through Java class loading which only occurs once.

org.eclipse.xtext.junit4.GlobalRegistries has some helpful facilities
for rewinding the registries.

Regards

Ed Willink

On 28/01/2016 14:06, Stefan Sobernig wrote:
> Dear all,
>
> I have a small Maven/Tycho setup to create standalone QVTo apps (based
> on TransformationExecutor & friends):
>
> https://github.com/mrcalvin/qvto-cli
>
> I was reported the following issue [*]:
>
> When running a testng test suite, to challenge the QVTo app, the QVTo
> compiler cannot resolve dynamically registered metamodels/ EPackages
> anymore, as in:
>
> https://github.com/mrcalvin/qvto-cli/blob/master/qvto-app/src/main/java/at/ac/wu/nm/qvto/App.java#L78
>
>
> When called as-is (java -jar ...) the app and transformation (incl. MM
> and M files specified at the command line) work as expected,
>
> I have the suspicion that the maven/testng (surefire) infrastructure
> have unwanted side effects, such as reset or emptying the EMF package
> registries (tested global and local ones).
>
> Is there prior experience out there? Any hints would be appreciated!
>
> Stefan
>
> [*] https://github.com/mrcalvin/qvto-cli/issues/1
>
>
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1721609 is a reply to message #1721594] Thu, 28 January 2016 15:32 Go to previous messageGo to next message
Eclipse UserFriend
Hi Ed,

Thx for the swift reply!

> NEVER EVER reset the global registries.

:) I should clarify. The app per se does not do any harm to the global
registries.

My suspicion is that the test harness (maven/testng) does, but bare of
deep testng knowledge, this remains a suspicion.

> They only initialize once
>> through Java class loading which only occurs once.

My suspicion is nourished by the fact, that the maven/testng integration
(surefire) plays different tricks on class loading. See
http://maven.apache.org/surefire/maven-surefire-plugin/examples/class-loading.html

The QVT standalone bundle/app reuses and preserves the Eclipse's
jar-in-jar. I played around with the various surefire parameters
(forkCount, ...) but there was no change in misbehavior.

>
> org.eclipse.xtext.junit4.GlobalRegistries has some helpful facilities
> for rewinding the registries.
>

Likely I need to watch out for such additions of other EMF aware test
harnesses, to learn more. Mmmmh.

Stefan
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1721617 is a reply to message #1721609] Thu, 28 January 2016 16:06 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 5625
Registered: July 2009
Senior Member
Hi

Clever class loaders .... nightmare.

It is probably worth instrumenting EMF to log the thread, class, uri,
object hash code of every ResourceSet, Resource, EPackage that is
created. Any duplicates offers excellent opportunities for metamodel
schizophrenia.

The org.eclipse.xtext.junit4.GlobalRegistries can be used to create a
Memento so that at the end of a test session you can compare the final
state of the global registries with the initial warmed up state and so
see who is registering what.

Regards

Ed Willink


On 28/01/2016 15:32, Stefan Sobernig wrote:
> Hi Ed,
>
> Thx for the swift reply!
>
>> NEVER EVER reset the global registries.
>
> :) I should clarify. The app per se does not do any harm to the global
> registries.
>
> My suspicion is that the test harness (maven/testng) does, but bare of
> deep testng knowledge, this remains a suspicion.
>
>> They only initialize once
>>> through Java class loading which only occurs once.
>
> My suspicion is nourished by the fact, that the maven/testng
> integration (surefire) plays different tricks on class loading. See
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/class-loading.html
>
> The QVT standalone bundle/app reuses and preserves the Eclipse's
> jar-in-jar. I played around with the various surefire parameters
> (forkCount, ...) but there was no change in misbehavior.
>
>>
>> org.eclipse.xtext.junit4.GlobalRegistries has some helpful facilities
>> for rewinding the registries.
>>
>
> Likely I need to watch out for such additions of other EMF aware test
> harnesses, to learn more. Mmmmh.
>
> Stefan
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1721833 is a reply to message #1721617] Sun, 31 January 2016 20:53 Go to previous messageGo to next message
Francois Le Fevre is currently offline Francois Le FevreFriend
Messages: 56
Registered: October 2014
Location: Paris Saclay, France
Member
Dear Ed,
I am the one that creates the ticket.

I have discover the project of Stefan, and I wanted to use it, improve it. So I wanted to add a unit test or an integration test.

What is strange is that the generated jar is fully functionnal!
If you clone the fork github reposiotry [1]
You should be able to make
cd qvt-bundle
mvn clean install
cd qvto-app
mvn clean install

Here the unit test will raise, then the integration test. They will complain on the fact that qvto is no able to resolve the metamodel uri.

But if you go to the target folder
cd ./qvto-app/target

java -jar qvto-app-0.1-SNAPSHOT.jar -xform ../src/test/resources/qvto4abcmodel/transforms/transformationABC.qvto -src ../src/test/resources/qvto4abcmodel/modelSource/MMSource.ecore ../src/test/resources/qvto4abcmodel/modelSource/MSource.xmi -tgt ../src/test/resources/qvto4abcmodel/modelSource/MMSource.ecore ../src/test/resources/qvto4abcmodel/modelTarget/my.xmi

and the out.xml has been well transformed !

more out.uml
<?xml version="1.0" encoding="UTF-8"?>
<mmsource:root xmi:version="20131001" xmlns:xmi="http://www.omg.org/spec/XMI/20131001" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mmsource="http://www.mmsource.fr" xsi:schemaLocation="http://www
.mmsource.fr file:/home/flefevre/gitHubs/qvto-cli/qvto-app/target/../src/test/resources/qvto4abcmodel/modelSource/MMSource.ecore" xmi:id="_eKgaUchbEeWavJlXm9pMtQ">
<element xmi:type="mmsource:B" xmi:id="_eKgaUshbEeWavJlXm9pMtQ" Id="10" b="Quentin work at FraBen Company"/>
<element xmi:type="mmsource:B" xmi:id="_eKgaU8hbEeWavJlXm9pMtQ" b="Bob"/>
<element xmi:type="mmsource:C" xmi:id="_eKgaVMhbEeWavJlXm9pMtQ" Id="15" c="Anthony"/>
</mmsource:root>

So I do not understand why we can not have the same behavior by running the test inside eclipse testng or in maven...

That's strange.

Francois

[1]: https://github.com/flefevre/qvto-cli
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1721864 is a reply to message #1721833] Mon, 01 February 2016 06:53 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 5625
Registered: July 2009
Senior Member
Hi

Different behaviour in different contexts is not that strange at all,
particularly with different paths.

You at least have two 'identical' versions to compare. You need to
instrument to see where a significant difference arises. I suggest
logging all ResourceImpl construction.

Regards

Ed Willink


On 31/01/2016 20:53, Francois Le Fevre wrote:
> Dear Ed,
> I am the one that creates the ticket.
>
> I have discover the project of Stefan, and I wanted to use it, improve
> it. So I wanted to add a unit test or an integration test.
>
> What is strange is that the generated jar is fully functionnal!
> If you clone the fork github reposiotry [1]
> You should be able to make
> cd qvt-bundle
> mvn clean install
> cd qvto-app
> mvn clean install
>
> Here the unit test will raise, then the integration test. They will
> complain on the fact that qvto is no able to resolve the metamodel uri.
>
> But if you go to the target folder
> cd ./qvto-app/target
>
> java -jar qvto-app-0.1-SNAPSHOT.jar -xform
> ../src/test/resources/qvto4abcmodel/transforms/transformationABC.qvto
> -src ../src/test/resources/qvto4abcmodel/modelSource/MMSource.ecore
> ../src/test/resources/qvto4abcmodel/modelSource/MSource.xmi -tgt
> ../src/test/resources/qvto4abcmodel/modelSource/MMSource.ecore
> ../src/test/resources/qvto4abcmodel/modelTarget/my.xmi
>
> and the out.xml has been well transformed !
>
> more out.uml <?xml version="1.0" encoding="UTF-8"?>
> <mmsource:root xmi:version="20131001"
> xmlns:xmi="http://www.omg.org/spec/XMI/20131001"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:mmsource="http://www.mmsource.fr" xsi:schemaLocation="http://www
> mmsource.fr
> file:/home/flefevre/gitHubs/qvto-cli/qvto-app/target/../src/test/resources/qvto4abcmodel/modelSource/MMSource.ecore"
> xmi:id="_eKgaUchbEeWavJlXm9pMtQ">
> <element xmi:type="mmsource:B" xmi:id="_eKgaUshbEeWavJlXm9pMtQ"
> Id="10" b="Quentin work at FraBen Company"/>
> <element xmi:type="mmsource:B" xmi:id="_eKgaU8hbEeWavJlXm9pMtQ"
> b="Bob"/>
> <element xmi:type="mmsource:C" xmi:id="_eKgaVMhbEeWavJlXm9pMtQ"
> Id="15" c="Anthony"/>
> </mmsource:root>
>
> So I do not understand why we can not have the same behavior by
> running the test inside eclipse testng or in maven...
>
> That's strange.
>
> Francois
>
> [1]: https://github.com/flefevre/qvto-cli
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1722101 is a reply to message #1721864] Tue, 02 February 2016 18:12 Go to previous messageGo to next message
Eclipse UserFriend
Hi,

> Different behaviour in different contexts is not that strange at all,
> particularly with different paths.

I have narrowed it down to a MissingResourceException showing up in the
testng setup only.

In the course of setting up a metamodel registry, the testng-driven
setup fails on resolving a plugins.properties file appropriately for
org.eclipse.emf.common.CommonPlugin.

The failing line:

org.eclipse.emf.common.CommonPlugin.INSTANCE.getString("_UI_OK_diagnostic_0")

shouts with

The string resource '_UI_OK_diagnostic_0' could not be located

When stepping into the respective code region, it seems that another,
unrelated plugin.properties is resolved and loaded (one from a UML2
plugin?).

Time is up for today. Any hints are welcome.

Stefan
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1722103 is a reply to message #1722101] Tue, 02 February 2016 18:20 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 5625
Registered: July 2009
Senior Member
Hi

? classpath order

? explicit initialization. Occasionally EMF needs an explicit wakeup.
Check that EcorePackgeImpl.init is called before UMLPackageImpl.init

Regards

Ed Willink


On 02/02/2016 18:12, Stefan Sobernig wrote:
> Hi,
>
>> Different behaviour in different contexts is not that strange at all,
>> particularly with different paths.
> I have narrowed it down to a MissingResourceException showing up in the
> testng setup only.
>
> In the course of setting up a metamodel registry, the testng-driven
> setup fails on resolving a plugins.properties file appropriately for
> org.eclipse.emf.common.CommonPlugin.
>
> The failing line:
>
> org.eclipse.emf.common.CommonPlugin.INSTANCE.getString("_UI_OK_diagnostic_0")
>
> shouts with
>
> The string resource '_UI_OK_diagnostic_0' could not be located
>
> When stepping into the respective code region, it seems that another,
> unrelated plugin.properties is resolved and loaded (one from a UML2
> plugin?).
>
> Time is up for today. Any hints are welcome.
>
> Stefan
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1722768 is a reply to message #1722103] Mon, 08 February 2016 23:35 Go to previous messageGo to next message
Eclipse UserFriend
Hi,

> ? classpath order

The MissingResourceException being a mere symptom, certainly, it turned
out that maven classloader hierarchy [*] does not respect the
jars-in-jar structure of the qvto-bundle jar (it is a jar including all
plug-in jars and the org.eclipse.jdt.internal.jarinjarloader.*
infrastructure).

In a context outside maven (i.e.,
org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader as main class) all
works as expected, but maven's classloaders do not unfold the
jars-in-jar into the classpath and prep them for resource resolution.

For example, simply trying to run a basic QVTo standalone CLI app from
within maven using

mvn clean compile exec:java -Dexec.mainClass="at.ac.wu.nm.qvto.BasicTest"

yields the same MissingResourceException.

I need to find a way to customize the maven classloader hierarchy [*],
and marry it with necessary essence of
org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader. Alas, I am pretty
clueless about maven internals. I need to reach out to the maven user
list first.

Stefan

[*] http://maven.apache.org/guides/mini/guide-maven-classloading.html
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1722769 is a reply to message #1722768] Mon, 08 February 2016 23:37 Go to previous messageGo to next message
Eclipse UserFriend
> In a context outside maven (i.e.,
> org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader as main class) all
> works as expected, but maven's classloaders do not unfold the
> jars-in-jar into the classpath and prep them for resource resolution.

This implies that the observed behavior is not due to testng or maven's
testng plugins (surefire).

Stefan
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1722816 is a reply to message #1722768] Tue, 09 February 2016 11:36 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 5625
Registered: July 2009
Senior Member
Hi

This sounds very unlikely. IIRC the QVTo distribution structure is
rather boring; similar to many other Java projects. It should not
therefore cause problems that do not occur with other Eclipse projects
as well.

Regards

Ed Willink


On 08/02/2016 23:35, Stefan Sobernig wrote:
> Hi,
>
>> ? classpath order
> The MissingResourceException being a mere symptom, certainly, it turned
> out that maven classloader hierarchy [*] does not respect the
> jars-in-jar structure of the qvto-bundle jar (it is a jar including all
> plug-in jars and the org.eclipse.jdt.internal.jarinjarloader.*
> infrastructure).
>
> In a context outside maven (i.e.,
> org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader as main class) all
> works as expected, but maven's classloaders do not unfold the
> jars-in-jar into the classpath and prep them for resource resolution.
>
> For example, simply trying to run a basic QVTo standalone CLI app from
> within maven using
>
> mvn clean compile exec:java -Dexec.mainClass="at.ac.wu.nm.qvto.BasicTest"
>
> yields the same MissingResourceException.
>
> I need to find a way to customize the maven classloader hierarchy [*],
> and marry it with necessary essence of
> org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader. Alas, I am pretty
> clueless about maven internals. I need to reach out to the maven user
> list first.
>
> Stefan
>
> [*] http://maven.apache.org/guides/mini/guide-maven-classloading.html
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1722841 is a reply to message #1722816] Tue, 09 February 2016 14:39 Go to previous messageGo to next message
Eclipse UserFriend
Hi Ed,

> This sounds very unlikely. IIRC the QVTo distribution structure is
> rather boring;

This is not about QVTo distribution structure, it is the way the plugin
jars relevant for QVTo are wrapped into a single superjar:

I use Eclipse Tycho [1] to obtain and to repackage a number of Eclipse
plug-in jars into a Maven artifact [2]. This artifact
is effectively a "jars-in-jar":

superjar.jar
`- plugin1.jar
`- plugin2.jar
`- *.jar
`- org.eclipse.jdt.internal.jarinjarloader.*

When deployed, I set
org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader as the superjar's
main class. Then everything is fine. However, when used from within
maven (mvn compile, mvn java:exec, mvn test) the maven-internal
classpath is unaware of the jars in the superjar.

For example, (and therefore the MissingResourceException), maven
attempts to look up plugin.properties (actually at the root of
org.eclipse.emf.common*.jar) in superjar.jar, rather than
superjar.jar!org.eclipse.emf.common*.jar ...

Stefan
Re: QVTo standalone: EMF package registries w/o maven/testng? [message #1722842 is a reply to message #1722841] Tue, 09 February 2016 14:46 Go to previous message
Eclipse UserFriend
> I use Eclipse Tycho [1] to obtain and to repackage a number of Eclipse
> plug-in jars into a Maven artifact [2]. This artifact
> is effectively a "jars-in-jar":

The missing refs:

[1] https://eclipse.org/tycho/
[2] https://github.com/mrcalvin/qvto-cli/blob/master/qvt-bundle/qvto.product
Previous Topic:Keeping xmi:id for migration
Next Topic:[QVTo] How to load separated (externalized) profile applications in QVTo transformations ?
Goto Forum:
  


Current Time: Mon Feb 19 16:00:28 GMT 2018

Powered by FUDForum. Page generated in 0.02574 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software