Hi,

It appears that incremental transformation support was implemented in
the Mars release using a new Trace utility class and related IContext.
Together, these are supposed to allow a client to inject Traces
obtained from a previous transformation execution into a
TransformationExecutor for the next transformation, so that the latter
can find traceability of mappings from the previous transformation.

However, I am unable to get it to work. I create an EvaluationContext
and set a Trace object into the session data and then I see that
execution of a transformation puts all of its traces into that Trace
object for me. So far, so good.

Now, when I take this resulting Trace object and set it into the
EvaluationContext of the transformation, it is completely ignored, and
the output traces are just the traces for mappings evaluated by the
second transformation.

The problem seems to be that in the
QVTOperationalEvaluationEnv.RootInternal class, the constructor just
ignores the incoming Trace object in the IContext:

RootInternal(IContext context) {
assert context != null;
myContext = context;
myIsDeferredExecution = false;
myTraces = TraceFactory.eINSTANCE.createTrace();
}

See how 'myTraces' is just created new, instead of being taken from the
context.

Is there something else that I should be doing to make incremental
transformations based on previous traces work? Any example code that I
can be directed to look at?

Thanks,

Christian

import emf.tools;

modeltype ecore "strict" uses ecore('http://www.eclipse.org/emf/2002/Ecore');

transformation TestIncremental(in e : ecore, out ecore);

main() {
	e.objects()[EPackage]->map mapPackage();
}

mapping EPackage::mapPackage() : EPackage {
	if (name = null or self.name.startsWith("_")) {
		name := self.name + "_" + this.currentTimeMillis().toString();
	};
	eClassifiers := self.eClassifiers.map mapClassifier(name);
}

mapping EClassifier::mapClassifier(packName : String) : EClass {
	if (name = null or self.name.startsWith("_")) {
		name := self.name + "_" + this.currentTimeMillis().toString();
	}
}

<?xml version="1.0" encoding="UTF-8"?>
<ecore:EPackage xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:ecore="http://www.eclipse.org/emf/2002/Ecore" name="test" nsURI="http://test/1.0" nsPrefix="test">
  <eClassifiers xsi:type="ecore:EClass" name="_Class"/>
  <eClassifiers xsi:type="ecore:EDataType" name="DataType" instanceClassName="java.lang.String"/>
</ecore:EPackage>

	@Test
	public void testInremental() {
		URI uri = URI.createPlatformPluginURI("/qvto.incremental/transforms/TestIncremental.qvto", true); //$NON-NLS-1$
		TransformationExecutor executor = new TransformationExecutor(uri);
		
		ExecutionContextImpl context = new ExecutionContextImpl();
		context.getSessionData().setValue(QVTEvaluationOptions.INCREMENTAL_UPDATE_TRACE, Trace.createEmptyTrace());

		BlackboxRegistry blackboxRegistry = TransformationExecutor.BlackboxRegistry.INSTANCE;
		blackboxRegistry.registerModule(EmfToolsLibrary.class, "emf.tools", "tools", new String[] {"http://www.eclipse.org/emf/2002/Ecore"});
		
		URI inModelUri = URI.createPlatformPluginURI("/qvto.incremental/mm/in.ecore", true); //$NON-NLS-1$
		ModelExtent inExtent = new BasicModelExtent(new ResourceSetImpl().getResource(inModelUri, true).getContents());

		String clsName, dtName;
		try {
			ModelExtent outExtent = new BasicModelExtent();
			ExecutionDiagnostic result = executor.execute(context, inExtent, outExtent);
			assertTrue(result.getSeverity() == Diagnostic.OK);
			
			EPackage pack = (EPackage) outExtent.getContents().get(0);
			clsName = pack.getEClassifiers().get(0).getName();
			dtName = pack.getEClassifiers().get(1).getName();
		}
		finally {
			executor.cleanup();
		}

		String clsName2, dtName2;
		try {
			ModelExtent outExtent = new BasicModelExtent();
			ExecutionDiagnostic result = executor.execute(context, inExtent, outExtent);
			assertTrue(result.getSeverity() == Diagnostic.OK);

			EPackage pack = (EPackage) outExtent.getContents().get(0);
			clsName2 = pack.getEClassifiers().get(0).getName();
			dtName2 = pack.getEClassifiers().get(1).getName();
		}
		finally {
			executor.cleanup();
		}
		
		assertEquals(dtName, dtName2);
		assertFalse(clsName.equals(clsName2));
	}

		String clsName2, dtName2;
		try {
			TransformationExecutor executor2 = new TransformationExecutor(uri);
			ExecutionContextImpl context2 = new ExecutionContextImpl();
			context2.getSessionData().setValue(QVTEvaluationOptions.INCREMENTAL_UPDATE_TRACE, context.getSessionData().getValue(QVTEvaluationOptions.INCREMENTAL_UPDATE_TRACE));
			
			ModelExtent outExtent = new BasicModelExtent();
			ExecutionDiagnostic result = executor2.execute(context2, inExtent, outExtent);
			assertTrue(result.getSeverity() == Diagnostic.OK);

			EPackage pack = (EPackage) outExtent.getContents().get(0);
			clsName2 = pack.getEClassifiers().get(0).getName();
			dtName2 = pack.getEClassifiers().get(1).getName();
		}
		finally {
			executor.cleanup();
		}

As it turns out, having prototyped a simple change that satisfies my
use case, I'm not sure that this is a change worth pursuing in QVTo for
my purposes.

I spent some time looking through the existing JUnit tests to see how
to build a regression test for my changes, and of course I saw the
tests that cover the incremental transformation feature implemented in
Mars. And digging deeper to see why those tests pass it became
apparent that there are two quite different ways in which traces are
used by the QVTo TransformationExecutor. These two different ways are
illustrated in the evaluation-visitor's handling of mapping operations:

* a trace found in the executor's current output trace model completely
short-circuits the mapping execution, immediately returning a
found-in-trace result

* a trace found in the IContext trace model supplied by the application
from a previous transformation half short-circuits the mapping
execution. The mapping result is not re-created, instead re-used from
before, but the rest of the mapping execution is still done, including
(AFAICT) executing the mapping body on the pre-existing result, putting
the pre-existing result into the extent, and adding a trace to the
current output trace model so that subsequent evaluations will
short-circuit completely as above

My "fix" for my scenario basically initialized the transaction
executor's output trace (in the evaluation environment) as a copy of
the incoming trace model supplied by the application from the previous
execution. This caused a bunch of test cases to fail because now they
were short-circuiting in the wrong way, such that output extents were
not being populated that should have been. Clearly, that's not good.

So, what this means is that the trace model supplied by the application
can only implement incremental evaluation of mapping rules. It cannot
be queried by a mapping using the invresolve and invresolveIn
operations, which only look at the output trace model, not the input
traces. And I don't want to start fiddling with those operations,
because that would probably break even more stuff.

So, for my purposes, I've found another solution to my original problem
that doesn't require changing how the trace history is used by the
evaluation environment. If I ever find another problem that can't
reasonably be addressed another way, I may have to revisit this
trace-model business and will almost certainly need help from the QVTo
committer team.

Thanks,

Christian

Hi, Sergey,


In my case, I have a sequence of transformations applied to the same
extents that map different subsets of the inputs. I also have an
extension point that contributes additional transformations, especially
for handling of application- or domain-specific concerns such as UML
profiles. One of the needs that these plugged-in transformations have
is to be able to query which source elements generated a particular
target element using the invresolve or invresolveone operation. I had
expected that the trace model supplied to the transformation executor
would feed into these resolve operations as it does into the mapping
execution guard, but it doesn't and I can see that it would be
difficult to make it so.

public class AnnotatedJavaLibrary {

	@Operation (kind=Kind.HELPER, withExecutionContext=true)
	public void logToConsole(IContext context, String param) {
		context.getLog().log(param);
	}
}

	org.eclipse.m2m.qvt.oml.util.Trace updateTrace = context.getTrace();

So, as it turns out, I don't have a bugzilla to raise and I have worked
out a solution that operates more explicitly on the trace model without
trying to execute the transformations as incremental updates.

Cheers,

Christian

For reference, you can see what I'm doing, here:

https://git.eclipse.org/r/#/c/75597/

cW


On 2016-06-21 12:46:30 +0000, Christian W. Damus said:

> Hi, Sergey,
>
> That's basically exactly what I have done in Papyrus: I've added a
> different session-data object in the ExecutionContext that a suite of
> black-box queries can access via the IContext.
>
>
>
> I'm not sure that it is a generally useful mechanism, but if you think
> it would be, we could consider a contribution. In particular, I use a
> different key in the session-data so that the evaluation of mappings
> doesn't spend time looking through incremental-update traces that
> aren't meant to be used in that way. So, it really is a fairly
> application-specific design.
>
> cW
>
>

It appears that incremental transformation support was implemented in
the Mars release using a new Trace utility class and related IContext.
Together, these are supposed to allow a client to inject Traces
obtained from a previous transformation execution into a
TransformationExecutor for the next transformation, so that the latter
can find traceability of mappings from the previous transformation.

transformation Main(in input: ecore, out output : ecore) access T1,T2,T3;

modeltype ecore uses 'http://www.eclipse.org/emf/2002/Ecore';

main() {
       
       var intermediate1 = ecore.createEmptyModel();
       var step1 = new T1(input, intermediate1);
       step1.transform();
       
       var intermediate2 = ecore.createEmptyModel();
       var step2 = new T2(intermediate1, intermediate2);
       step2.transform();
       
       var step3 = new T3(intermediate2, output);
       step2.transform();

       ...
}