Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » M2T (model-to-text transformation) » [ACCELEO] Big integers(Problems handling big numbers)
[ACCELEO] Big integers [message #1791339] Thu, 28 June 2018 08:39 Go to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Hi,

I had some problems using integers bigger than 2^31.

Running the following code, it looks like Acceleo (OCL?) is able to handle 64bit integers,
using for them the acceleo type Integer, but the actual java class used to store them looks like to be Integer or Long (depending on the integer value).
The result is that some actions/commands may not work as expected, like the following ones:
* .oclIsKindOf(Integer) returns false for big (Long) values
* ->filter(Integer) removes big (Long) values
* ->sortedBy(v | -v) fails if there is at least one small (Integer) value and one big (Long) value, throwing a java.lang.ClassCastException (the result is an "OclInvalid" object)

----------------------------------
Here is an example

[query public powerOf2(x : Integer) : Integer =
if (x = 1) then
    2
else if (x > 1) then
    let new_x : Integer = x-1 in
    2 * powerOf2( new_x )
else
    -1
endif
endif
/]

[template public intTest(arg : OclAny) {
values  : Sequence(Integer) = Sequence(Integer) { 1 .. 70 };
results : Sequence(Integer) = values->collect(powerOf2());
}]
[file ('stdout', false, 'UTF-8')]

[for (x : Integer | values)]
2^[x/] = ( int ? [powerOf2(x).oclIsKindOf(Integer)/] ) -> [powerOf2(x)/] [if (
    (powerOf2(x) <> results->at(x))
)]different values[/if]

[/for]

Number of all computed values = [results->size()/] (including zeroes)
Number of all filtered values = [results->filter(Integer)->select(v : Integer | v<>0)->size()/]

Sort throws java.lang.ClassCastException -> [results->sortedBy(v | -v)/]

[/file]
[/template]



----------------------------------
Here are some lines of the output

...
2^30 = ( int ? true ) -> 1073741824
2^31 = ( int ? false ) -> 2147483648
...
2^62 = ( int ? false ) -> 4611686018427387904
2^63 = ( int ? false ) -> -9223372036854775808
2^64 = ( int ? true ) -> 0
...
Number of all computed values = 70 (including zeroes)
Number of all filtered values = 30

Sort throws java.lang.ClassCastException -> invalid



----------------------------------
Just a note: Acceleo has the type UnlimitedNatural, but, at the end, it still handles at most 64bit integers.
Some commands works, some other not.
* .oclIsKindOf(UnlimitedNatural) returns true for UnlimitedNatural, returns false for Integer, like literal values
* ->filter(UnlimitedNatural) keeps all UnlimitedNatural objects
* ->sortedBy(v | -v) fails if there is at least one small (<31bit) value and one big (>=31bit) value, throwing a java.lang.ClassCastException (the result is an "OclInvalid" object)
* literals depending on the context, the literals are Integer, so here and there acceleo complains that UnlimitedNatural is not an Integer (and viceversa)

----------------------------------
Any ideas/suggestion about how avoid this kind of problem, is really appreciated.


Best regards,
Nicola


Re: [ACCELEO] Big integers [message #1791483 is a reply to message #1791339] Sat, 30 June 2018 10:35 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

IIRC at the time Acceleo was developed the Ecore support for the Classic Eclipse OCL did not adequately support greater than 32 bit integers. The OCL support for 64 bit was improved; this was not quite trivial since 32 bit API compatibility had to be preserved. Parsers that processed 64 bit literals needed to use the new 64 bit API.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=378511 suggests that Acceleo decided not to track this Eclipse OCL evolution, but suggests that if you arrange to set the USE_LONG_INTEGERS option your problem may be solved.

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=344368 for the gory details of the upgrade.

(The newer Pivot-based Eclipse OCL uses a Number as its representation and so grows automatically as necessary from Integer to Long to BigInteger.)

UnlimitedNatural is a different issue. As a mathematician you might think that the UnlimitedNatural type supports a Natural number; no UnlimitedNatural is a special purpose type solely for UML collection multiplicities. It is better to think of UnlimitedNatural as an Enumeration of all the non-negative integers AND the extra unlimited/* value which is not a valid Integer or Real. Avoid UnlimitedNatural wherever possible.

Regards

Ed Willink
Re: [ACCELEO] Big integers [message #1791693 is a reply to message #1791483] Wed, 04 July 2018 09:15 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Hi,

thanks for the reply.

I tried to enable the USE_LONG_INTEGERS flag at workspace level, but it looks like the effect is related only to literals.

The following test can be compiled and runned only if USE_LONG_INTEGERS is set to true.

template public intTest2(arg : OclAny) {
v1 : Integer = 45;
v2 : Integer = 4503599627370496;
}]
[file ('stdout', false, 'UTF-8')]

-------
v1 is int  [v1.oclIsKindOf(Integer)/]
v1 is long [v1.oclIsKindOf(ELong)/]
-------
v2 is int  [v2.oclIsKindOf(Integer)/]
v2 is long [v2.oclIsKindOf(ELong)/]
-------

[/file]

[/template]


But still it uses Integer class for small values, and Long class for big values:

-------
v1 is int  true
v1 is long false
-------
v2 is int  false
v2 is long true
-------


So, the behavior of .oclIsKindOf(Integer) , ->filter(Integer) and ->sortedBy(v | -v) does not change.

If there isn't anything better, I'll try to replace them with queries and/or java code

Best regards,
Nicola


Re: [ACCELEO] Big integers [message #1791711 is a reply to message #1791693] Wed, 04 July 2018 12:32 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

Thanks for the clear example. https://bugs.eclipse.org/bugs/show_bug.cgi?id=536686 raised.

Pending a fix you may need to do (v1.oclIsKindOf(Integer) or v1.oclIsKindOf(ELong)).

Regards

Ed Willink

Re: [ACCELEO] Big integers [message #1791724 is a reply to message #1791711] Wed, 04 July 2018 13:28 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Hi,

I found another problem, probably it is also related to the behavior of oclIsKindOf.

[template public intTest(arg : OclAny) {
vList : Sequence(Integer) = Sequence(Integer) { 1, 4503599627370496, 3}; 
}]
[file ('stdout', false, 'UTF-8')]

// ------ OclAny
[for (it : OclAny | vList)]
[i/])  [it/]
[/for]

// ------ Integer
[for (it : Integer | vList)]
[i/])  [it/]
[/for]

[/file]

[/template]


The "Long" value is removed, if the "for" iterator is typed with Integer

// ------ OclAny
1)  1
2)  4503599627370496
3)  3

// ------ Integer
1)  1
3)  3


Here, the only workaround I can find is "using OclAny as iterator Type, and cast to Integer/Long", like

[for (it : OclAny | vList)]
[i/])  [it/] [1 + (
                if (it.oclIsKindOf(Integer)) then
                    it.oclAsType(Integer)
                else 
                if (it.oclIsKindOf(ELong)) then
                    it.oclAsType(ELong)
                else
                    null
                endif endif) /]
[/for]


(of course, I can use a query to reduce the code)


best regards
Nicola
Re: [ACCELEO] Big integers [message #1791731 is a reply to message #1791724] Wed, 04 July 2018 14:13 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi Nicola

The oclIsKindOf/oclIsTypeOf functionality was wrong. I've found an acceptable fix which is building now on: https://hudson.eclipse.org/ocl/job/ocl-branch-tests/111/

A regular N-build should be available in about an hour from https://www.eclipse.org/modeling/mdt/downloads/?project=ocl

This should solve the problem (unless you also use the debatable case of whether the unlimited * literal is an Integer.)

Regards

Ed Willink
Re: [ACCELEO] Big integers [message #1791733 is a reply to message #1791731] Wed, 04 July 2018 14:25 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Thanks a lot.

I'm not going to use unlimited literal.

Best regards,
Nicola
Re: [ACCELEO] Big integers [message #1791738 is a reply to message #1791733] Wed, 04 July 2018 15:10 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Hi Ed,

I tried the build 111 in jenkins (org.eclipse.ocl-6.5.0.N20180704-1407)

oclIsKindOf/oclIsTypeOf return true for both small and big integers

But

* filter still removes big integers
(Sequence(Integer) { 1, 4503599627370496, 3})->filter(integer)


* the Integer iterator skip big integers
[for (it : Integer | Sequence(Integer) { 1, 4503599627370496, 3})]



Filter can be easily replaced with something like
(Sequence(Integer) { 1, 4503599627370496, 3})->select(oclIsKindOf(Integer))



But, really I would like to avoid OclAny iterators.
Do you think that anything can be done about the "for iterator"?

Best regards,
Nicola
Re: [ACCELEO] Big integers [message #1791746 is a reply to message #1791738] Wed, 04 July 2018 16:39 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

oclIsKindOf is an OCL functionality that I have fixed.

filter is an Acceleo functionality; you'll have to raise an Acceleo Bugzilla and wait for a response; the regularity of comments from the Obeo committers may give you some clue as to how quickly that response may come.

Regards

Ed Willink
Re: [ACCELEO] Big integers [message #1791788 is a reply to message #1791746] Thu, 05 July 2018 09:48 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Hi Ed,

again, thanks for your help.

I'm going to report the problem in Acceleo Bugzilla.

If I can ask, do you think is better to write one single Bug report with everything or split in several reports, one for each place where the problem can be found?
* the filter command
* the "For" construct
* the "Let" construct

I found the "let" this morning:
It simply remove all let constructs that are "big integers" (like it does with invalid values).
I suppose that also this is an acceleo problem.

[template public intTest(arg : OclAny) {
v : Integer = 860287970189639680;
}]
[file ('stdout', false, 'UTF-8')]
v   [v/]
>> let
[let var : Integer = 860287970189639680]
	var [var/]
[/let]
[let var : Integer = v]
	v   [var/]
[/let]
<< let
[/file]
[/template]


v   860287970189639680
>> let
<< let


---------------------------------------------

Regarding the sortedBy

(Sequence(Integer) { 1, 4503599627370496, 3})->sortedBy(v | -v)


thorws a ClassCastException in an Ocl class: org.eclipse.ocl.EvaluationVisitorImpl.

java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
	at java.lang.Long.compareTo(Long.java:54)
	at org.eclipse.ocl.EvaluationVisitorImpl$2.compare(EvaluationVisitorImpl.java:2193)
	at java.util.TimSort.countRunAndMakeAscending(TimSort.java:356)
	at java.util.TimSort.sort(TimSort.java:234)
	at java.util.Arrays.sort(Arrays.java:1512)
	at java.util.ArrayList.sort(ArrayList.java:1454)
	at java.util.Collections.sort(Collections.java:175)
	at org.eclipse.ocl.EvaluationVisitorImpl.evaluateSortedByIterator(EvaluationVisitorImpl.java:2156)
    ....
    at org.eclipse.acceleo.engine.internal.evaluation ...
    ...


If I can try to blindly propose a fix, the method
private Comparator<Object> getComparatorForSortedBy(final Map<Object, Comparable<Object>> map, IteratorExp<C, PM> ie)
should return a special comparator for integers. Something like:

Comparator<Object> {
	
	@Override
	public int compare(Object o1, Object o2) {
		int result = 0;
		
		    /// o1   integer
		if (o1 instanceof Integer) {
			if (o2 instanceof Integer) {
				result = ((Integer) o1).compareTo((Integer) o2);
				
			} else if (o2 instanceof Long) {
				Long t1 = new Long( ((Integer) o1 ).longValue() );
				result = t1.compareTo((Long) o2);
				
			} else if (o2 instanceof BigInteger) {
				BigInteger t1 = new BigInteger( ((Integer) o1 ).toString() );
				result = t1.compareTo( (BigInteger) o2);
			} else {
				// not valid values
			}
		}
		else
			/// o1   long
		if (o1 instanceof Long) {
			if (o2 instanceof Integer) {
				Long t2 = new Long( ((Integer) o2 ).longValue() );
				result = ((Long) o1).compareTo(t2);

			} else if (o2 instanceof Long) {
				result = ((Long) o1).compareTo((Long) o2);

			} else if (o2 instanceof BigInteger) {
				BigInteger t1 = new BigInteger( ((Long) o1 ).toString() );
				result = t1.compareTo( (BigInteger) o2);
			} else {
				// not valid values
			}
		}
		else
			/// o1   BigInteger
		if (o1 instanceof BigInteger) {
			if (o2 instanceof Integer) {
				BigInteger t2 = new BigInteger( ((Integer) o1 ).toString() );
				result = ((BigInteger) o1).compareTo(t2);

			} else if (o2 instanceof Long) {
				BigInteger t2 = new BigInteger( ((Long) o1 ).toString() );
				result = ((BigInteger) o1).compareTo(t2);

			} else if (o2 instanceof BigInteger) {
				result = ((BigInteger) o1).compareTo((BigInteger) o2);
			} else {
				// not valid values
			}
		}

		return result;
	}
}


I do not have a deep knowledge of Ocl code, so I have no idea about the impact of this change, nor if it is actually a solution.


Best regards,
Nicola
Re: [ACCELEO] Big integers [message #1791790 is a reply to message #1791788] Thu, 05 July 2018 10:58 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Hi,

I was studing the ->filter(Integer)

In acceleo documentation, they declare that filter (OclType type) it is an easier way to write select(e | e.oclIsKindOf(type)).oclAsType(type).

So, I tried:
filter         [(Sequence(Integer) { 1, 888888888888, 2})->filter(Integer)/]
select         [(Sequence(Integer) { 1, 888888888888, 2})
					->select(e | e.oclIsKindOf(Integer))
               /]
select-collect [(Sequence(Integer) { 1, 888888888888, 2})
					->select(e | e.oclIsKindOf(Integer))
					->collect(e | e.oclAsType(Integer))
               /]


produces
filter         12
select         18888888888882
select-collect invalid


Both the "filter" and "select-collect" results are not what I expected.

Reducing the test to oclAsType
1 [1.oclIsKindOf(Integer)           /] [1.oclAsType(Integer)/]
I [888888888888.oclIsKindOf(Integer)/] [888888888888.oclAsType(Integer)/]
L [888888888888.oclIsKindOf(Integer)/] [888888888888.oclAsType(ELong)/]


there is something strange
1 true 1
I true invalid
L true 888888888888



Best regards
Nicola
Re: [ACCELEO] Big integers [message #1791801 is a reply to message #1791790] Thu, 05 July 2018 13:01 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

I suspect that the Acceleo problem is lack of support for the 64-bit OCL enhancement. So just one Bugzilla.

Your final example looks right to me. Is there a typo, or what are you trying to show?

Regards

Ed Willink
Re: [ACCELEO] Big integers [message #1791805 is a reply to message #1791801] Thu, 05 July 2018 13:37 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

I took a look at the filter() code in AcceleoLibraryOperationVisitor. Unfortunately it uses EClassifier.isInstance() directly, thereby bypassing the old (and new fixed) functionality in oclIsKindOf(). Overall isInstance() is used in many places so expect many varieties of the Long-is-not-an-Integer phenomenon.

Regards

Ed Willink


Re: [ACCELEO] Big integers [message #1791806 is a reply to message #1791801] Thu, 05 July 2018 13:45 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Hi,

Sorry if it was not so clear. I'll try to explain:
the idea behind the last example was:
I have an object (888888888888), that is checked to be a valid Integer by oclIsKindOf, but the oclAsType(Integer) return invalid.

One more example on this point
[( let a : Integer = 888888888888 in  /*  -> by construction, it is an ocl Integer */
       a.oclAsType(Integer)           /*  -> the result is an InvalidObject        */
        .oclIsInvalid())              /*  -> returns true                          */
/]


Here I have a variable declared as Integer, then I try to "cast" it to Integer
and the result is an Invalid value.

I do not know if it is an error, but it looks strange to me.

Please, note that running the same test, with a small literal values:
-> .oclAsType(Integer) results in the choosen value
-> so .oclIsInvalid() returns false

Best regards,

Nicola



Re: [ACCELEO] Big integers [message #1791812 is a reply to message #1791806] Thu, 05 July 2018 14:07 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
https://bugs.eclipse.org/bugs/show_bug.cgi?id=536724 raised.

Nicola
Re: [ACCELEO] Big integers [message #1791823 is a reply to message #1791812] Thu, 05 July 2018 15:34 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

"I have an object (888888888888), that is checked to be a valid Integer by oclIsKindOf, but the oclAsType(Integer) return invalid."

Yes the oclAsType implementation was also value- rather than type-driven ... Fix on the way.

Regards

Ed Willink
Re: [ACCELEO] Big integers [message #1791839 is a reply to message #1791823] Fri, 06 July 2018 06:37 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Thank you.

Did you have a chance to check the sortedBy ocl operation?

I mention it in the second half of https://www.eclipse.org/forums/index.php?t=msg&th=1093955&goto=1791788&#msg_1791788

Best regards,
Nicola
Re: [ACCELEO] Big integers [message #1791848 is a reply to message #1791839] Fri, 06 July 2018 08:24 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

Sorry, missed the sortedBy complication.

This demonstrates why the new Pivot OCL has a polymorphic IntegerValueImpl with derived BigIntegerValueImpl, IntIntegerValueImpl, LongIntegerValueImpl etc. It works for BigInteger, Long and Integer. The Classic Ecore OCL support is only for Integer and in many cases Long and occasionally BigInteger; the Long functionality was a difficult retrofit that has consistently failed to accommodate all possibilities. Unfortunately I do not regard the new Pivot OCL as a solid enough prototype for OCL 'next' and so it is understandable that Acceleo and QVTo have yet to migrate.

Looking to see whether "<" is also broken, it appears that both arguments are normalized by NumberUtil.higherPrecisionNumber(). A simpler sortedBy comparator can do the same. Yet another fix underway.

Regards

Ed Willink
Re: [ACCELEO] Big integers [message #1791964 is a reply to message #1791848] Mon, 09 July 2018 07:42 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Thank you,

Now I just have to wait for any response from Acceleo team (https://bugs.eclipse.org/bugs/show_bug.cgi?id=536724)

Best regards,
Nicola
Re: [ACCELEO] Big integers [message #1792360 is a reply to message #1791964] Mon, 16 July 2018 12:59 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

Sorry, five Classic OCL fixes that arose from your observations have now been reverted for today's M1(a) release. Unfortunately the Classic Eclipse OCL support for long integers is flaky and changing it just moves the code from one bad place to another with disruption for Acceleo and QVTo legacy users.

Regards

Ed Willink
Re: [ACCELEO] Big integers [message #1792398 is a reply to message #1792360] Tue, 17 July 2018 08:25 Go to previous messageGo to next message
Nicola Serreli is currently offline Nicola SerreliFriend
Messages: 24
Registered: July 2009
Junior Member
Hi Ed,

I'm really sad to hear that.

oclAsType(Integer)
oclIsKindOf(Integer)
->sortedBy(..)

[let value : Integer ...]
[for value : Integer ...]
->filter(Integer)

The current support of "Abstract" Integer in Ocl-Acceleo is quite full of pitfall,
and the code for workarounds may obfuscate the real code.

So, I was tring to avoid at all Ocl Integers and use myown emf data type, trying to enforce java.lang.Long
(or java.math.BigInteger) through the whole execution of Acceleo generator.

For example:
<eClassifiers xsi:type="ecore:EDataType" name="UINT32" instanceClassName="java.lang.Long"/>
<eClassifiers xsi:type="ecore:EDataType" name="UINT64" instanceClassName="java.math.BigInteger"/>


It looks good: using as expected type UINT32 (or UINT64) works correctly for most of operations/blocks.
 oclAsType(UINT32)
 oclIsKindOf(UINT32)
->sortedBy(v : UINT32 | v)
[let value : UINT32 ...] -> not working
[for value : UINT32 ...]
->filter(UINT32)


Until I try to make any operation on this kind of data.
As example, increasing by one an UINT32 convert it into an Ocl Integer, so I fall in the same problems as before.
(uint32_value    ).oclIsKindOf(UINT32) -> true
(uint32_value + 1).oclIsKindOf(UINT32) -> false


There is a way to ask Ocl to use mine implementation of operations (like + - * / ) in order to enforce the result to be of the right type?


Best regards,
Nicola
Re: [ACCELEO] Big integers [message #1792407 is a reply to message #1792398] Tue, 17 July 2018 09:30 Go to previous message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6111
Registered: July 2009
Senior Member
Hi

Bottom line 1: super-32 bit integers are not supported in Acceleo. Even if you contrive to get something working this year, it may break next year.

Bottom line 2: the super-32 bit support in Classic Eclipse OCL is very troublesome and too hard to resolve within the limits of API compatibility. Hence the Pivot OCL redesign that addresses many of the issues.

Regards

Ed Willink
Previous Topic:Self(Operation).setType(newType: Type)
Next Topic:[Acceleo] Maximum size of a module
Goto Forum:
  


Current Time: Sun Nov 18 08:22:05 GMT 2018

Powered by FUDForum. Page generated in 0.02964 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top