| AbstractDoubleColumn: rounding decimal values [message #909616] |
Fri, 07 September 2012 08:30  |
 Birgit Körschgen
Messages: 1
Registered: September 2012 |
Junior Member |
|
|
It seems that the method org.eclipse.scout.rt.client.ui.basic.table.columns.AbstractDoubleColumn.decorateCellInternal(Cell cell, ITableRow row) doesn't round decimal values correctly.
As an example: when you have a value 0.25 and maxFractionDigits set on 1, then you get 0.2. On my opinion it should be 0.3 (which would be mathematical correct).
The reason for this seems to be that java uses RoundingMode.HALF_EVEN as default (see also the javadoc for DecimalFormat and RoundingMode). This rounding mode "rounds towards the "nearest neighbor" unless both neighbors are equidistant, in which case, round towards the even neighbor".
The following little program visualizes this:
public class XXX {
public static void main(String[] args) {
DecimalFormat f = (DecimalFormat) NumberFormat.getNumberInstance();
f.setGroupingUsed(true);
f.setRoundingMode(RoundingMode.HALF_EVEN);
//f.setRoundingMode(RoundingMode.HALF_UP);
f.setMinimumFractionDigits(0);
f.setMaximumFractionDigits(0);
System.out.println(f.format(22.4));
System.out.println(f.format(22.5));
System.out.println(f.format(22.6));
System.out.println();
f.setMinimumFractionDigits(1);
f.setMaximumFractionDigits(1);
System.out.println(f.format(2.24));
System.out.println(f.format(2.25));
System.out.println(f.format(2.26));
System.out.println();
f.setMinimumFractionDigits(2);
f.setMaximumFractionDigits(2);
System.out.println(f.format(0.224));
System.out.println(f.format(0.225));
System.out.println(f.format(0.226));
System.out.println();
}
}
So, why can't we do it with mathematic correctness?
|
|
|
] 
Search
Help
Register
Login
Home
