| AbstractDoubleColumn: rounding decimal values [message #909616] |
Fri, 07 September 2012 08:30  |
Eclipse User |
|
|
|
It seems that the method org.eclipse.scout.rt.client.ui.basic.table.columns.AbstractDoubleColumn.decorateCellInternal(Cell cell, ITableRow row) doesn't round decimal values correctly.
As an example: when you have a value 0.25 and maxFractionDigits set on 1, then you get 0.2. On my opinion it should be 0.3 (which would be mathematical correct).
The reason for this seems to be that java uses RoundingMode.HALF_EVEN as default (see also the javadoc for DecimalFormat and RoundingMode). This rounding mode "rounds towards the "nearest neighbor" unless both neighbors are equidistant, in which case, round towards the even neighbor".
The following little program visualizes this:
public class XXX {
public static void main(String[] args) {
DecimalFormat f = (DecimalFormat) NumberFormat.getNumberInstance();
f.setGroupingUsed(true);
f.setRoundingMode(RoundingMode.HALF_EVEN);
//f.setRoundingMode(RoundingMode.HALF_UP);
f.setMinimumFractionDigits(0);
f.setMaximumFractionDigits(0);
System.out.println(f.format(22.4));
System.out.println(f.format(22.5));
System.out.println(f.format(22.6));
System.out.println();
f.setMinimumFractionDigits(1);
f.setMaximumFractionDigits(1);
System.out.println(f.format(2.24));
System.out.println(f.format(2.25));
System.out.println(f.format(2.26));
System.out.println();
f.setMinimumFractionDigits(2);
f.setMaximumFractionDigits(2);
System.out.println(f.format(0.224));
System.out.println(f.format(0.225));
System.out.println(f.format(0.226));
System.out.println();
}
}
So, why can't we do it with mathematic correctness?
|
|
|
|
|
|
Powered by
FUDForum. Page generated in 0.04924 seconds
Home