I'm not sure if this is the problem, but where have you placed your controls (e.g. Reporter smartfield)? Usually there is the form, within the form a MainBox, and within this MainBox there is another box (GroupBox) where you have to place your other controls (except OK & cancel buttons, which are placed directly in the MainBox).
Stephan is right: Placing a groupbox inside the mainbox is the recommended way to do this.
Here are some details to make it more clear: The margin is controlled by the property borderVisible. If set to true, there is one, otherwise not. But why is there no margin at the mainbox and no border at the inside groupbox? That's because of the property borderDecoration which is set to auto. Auto means the following: If it's the mainbox don't create any margin, if the groupbox has no label, don't paint any border.
I added a groupBox and now it looks fine, thanks for the tip. But this means that I have to change all my forms when switching from a SWT client to Swing client.
I don't want to see the GroupBox. This means for a Swing client (label visible = false) and for a SWT client (border visible = fase). If you mix these settings only one of the two looks nice, not both at the same time. So developing these two clients next to each other is difficult.
Its not a problem for me because I only need one of the two clients.
You're absolutely right. The SWT and the Swing UI treat the borders of the group boxes not in the same way. As you said people mostly don't use swing and swt ui for the same app, so it typically is no problem. But it might get one when using the swt and web ui, which is far more common. The web ui works in the same way as the swing ui.
So in my opinion the behaviour should be fixed for the swt ui to make it consistent.