Hi had the same problem a few weeks ago.
To solve this i added a MouseDoubleClick listener to the graph and get a handle to the underlying data object (you can get the node under the click location from the graph and the x,y values from the event).
spawn a shell (not a dialog) at your click location which contains your edit form. I used the factory pattern to create the different edit forms based on the underlying data object.
Three hints for the shell.
You have to map your relative to the display coordinates ( display.map() )
set the SWT.NO_TRIM flag in the style attribute
add an mouse listener which fakes the move event, cause your shell is not movable if you have the NO_TRIM flag set. There are some snippets out for custom shells, maybe you want something other than rectangular *g*
After edit update your model, dispose the shell and refresh your graphviewer, thats all
Btw there is an really nice animation library (trident) with which you can fade your shell in and out.
If you only need basic editing functionallity of existing nodes this solution works quite well.