Sketch Documentation
The API is currently composed of three parts: a sketch Tool, a Recognizer and a sketch Bank. The figure below shows how the API integrates with GEF, through AbstractTool and GraphicalEditor. The upper part of the figure is an example of integration -- it shows a GEF editor (ShapesDiagramEditor) and three tools (for Square, Triangle and a connection). The integration is made through the SketchTool -- a tool that observes the user's clicks and drags and provides a visual feedback of what is being drawn on the editor. Once the sketch is finished, the tool also passes the points to the SketchManager, which in turn processes those points and updates the editor.
Sketch Recognition
Basically, the recognition is made using a chain of algorithms to take part in a decision. The SketchManager gives a list of points captured by the SketchTool to SketchElementRecognizer, and then periodically probes the recognizer for a result. The SketchElementRecognizer creates the chain using a set of algorithms. If the first algorithm of the chain is not 'sure' about what the sketch means, it can pass the decision to the next one on the chain, returning an element or null, if no element could be determined.
The first algorithm on the current chain was made using an approach based on Levenshtein's algorithm for string distance. It receives the user's sketch transformed in a set of points and places them on a grid, then each point is transformed to a number according to the next point's position. This algorithm is described in detail here.
For each element there is a set of stored words that describes its many forms, like drawn by the user. The recognizer compares each new sketch with the words of each element, calculating the distance between them. If the distance is smaller for a square than for a triangle, for example, then the element is probably a square.
The Chain of Responsibility pattern is applied to allow other algorithms to take part on the decision regarding what element is meant by a given sketch. Currently if a sketch is not recognized by the LevenshteinHandler, it is passed to ConnectionHandler, which analyses if the user is trying to connect two elements.