Sketch Based Interfaces: Early Processing for Sketch Understanding

by Sezgin, T. M., Stahovich, T. and Davis, R.

Summary:

Sezgin et al. describe the motivation for the paper as enabling users to interact with computers in a more natural way, thus, freehand sketching, rather than using a gesture-based system. To do this, they propose an implemented system that converts pixels into simple geometric shapes such as lines, rectangles, curves and their combinations. The authors make us of the temporal information of the strokes that is inherent in on-line sketching. It is assumed that the user will use single strokes for drawing.

Their proposed system has three stages: approximation, beautification and simple recognition. During approximation, Sezgin deals with the conversion by handling vertices in line segments of the stroke and curved segments separately. They work on the intuition that while drawing, people tend to slow down at corners.

Therefore they take speed data and stroke direction as their input for vertex detection. In order to cast out noise, the authors use average based filtering and look for the minima of speed data above a certain threshold while the curvature is already low and vice versa. The thresholds are empirically determined as the means of each data set. Since both of these information sources give rarely good results when used stand alone, the authors suggest developing a hybrid algorithm that first takes the vertex candidates from both sources and deciding on the best fits while taking into account the system's certainty. Initial fit is achieved by taking the intersection of the candidate sets from both sources. Then at each cycle two new fits are created, one with best curvature candidate and one with best speed candidate and with respect to their least squares error , the new fit is chosen. The upper bound for the error term is also set beforehand.

Handling curve segments is done after finding the vertices. Sezgin draws the attention to the fact that the ratio of the Euclidean distance between two vertices and the actual arc length between them is likely to be 1 when there is a line segment and lower when there is a curve. After determining these ratios the approximation of the curves are done with Bezier curves.

After finding the vertices and approximating the line and curve segments beautification follows and simple recognition is achieved by examining simple properties of geometric objects that can be drawn with lines and curves.

Sezgin identifies the future work as including scale space theory to be able to recognize sketches with features that have different scales.

Discussion:

Sezgin's implemented system is well defined in terms of what is does achieve but, at the same time it seems to be avoiding most of the aspects that we consider inherent in freehand sketching and the feeling of naturalness. The idea for creating a hybrid fit with the information taken from both the curvature and speed data, and using the intuition that people tend to slow down when drawing corners does sound excellent in the way that it is shown in the paper. However, when people draw sketches they sometimes do it because they want to design something that they do not know how yet. So, they slow down, and maybe stop, to gather their thoughts about how the sketch should continue. Since this would decrease the threshold for speed data, more candidate vertices would needed to be evaluated and possible a number of them will be included in the hybrid fit. Another point is that, in the article, the determining of the error term is not discussed thoroughly as this might also be affected with this intuition.

The mentioning of handling over traces at the end of the article does also seem to be unnecessary since the authors only considered to work on over tracing just for the beautification process, as can be seen from the examples provided. In natural freehand sketching people use over tracing to not only as touch-up strokes to make the understanding of it clear but also to put emphasis on that particular object.

Last but not the least; the system does not seem to give the user the freedom to create objects and understand them with multi-strokes which is very common in common while sketching with pen and paper.

Citation:

Sezgin, T. M., Stahovich, T., and Davis, R. 2001. Sketch based interfaces: early processing for sketch understanding. In Proceedings of the 2001 Workshop on Perceptive User interfaces (Orlando, Florida, November 15 - 16, 2001). PUI '01, vol. 15. ACM Press, New York, NY, 1-8.