Sketched Symbol Recognition using Zernike Moments

by Hse, H. and Newton, A. R.

Summary:

Hse begins his article by first drawing the attention to the increasing interest to the sketch-based interfaces and he says graphical symbol recognition is the aim of this paper. Symbol recognition in sketches is achieved by adopting three different approaches in the community: statistical, structural and rule-based. In his work, Hse chooses a statistical approach where Zernike moments are used for symbol representation. Zernike moments are rotation and reflection invariant, thus qualifying to be effective for symbol recognition. The author proposes a method where the evaluation of Zernike moment features is done with three classification methods: Support Vector Machines (SVM), Minimum Mean Distance (MMD), and Nearest Neighbor (NN).

Before going further with the descriptions of Zernike moments and evaluation of the features, a pre-processing is necessary due to the scale and translation invariability of Zernike moments. Another pre-processing includes the approximation and interpolation of the data points to ease the calculation of the moments. Hse, then explains the Zernike moments briefly and goes on with the choice of SVM classifier since other classifiers have pretty straight forward algorithms.

In giving experimental results, two methods have been proposed. First one suggests that the user can train the recognizer with his or her own data, namely the user dependent test, while the second one suggests using a pre-trained recognizer that adapts itself with the addition of user examples, namely the user independent test. In the user dependent test, classifiers are trained and tested on single user's dataset where a cross validation of K=10 folds has been used. The classifiers are then evaluated on the dataset of all other users. This experiment results in higher accuracy rates with the SVM classifier when data is interpolated and with Zernike moments of order 8. For higher orders SVM classifier's accuracy is increases slightly, however there is a decrease in other classifiers.

In the user independent, classifiers are trained and tested with N-fold cross validation where N is the number of users. The test is done with a user's dataset where all other datasets are used for training. SVM classifier performs better in this experiment too, with data points interpolated and Zernike moments of order 8.

Discussion:

Hse's paper provides yet another way of recognizing low-level graphical shapes, this time using Zernike moments as features and evaluating them on three classifiers: SVMs, MMD and NN. The accuracy rates that have been achieved are impressing when taking the method's capability of recognizing shapes that is independent on stroke order, direction, scale, translation, reflection and rotation.

Apart from its valuable outputs, the paper does not seem to consider the next steps for recognizing sketches as a whole. The beautification process that is mentioned in the paper puts doubts on the handling of filled-in and over traced shapes, which are not mentioned in the paper at all. This may be due to the fact that the paper is for the recognition of a limited set of symbols that need not be emphasized when drawn (UML diagrams, PowerPoint slides, etc.).

Citation:

Hse, H. and Newton, A. R. Sketched symbol recognition using Zernike moments. In ICPR ’04: Proceedings of the Pattern Recognition, 17th International Conference on (ICPR’04) Volume 1, pages 367–370, Washington, DC, USA, 2004.