gdt helps designers of gestures as follows. It allows a designer to explore attributes of their gestures that influence recognition so the designer can determine if their gesures will be difficult for the computer to disambiguate.
Feature-based recognizers categorize gestures using certain attributes of the gestures. These attributes are called features and may include such properties as total length of the gesture, total angle of the gesture, and size of bounding box of the gesture.
gdt is built with thirteen features built-in, although the two related to time are not generally used. gdt uses an implementation of Rubine's recognizer as described in his SIGGRAPH '91 paper, "Specifying Gestures by Example" and uses the features described in that paper:
During training, for each class the recognizer uses the feature vectors of the examples and computes a mean feature vector and covariance matrix (i.e., a table indicating how the features vary together and what their standard deviations are) for the class.
For a feature-based recognizer to work perfectly, the values of each feature will be normally distributed within a class and between classes the values of each feature will vary greatly. The next section describes how to use gdt to determine if this is the case.
Individual gestures or classes maybe selected by clicking on their icons. The Edit menu contains the common cut, copy, paste, and delete operations, which can be used on gestures or classes. You could use these operations for tasks such as removing mistakenly entered examples.
The File menu allows the user to save and load gesture sets and gesture classes. The File menu in the gesture set window saves and loads gesture sets whereas the File menu in the gesture class window saves and loads individual classes. gdt is indifferent to file extensions, but for the sake of other users you may wish to use ".gs" and ".gc" extensions to denote gesture set and gesture class files, respectively.
A potential problem that may arise in training feature-based recognizers is bad example gestures. You can use gdt to detect this by using "Set/Classification matrix" to classify the training examples. It will create a table that shows what percentage of examples from a class are classified into each category. The vertical axis is what class it really belongs to and the horizontal axis is how it is classified by the recognizer. If there are no extreme outlying examples, this table will simply have 100s along the diagonal. If there are outlying examples shown in the table, clicking on the table cell will bring up the appropriate category, with the offending example(s) highlighted. You can then delete (with "Edit/Delete") and enter a new example.
Another problem may be that two gesture classes are simply too similar to one another for the recognizer to disambiguate. gdt can compute the distances between classes, which is proportional to how different they are for recognition purposes. Use the "Set/Distance matrix" to see this table. It is read like distance tables on maps: cross index class A with class B to see the distance between classes. (The table is symmetric, but the entire table is shown for convenience.) There is a threshold slider on the right to gray out uninteresting inter-class distances (i.e., large distances).
If it happens that two classes are too close together, you can look at the individual features for the two classes by clicking on their cell in the table. The feature graph shows the values of each feature for each class. Using this graph, you can see which features are too similar and change one of the gestures to differentiate them.
The way the weights are assigned to features may also cause the recognizer to have difficulty classifying some gestures. This may happen if, for example, two classes (let's call them A and B) are distinguished from each other by their size while another class (C) in the same set has examples at two or more substantially different sizes. In order to classify C correctly, the features related to size will probably be given a low weight, which will make disambiguating A and B difficult. This can be discovered using the feature graph ("Set/Feature graph") and corrected by breaking C up into two or more separate classes (e.g., "C-big" and "C-little").
Important: For performance reasons, the classifier does not automatically retrain when the gesture set is changed. You must manually retrain by selecting "Set/Train" in the gesture set window.