Semantic matching for online handwritten-based graphical solutions

• Main Author: He, Lu.
• Other Authors: Hui Siu Cheung
• Format: Final Year Project
• Language: English
• Published: 2010
• Subjects: DRNTU::Engineering::Computer science and engineering::Information systems::Information storage and retrieval
• Online Access: http://hdl.handle.net/10356/39721
• Institution: Nanyang Technological University

The emergence of pen-based devices such PDAs and Tablets have greatly changed the way we interact with computers. As a natural form of user interface, freehand writing and sketching has recently drawn a lot of research attention. In the domain of mathematics, there has been a lot of research on Handwritten Mathematical Expression Recognition and Handwritten Mathematical Diagram Recognition. These researches greatly change the way we deal with mathematical documents. For example, users can write an expression on the handwriting pad and the Handwritten Mathematical Expression Recognition System will automatically converts the freehand writing to intended mathematical expressions in editable digital format. However, after conducting literature review, we found that there is little research dedicated to Mathematical Graph Recognition, which will be useful for matching and retrieving mathematical graphs (graphs of mathematical functions such as linear and quadratic functions). Also, the currently available mathematical graph editors have very limited functionality and don‟t support freehand sketch and other useful features that are expected by the users. Therefore, in this project, we first investigated various techniques for mathematical graph matching and retrieving. We then proposed a Feature-Based approach that effectively extracts both Structural, Semantic and Spatial features from the mathematical graphs so that a mathematical graph will be represented by its feature vectors within Vector Space Model (VSM). Various clustering techniques, including K-Means, Self-Organizing Map (SOM) and Agglomerative Hierarchical Clustering (AHC), were applied on mathematical graphs to improve the retrieval performance. Besides, we have developed a full-functional Mathematical Graph Editor that can handle all cases of complex mathematical graphs. A Mathematical Graph Retrieval System was also implemented to retrieve graphs based on the features we extracted during the Feature Extraction and Clustering phase. The performances of both training efficiency and retrieval accuracy for our Mathematical Graph Retrieval System are evaluated. The results show that our system can achieve up to 98% retrieval accuracy, which is a great achievement. At the end, possible future developments of the system are suggested. The improvements mainly focus on three areas: Feature Extraction/Clustering, Mathematical Graph Editor, and Mathematical Graph Retrieval System. They include adding more editing features to the Mathematical Graph Editor and allowing more searching criteria for the Mathematical Graph Retrieval System, which may include formulas for the graphs.