Sensor fusion (AR markers) for multi-touch interaction

• Main Author: Hanafiah Yusof.
• Other Authors: Louis-Philippe Demers
• Format: Final Year Project
• Language: English
• Published: 2009
• Subjects: DRNTU::Engineering::Computer science and engineering::Computer applications::Computers in other systems
• Online Access: http://hdl.handle.net/10356/17032
• Institution: Nanyang Technological University

The purpose of this project is to incorporate the use of Augmented Reality (AR) markers into an existing vision based multi-touch system that is capable of sensing multiple fingers. The scope of this project covers the development of a multi-touch table that is suited for the implementation of an application that uses AR markers as well as multiple fingers for interaction. The development of the multi-touch table is divided into hardware and software components. The hardware component of this project includes the design and construction of a 1st generation table for testing and learning purposes as well as a 2nd generation table meant for actual usage. At the software level, various existing computer vision toolkits are tested, utilized and combined to sense the presence of AR markers and to interpret the data into useful information. The hardware and software design methodology of the multi-touch table is presented in the report. The multi-touch table is designed to provide maximum functionality so as to extend its usage to future applications. The design and development process will be discussed in the report. This project is successfully implementing a vision based multi-touch table capable of sensing both fingers and AR markers using a combination of the Frustrated Total Internal Reflection and Diffused Illumination technologies. This method of combination does not currently exist. A virtual musical application, named DeeJay Touch, which incorporates the use of AR markers and multiple fingers for interaction is developed as a companion application.