On real time stereo image processing and sonification methodologies applied towards SVETA

• Main Author: G. Balakrishnan
• Format: Thesis
• Language: English; English
• Published: 2006
• Subjects: TA1501-1820 Applied optics. Photonics
• Online Access: https://eprints.ums.edu.my/id/eprint/10214/1/24%20PAGES.pdf; https://eprints.ums.edu.my/id/eprint/10214/2/FULLTEXT.pdf; https://eprints.ums.edu.my/id/eprint/10214/
• Institution: Universiti Malaysia Sabah

The main objective of this thesis is to develop a Stereo Vision based Electronic Travel Aid (SVETA) for visually impaired people. A hardware system is developed, which includes stereo cameras and stereo earphones molded in a headgear and Compact Computing Device (CCD) duly placed in a designed pouch. The stereo cameras capture stereo images of the environment. The captured images are processed and mapped to stereo musical sound patterns to the earphones. Earlier efforts mentioned in the literature towards single camera based vision aids, did not provide 3D information about the environment. Obstacle detection and its distance information are the significant features for comfortable blind navigation. In this thesis, to incorporate the distance information, stereo imaging techniques are proposed. Stereo image processing requirement in this application is critical and therefore conventional stereo matching methodology cannot be applied directly in this problem. The stereo image processing developed in this thesis, is designed to highlight the object properties from background and compute its distance in real time. Three methods are proposed for stereo image processing namely isolated object matching method, fuzzy relation method and improved area correlation method, whose merits and demerits are compared. The resultant image consists of 3D information of the objects with background suppressed. The resultant image is sonified to produce musical stereo acoustic patterns. Two methods are used for image sonification. The frequency of the sound depends on the height or elevation of the image pixels in the image plane. The amplitude of the sound depends on the intensity value of the image pixels. The left half of the image is sonified to left earphone and the right half of the image is sonified to the right earphone. The pleasantness of the sound is improved using octave frequencies and certain voice commands are also incorporated to alert the blind user about any impending obstacles. Blind and non blind volunteers were trained with the developed SVETA system and they were tested to identify the environment using SVETA. They were able to identify the objects based on its distance and other characteristics with the help of the musical sound. They were also able to navigate in indoor and restricted outdoor environments