Place detection through analysis of descriptor sequences of 3D point clouds

• Main Author: Mihankhah, Ehsan
• Other Authors: Wang Dan Wei
• Format: Theses and Dissertations
• Language: English
• Published: 2018
• Subjects: DRNTU::Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision; DRNTU::Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics; DRNTU::Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: http://hdl.handle.net/10356/73719
• Institution: Nanyang Technological University

Place awareness is a critical component of safe decision-making in autonomous mobile robotics. The significance of place awareness is more appreciable in large-scale outdoor environment in which, safety measures, mission planning, and responsive intelligent behaviour depend on individual circumstances of each specific place. It is necessary for high-level mission planning and autonomous decision-making module to be aware of current place in order to invoke place-specific task objectives in a large-scale mission for long-term autonomy. In this regard, detection of transition between places (e.g. indoor to outdoor transition), or detection of specific scenarios (e.g. travelled from room to road via the main corridor), seriously assists the autonomous mission planning in complex environment. The importance of place awareness in self-localization and mapping, as a fundamental requirement of autonomous mobile robots, inspired the development of a group of place detection techniques from this perspective. Recognition of a revisited place is the crucial requirement for cancelation of accumulated mapping drifts. Besides, identification of similar places contributes to selection of proper frame for merging the maps generated by multiple mobile robots. Moreover, map segmentation to meaningful places provides a more semantically informative model of the environment. Place recognition is also attended by developers of image retrieval algorithms. In this second approach, given an image of a place and a pool of unordered images from different places, image retrieval techniques identify best matching images from the pool to the image of interest, or alternatively, some more advanced methodologies classify unordered images into groups of related places. This approach for place recognition branches form the general image recognition and image classification research, which is dedicated to recognition of landmarks and invariant image features under changing environmental situation and viewpoint changes. There is also a third approach to place recognition that relies on external equipment, such as GPS, or relies on processing the available place-specific signals, such as WiFi access point signal, which has demonstrated acceptable performance under the assumption of perfect coverage. In this research, place detection using on-board equipment is studied and the external equipment are used only as supplementary facilities. Thus, the third approach stays out of the attention of this thesis. Through the first approach, researchers face major challenges to maintain metrically accurate map of the environment in large-scale outdoor environment. On the other hand, second approach confronts environmental changes, such as illumination, which highly affect feature selection and feature matching. Moreover, in the first approach, place recognition, which is supposed to assist the mapping module, depends itself on the mapping and localization module. Therefore, once mapping and localization loses accuracy for any reason, place recognition risks total failure. Second approach, which is free from metrical accuracy debate, confronts memory storage demand in long run, since these techniques need to store the feature models of different scenes, in form of a multi-dimensional vector structure, in a codebook for succeeding matching purpose. Moreover, the second approach corresponds a place to one snapshot, which should necessarily capture enough landmark features. It is important to note that, in general, a place might or might not contain a specific landmark, and therefore, it might or might not be representable through a single snapshot. Therefore, it is necessary to look for an alternative solution, which does not rely on metrical accuracy of environment model, and is not sensitive to feature-related concerns. A new methodology for place detection is proposed in this thesis in order to cover the aforementioned concerns through a different approach to place detection problem. The proposed methodology identifies places as meaningful subsets of three dimensional environments, in which, every observation made at different locations of the place is consistently described by similar place descriptors. To compare one place to another, unlike the common approach of comparing two respective descriptor structures, two matrices consisted of descriptor similarities between different observations made from each place are compared. This novel comparison approach enhances the detection confidence. The descriptor used for place characterization is the list of eigenvalues of Laplace-Beltrami operator applied to the polygonised surface meshes. These meshes are generated from the three dimensional point cloud data captured by the sensory system. The mentioned descriptor is non-feature-based, and is applied to the entire captured input. Although the mentioned descriptor has been applied for object identification before, mainly because limited perturbation in studied object results to limited perturbation in the object descriptor, to the best of our knowledge, through this thesis, this is the first time it is applied for characterization of places. Since the motivation for this research is to enhance place awareness in a long-term robot autonomy in large scale mixed outdoor and indoor environment exploration missions, it is assumed that the robot motion is continuous. This assumption makes possible to study the sequence of places and the pattern of descriptor changes in order to detect a revisited place with high confidence. Moreover, study of sequences enables recognition of revisited scenarios of motion as well. “Pattern of descriptor changes” is another new concept proposed through this research. Moreover, sensitivity of place detection to field of view limitation is addressed in this thesis, and a sensory system is suggested to achieve the desired outcomes. The advantages of the proposed methodology include independence from metrical accuracy, elimination of feature selection and feature association challenges, storage efficiency, and applicability for large-scale outdoor scenarios in addition to insensitivity to environment changes. In GPS denied environment, these properties make the proposed methodology preferable for meaningful environment segmentation, identification of revisited places, and recognition of revisited scenarios, which form the fundamental requirements for identification of loop-closure frames in self-localization and mapping, identification of proper frames for merging multiple maps generated by multiple robots, autonomous start-up location estimation, and autonomous place-specific task assignment. Moreover, whenever GPS data is available, automatic labelling of visited places becomes possible. If the observations are time-stamped, it becomes possible for the detection system to deliver the accurate timing for recognition of revisited places. This facilitates the analysis of other sensory inputs captured at the identified time for any further possible mission-specific analysis.