KNOWLEDGE REPRESENTATION WITH SPATIAL-TEMPORAL GRAPH ON SCENE UNDERSTANDING

• Main Author: Marzuki
• Format: Dissertations
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/57031
• Institution: Institut Teknologi Bandung

The increasing application of information technology to the understanding of the current scene aims to expand the application of visual data analysis to automated systems such as robotics, security systems, intelligent vehicles, health and others. Scene understanding is the integration of information at various levels which is a process carried out by machines to parse scene elements, classify and estimate geometry. Currently, the development of basic data for scene understanding, in addition to containing large-scale image data, is also comprehensive with information on the hierarchical structure and attributes of each scene category in the real world. This basic data can solve the problem of data limitations for researchers as a data exercise as well as test data on machine learning to expand information technology based on scene understanding. The expansion of visual analysis in information technology is expected to be able to make interpretations based on natural visual sensors (machine perception) like humans. Visual interpretation of scene understanding that is fundamental to machines is the machine's ability to visually identify places where an entity can move and navigate, map the relationships between objects and move objects in the spatial plane in real time. The application of these intelligent applications requires a data model that can represent a knowledge base that can be accessed in real time by machines during the information retrieval process. In this dissertation the problems of scene understanding that are discussed are: Scene understanding datasets if used in real time by machines can degrade their performance when processing query traversal quickly and rather poorly when reasoning to get semantic meaning. Semantic reasoning of scene categories through visual data processing can cause differences in the processing time and events if the query traversal decreases, while the application of understanding scenes in the real world has a high rhythm of change and a large computational burden for visual data processing. To solve the problems mentioned above, in this dissertation research was carried out in three stages. The first stage to solve the problem of using datasets in real time in real applications is solved by representing the scene understanding dataset with a property graph model as a knowledge based for the machine with the proposed Scene Understanding Knowledge Base Generator algorithm. The second stage is to complete the accuracy of scene categorization, a categorization inference algorithm based on graph reasoning with a probabilistic approach is proposed, and the third stage expands the use of observation graphs in the observation process carried out by machines for mapping and tracking objects in the spatial plane by developing a spatial mapping algorithm (Spatial Graph Mapping) and spatial-temporal graph tracker algorithm for object tracking. The proposals at the research stage of this dissertation are to be used in real systems simultaneously, so this research builds a Labelled Property Graphs (LPG) model as a spatialtemporal graph model to represent knowledge and machine inference as well as an expansion of scene understanding. The scene understanding database used is the SUN dataset and engine performance measurement on the algorithm developed using performance measurement with the Confusion matrix.