Learning and understanding user interface semantics from heterogeneous networks with multimodal and positional attributes

• Main Authors: ANG, Meng Kiat Gary, LIM, Ee-peng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Computing methodologies; Neural networks; Human-centered computing; User interface management systems; Artificial intelligence; Information systems; Multimedia information systems; Artificial Intelligence and Robotics; Databases and Information Systems; OS and Networks
• Online Access: https://ink.library.smu.edu.sg/sis_research/8323; https://ink.library.smu.edu.sg/context/sis_research/article/9326/viewcontent/3578522__1_.pdf
• Institution: Singapore Management University

User interfaces (UI) of desktop, web, and mobile applications involve a hierarchy of objects (e.g., applications, screens, view class, and other types of design objects) with multimodal (e.g., textual and visual) and positional (e.g., spatial location, sequence order, and hierarchy level) attributes...