System architecture and technology for real-time heterogeneous co-editing

• Main Author: Cho, Bryden Da Yang
• Other Authors: Tang Xueyan
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering::Software::Software engineering
• Online Access: https://hdl.handle.net/10356/155778
• Institution: Nanyang Technological University

Real-time co-editing systems support multiple geographically dispersed users to edit the same document over the Internet. After over two decades of extensive research and development in both academia and industries, real-world co-editors, such as Google Docs, Microsoft Office Online, and Codox Apps, are used by hundreds of millions of people around the world. Co-editing systems can be generally divided into two classes: one class is the homogeneous co-editing systems, in which all users must use the same editor in the same co-editing session; the other class is the heterogeneous co-editing systems, in which users may use different editors in the same co-editing session. Most research efforts in the past have focused on homogeneous co-editing systems, and nearly all real-world co-editing systems, such as those mentioned above, are homogeneous co-editors. In contrast, the research territory of heterogeneous co-editing technologies and systems has been underexplored. The potential benefits of supporting heterogeneous co-editing are multi-fold. First, there exist varieties of editors in the same editing domain that are used by different people in their individual work, hence it would be natural for them to expect that they can use their familiar editors for co-editing. Without the support for heterogeneous co-editing, users may have to give up on their favourite editors in co-editing work, which may adversely affect their work performance and even discourage them to collaborate. Second, a person is often more productive in their individual work when supported by his/her favourite editing tools; a group of people, by deduction, could be more productive in collaborative work when each of them is supported by their individually favourite editing tools. Third, challenging tasks, which are difficult to accomplish by an individual supported by a single specialized editing tool, may be effectively and efficiently accomplished by a group of people with diverse skill sets and supported by different editing tools with complementary editing features and functionalities. We believe that different editors will continue to exist as long as they are needed or liked by some people and the benefits of heterogeneous co-editing are evident, therefore supporting heterogeneous co-editing is an important direction for co-editing research and applications. The goal of this research is to advance the state-of-the-art co-editing technologies and systems by exploring the technical issues and their solutions for supporting heterogeneous co-editing. This research starts its exploration in the territory of heterogeneous co-editing from the plain-text co-editing domain because plain-text co-editing was the starting point for homogeneous co-editing research and has also served as the foundation for (both homogeneous and heterogeneous) co-editing in more advanced domains like rich-text editors, word processors, and even 3D digital media design tools, etc. This research is, however, not confined to the domain of plain-text co-editing: we aim to investigate issues and devise solutions that are not only applicable for heterogeneous co-editing in the plain-text domain, but are also general and extendable to support heterogeneous co-editing in other advanced editing domains. This research is driven by building and continuously experimenting with a working prototype system, named CoVim+CoEmacs, which allows people to use two different plain-text editors, namely Vim and Emacs, to edit shared text documents at the same time. The reason for choosing Vim and Emacs is that they are among the most commonly used plain-text editors, with very different editing capabilities and interaction styles, hence acting as excellent vehicles for studying heterogeneous co-editing issues in the plain-text domain. As a by-product, we reckon that heterogeneous co-editing systems may offer a sort of solution or alleviation to the so-called editor wars, e.g., the "holy war" between Vim and Emacs in the plain-text editing domain. When using rivaling editors (e.g., Vim and Emacs) in the same co-editing session, users can complement the shortcomings of other editors and accomplish complex tasks quicker and more efficiently. As a result, they may better appreciate the editing features and capabilities of other rivaling editors, learn from each other, and be more inclined to have peace with each other. In general, we believe that collaboration can help create harmony among people and resolve many challenges faced by humanity, including the editor wars. This research has identified and investigated two key issues in supporting heterogeneous co-editing: (1) the heterogeneous co-editing system architecture design and its core supporting techniques; and (2) general and efficient operation derivation solutions for deriving operations from different editors. The research outcomes achieved are summarized below. Inspired by the Transparent Adaptation (TA) approach, originally invented from the research in supporting homogeneous co-editing, and drawing insights from building the prototype CoVim+CoEmacs, we have conceptualized and experimentally validated a general TA+ architecture and its key technical components for building heterogeneous co-editors. Apart from inheriting the TA architecture, comprising the following functional components: (1) a Single-User component to provide conventional single-user editing functionalities in any application domain (e.g., plain-text editing, word processing, or 3D design); (2) a Generic Collaboration Engine component to provide generic co-editing capabilities including consistency maintenance; and (3) a Collaboration Adapter component to bridge between the SA and GCE components, the TA+ system architecture introduces one new core component ̶ the Heterogeneous Collaborator (HC), which plays the central role in resolving the heterogeneity issues among different editors used in the same co-editing session. This HC component encapsulates the following key technical sub-components: (1) a Common Model (CM) approach that resolves the data and operations heterogeneities among different editors by utilizing a common model representation; and (2) the Multi-Model Conversion (MMC) technical solutions for converting between the common data and operation model representation and their respective private models in each individual editor. The HC component decouples the heterogeneity issues from the rest of the TA+ system, which helps reduce complexity, provides good system modularity and component reusability in heterogeneous co-editors. We have designed and implemented the technical solutions for the HC component for plain-text co-editing in the CoVim+CoEmacs prototype. The general TA+ architecture and its key technical components for building heterogeneous co-editors have contributed to the advancement of state-of-the-art co-editing system design and implementation techniques. The other major technical challenge in building heterogeneous co-editors based on the TA/TA+ approach is to effectively, efficiently, and correctly derive user-editing operations from different editors with comprehensive and diverse suites of editing features, functionalities, and interaction styles. In early TA-based homogeneous co-editors, an interception-based-derivation approach was invented, which intercepts the user interaction events with the editor and converts those events into meaningful operations (e.g., Insert and Delete), and propagate those converted operations to remote co-editing sites for consistency maintenance and replay purposes. This approach was reasonably successful in supporting homogeneous co-editors (e.g., CoWord and CoVim 1.0) since a single editor is used at all co-editing sites. Thus, a single set of interception-based-derivation techniques (for supporting all editing functionalities of one editor) would be adequate. However, this approach is not suitable for heterogeneous co-editors due to the multiplied complexity in dealing with multiple editors with comprehensive and diverse editing functionalities, interaction styles, and various API (Application Programming Interface) constraints. In this research, we proposed a novel state-based operation derivation solution, which derives the editing operation performed by the user by examining the differences between the document states before and after user interactions that may affect the document states. This state-based operation derivation approach is particularly suitable for supporting heterogeneous co-editing as it can derive operations in a generalized way, i.e., one common set of state-based operation derivation algorithms can be used for all editors (regardless of their editing capabilities and interaction styles). To realize this state-based operation derivation approach in heterogeneous plain-text co-editing, we have devised a novel Real-Time Intention-Preserving Operation Derivation (RT-IPOD) solution that is capable of deriving various complex editing operations (e.g., macros, built-in commands, and special features, etc.) supported by the comprehensive and diverse editing functionalities and interaction styles of different plain-text editors for editing on a variety of plain-text document (e.g., HTML and source files of various programming languages, etc.). The RT-IPOD solution has been implemented and experimentally validated using the CoVim+CoEmacs prototype. Our solution is the only state-based operation derivation solution capable of meeting all three general design requirements, i.e., correctness, intention-preservation, and real-time co-editing constraints, for operation derivation solutions in real-time co-editing systems, which represents an important advancement to the state-of-the-art on operation derivation in real-time co-editing.