Occlusion-aware stroke-based drawing, inbetweening, and painting

• Main Author: Jiang, Jie
• Other Authors: Seah Hock Soon
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Computer graphics
• Online Access: https://hdl.handle.net/10356/159319
• Institution: Nanyang Technological University

The traditional 2D animation process is labor-intensive and time-consuming. Advances in technology have helped artists produce animation more efficiently with graphical algorithms by reducing tedious works like drawing, inbetweening, and painting, which are three consecutive and tightly-coupled stages in the animation process. The drawing stage affects the inbetweening stage, which affects the painting stage. Artists typically draw with layers to organize strokes. When a drawing is composited, the layers are stacked together, which introduces another major challenge, that is occlusion resolution. Ideally, for a set of drawings, the occlusion must be properly resolved before proceeding to the inbetweening and painting stages. This is because it is costly to modify the drawings once the inbetweening and painting are done as it will need to go through the entire animation process to re-adjust occlusion and re-paint the drawings. There are three major difficulties, which are gaps, occlusion, and painting. In this research, we investigate these three problems. We propose a gap handling method for vector graphics, which dynamically and adaptively closes gaps during the drawing process. The presence of gaps prevents regions from being properly closed, which subsequently makes the regions un-colorable. Most existing gap handling methods require a fixed input gap size to determine gaps to close, making the methods sensitive to gap size and stroke modifications. In addition, the resulting appearance is not well considered when closing gaps, which usually results in undesired stroke connections. In our method, we introduce gap point analysis, which dynamically determines gap sizes based on the geometry of the strokes as they are being drawn in real time. This makes the method independent of gap size and adaptive to the changes in the drawings. To preserve appearance, we propose energy functions to estimate stroke connections, which considers angle deviation and distance deviation. This ensures gap closing results to be smooth. The experimental results demonstrate that our method performs well in handling drawings with different artistic styles. A major challenge of existing occlusion resolution methods is the requirement of closed shapes, which needs users to draw complete shapes of objects. We represent occlusion with boundary strokes, which have occluding sides that represent occluding surfaces with depths. With this representation, users can specify occlusion with either open or closed strokes rather than having to draw complete shapes. We further propose a geometry-based method to find the regions of occluding surfaces. Occlusion is dynamically resolved as strokes change. In addition, we combine boundary strokes with conventional layers. Users can manage strokes with layers and each stroke has an absolute depth for occlusion resolution, which allows inter-layer and intra-layer occlusion resolution. Our proposed boundary stroke-based drawing provides immediate feedback to users and offers them a flexible way to dynamically manage occlusion. We extend the boundary strokes to tackle occlusion in 2D stroke-based inbetweening to produce intermediate frames with occlusion resolved. The major difficulty of occlusion resolution in 2D stroke-based inbetweening is that as strokes are animated, the occluded portions are not invariant across the frames. Our boundary stroke-based representation solves this problem. Although strokes are animated, the occluding surfaces represented with boundary strokes are dynamically updated so that the occlusion in each frame can be properly represented. Due to the nature of boundary strokes, occlusion can be dynamically resolved based on geometry even if the animated drawings are with topology change and our experimental results also demonstrate this. Users also have control over occlusion throughout the drawing sequence. Finally, we propose a stroke-based painting approach for static drawings as well as animated drawings. It is a challenge to build the connection between painting and drawing to achieve drawing painting, color update, and auto-painting. In auto-painting, a continuous region in one frame can be split into several unconnected sub-regions in other frames, which is a challenge in auto-painting. In addition, the ability in painting regions formed by strokes from the same layer or different layers is also desired. We propose a stroke-based painting model by introducing color strokes that associate colors with strokes. A painted drawing can be represented with strokes. With this representation, users can paint a drawing with inter-layer and intra-layer regions. We also propose a region color resolution method. It allows users to edit drawings with colors being dynamically updated. When drawings are animated, each frame can be painted based on the strokes of the frame without cross-frame information. In summary, gaps are dynamically handled while drawing in real time and boundary strokes can be used to specify and resolve occlusion. Inbetween frames are computed from the key drawings with occlusion automatically resolved. Colors can be further added onto the drawings and the information stays with strokes. The color information of a reference frame can be used to auto-paint entire drawing sequence. When a frame is edited, the gaps, occlusion, and colors are dynamically updated without cross-frame information, which enables flexible frame-wise editing. Moreover, users can control the occlusion and colors over entire drawing sequence with strokes. Our stroke-based methods handle the gap, occlusion, and painting problems, which ensure the drawing, inbetweening, and painting stages are done. Meanwhile, they improve drawing editability during the animation process.