Showing papers in "Graphical Models \/graphical Models and Image Processing \/computer Vision, Graphics, and Image Processing in 2021"

Landmark Detection and 3D Face Reconstruction for Caricature using a Nonlinear Parametric Model

Abstract: Caricature is an artistic abstraction of the human face by distorting or exaggerating certain facial features, while still retains a likeness with the given face. Due to the large diversity of geometric and texture variations, automatic landmark detection and 3D face reconstruction for caricature is a challenging problem and has rarely been studied before. In this paper, we propose the first automatic method for this task by a novel 3D approach. To this end, we first build a dataset with various styles of 2D caricatures and their corresponding 3D shapes, and then build a parametric model on vertex based deformation space for 3D caricature face. Based on the constructed dataset and the nonlinear parametric model, we propose a neural network based method to regress the 3D face shape and orientation from the input 2D caricature image. Ablation studies and comparison with state-of-the-art methods demonstrate the effectiveness of our algorithm design. Extensive experimental results demonstrate that our method works well for various caricatures. Our constructed dataset, source code and trained model are available at https://github.com/Juyong/CaricatureFace .

VGF-Net: Visual-Geometric fusion learning for simultaneous drone navigation and height mapping

Abstract: The drone navigation requires the comprehensive understanding of both visual and geometric information in the 3D world. In this paper, we present a Visual-Geometric Fusion Network (VGF-Net), a deep network for the fusion analysis of visual/geometric data and the construction of 2.5D height maps for simultaneous drone navigation in novel environments. Given an initial rough height map and a sequence of RGB images, our VGF-Net extracts the visual information of the scene, along with a sparse set of 3D keypoints that capture the geometric relationship between objects in the scene. Driven by the data, VGF-Net adaptively fuses visual and geometric information, forming a unified Visual-Geometric Representation. This representation is fed to a new Directional Attention Model (DAM), which helps enhance the visual-geometric object relationship and propagates the informative data to dynamically refine the height map and the corresponding keypoints. An entire end-to-end information fusion and mapping system is formed, demonstrating remarkable robustness and high accuracy on the autonomous drone navigation across complex indoor and large-scale outdoor scenes.

Heterogeneous porous scaffold generation using trivariate B-spline solids and triply periodic minimal surfaces

Abstract: A porous scaffold is a three-dimensional network structure composed of a large number of pores, and triply periodic minimal surfaces (TPMSs) are one of the conventional tools for designing porous scaffolds. However, discontinuity, incompleteness, and high storage space requirements are the three main shortcomings of porous scaffold design using TPMSs. In this study, we developed an effective method for heterogeneous porous scaffold generation to overcome the abovementioned shortcomings of porous scaffold design. The input of the proposed method is a trivariate B-spline solid with a cubic parametric domain. The proposed method first constructs a threshold distribution field (TDF) in the cubic parametric domain, and then produces a continuous and complete TPMS within it. Finally, by mapping the TPMS in the parametric domain to the trivariate B-spline solid, a continuous and complete porous scaffold is generated. Moreover, we defined a new storage space-saving file format based on the TDF to store porous scaffolds. The experimental results presented in this paper demonstrate the effectiveness and efficiency of the method using a trivariate B-spline solid, as well as the superior space-saving of the proposed storage format.

Learning 3D face reconstruction from a single sketch

Abstract: 3D face reconstruction from a single image is a classic computer vision problem with many applications. However, most works achieve reconstruction from face photos, and little attention has been paid to reconstruction from other portrait forms. In this paper, we propose a learning-based approach to reconstruct a 3D face from a single face sketch. To overcome the problem of no paired sketch-3D data for supervised learning, we introduce a photo-to-sketch synthesis technique to obtain paired training data, and propose a dual-path architecture to achieve synergistic 3D reconstruction from both sketches and photos. We further propose a novel line loss function to refine the reconstruction with characteristic details depicted by lines in sketches well preserved. Our method outperforms the state-of-the-art 3D face reconstruction approaches in terms of reconstruction from face sketches. We also demonstrate the use of our method for easy editing of details on 3D face models.

Scale-Adaptive ICP

Abstract: We present a new scale-adaptive ICP (Iterative Closest Point) method which aligns two objects that differ by rigid transformations (translations and rotations) and uniform scaling. The motivation is that input data may come in different scales (measurement units) which may not be known a priori, or when two range scans of the same object are obtained by different scanners. Classical ICP and its many variants do not handle this scale difference problem adequately. Our novel solution outperforms three different methods that estimate scale prior to alignment and a fourth method that, similar to ours, jointly optimizes for scale during the alignment.

Single Image Tree Reconstruction via Adversarial Network

Abstract: Realistic 3D tree reconstruction is still a tedious and time-consuming task in the graphics community. In this paper, we propose a simple and efficient method for reconstructing 3D tree models with high fidelity from a single image. The key to single image-based tree reconstruction is to recover 3D shape information of trees via a deep neural network learned from a set of synthetic tree models. We adopted a conditional generative adversarial network (cGAN) to infer the 3D silhouette and skeleton of a tree respectively from edges extracted from the image and simple 2D strokes drawn by the user. Based on the predicted 3D silhouette and skeleton, a realistic tree model that inherits the tree shape in the input image can be generated using a procedural modeling technique. Experiments on varieties of tree examples demonstrate the efficiency and effectiveness of the proposed method in reconstructing realistic 3D tree models from a single image.

TopoKnit: A Process-Oriented Representation for Modeling the Topology of Yarns in Weft-Knitted Textiles

Abstract: Machine knitted textiles are complex multi-scale material structures increasingly important in many industries, including consumer products, architecture, composites, medical, and military. Computational modeling, simulation, and design of industrial fabrics require efficient representations of the spatial, material, and physical properties of such structures. We propose a process-oriented representation, TopoKnit, that defines a foundational data structure for representing the topology of weft-knitted textiles at the yarn scale. Process space serves as an intermediary between the machine and fabric spaces, and supports a concise, computationally efficient evaluation approach based on on-demand, near constant-time queries. In this paper, we define the properties of the process space, and design a data structure to represent it and algorithms to evaluate it. We demonstrate the effectiveness of the representation scheme by providing results of evaluations of the data structure in support of common topological operations in the fabric space.

Combining convex hull and directed graph for fast and accurate ellipse detection

Abstract: Detecting ellipses from images is a fundamental task in many computer vision applications. However, due to the complexity of real-world scenarios, it is still a challenge to detect ellipses accurately and efficiently. In this paper, we propose a novel method to tackle this problem based on the fast computation of convex hull and directed graph, which achieves promising results on both accuracy and efficiency. We use Depth-First-Search to extract branch-free curves after adaptive edge detection. Line segments are used to represent the curvature characteristic of the curves, followed by splitting at sharp corners and inflection points to attain smooth arcs. Then the convex hull is constructed, together with the distance, length, and direction constraints, to find co-elliptic arc pairs. Arcs and their connectivity are encoded into a sparse directed graph, and then ellipses are generated via a fast access of the adjacency list. Finally, salient ellipses are selected subject to strict verification and weighted clustering. Extensive experiments are conducted on eight real-world datasets (six publicly available and two built by ourselves), as well as five synthetic datasets. Our method achieves the overall highest F-measure with competitive speed compared to representative state-of-the-art methods.

Learning a shared deformation space for efficient design-preserving garment transfer

Abstract: Garment transfer from a source mannequin to a shape-varying individual is a vital technique in computer graphics. Existing garment transfer methods are either time consuming or lack designed details especially for clothing with complex styles. In this paper, we propose a data-driven approach to efficiently transfer garments between two distinctive bodies while preserving the source design. Given two sets of simulated garments on a source body and a target body, we utilize the deformation gradients as the representation. Since garments in our dataset are with various topologies, we embed cloth deformation to the body. For garment transfer, the deformation is decomposed into two aspects, typically style and shape. An encoder-decoder network is proposed to learn a shared space which is invariant to garment style but related to the deformation of human bodies. For a new garment in a different style worn by the source human, our method can efficiently transfer it to the target body with the shared shape deformation, meanwhile preserving the designed details. We qualitatively and quantitatively evaluate our method on a diverse set of 3D garments that showcase rich wrinkling patterns. Experiments show that the transferred garments can preserve the source design even if the target body is quite different from the source one.

Visually smooth multi-UAV formation transformation

Abstract: Unmanned airborne vehicles (UAVs) are useful in both military and civilian operations. In this paper, we consider a recreational scenario, i.e., multi-UAV formation transformation show. A visually smooth transformation needs to enforce the following three requirements at the same time: (1) visually pleasing contour morphing - for any intermediate frame, the agents form a meaningful shape and align with the contour, (2) uniform placement - for any intermediate frame, the agents are (isotropically) evenly spaced, and (3) smooth trajectories - the trajectory of each agent is as rigid/smooth as possible and completely collision free. First, we use the technique of 2-Wasserstein distance based interpolation to generate a sequence of intermediate shape contours. Second, we consider the spatio-temporal motion of all the agents altogether, and integrate the uniformity requirement and the spatial coherence into one objective function. Finally, the optimal formation transformation plan can be inferred by collaborative optimization. Extensive experimental results show that our algorithm outperforms the existing algorithms in terms of visual smoothness of transformation, boundary alignment, uniformity of agents, and rigidity of trajectories. Furthermore, our algorithm is able to cope with some challenging scenarios including (1) source/target shapes with multiple connected components, (2) source/target shapes with different typology structures, and (3) existence of obstacles. Therefore, it has a great potential in the real multi-UAV light show. We created an animation to demonstrate how our algorithm works; See the demo at https://1drv.ms/v/s!AheMg5fKdtdugVL0aNFfEt_deTbT?e=le5poN .

Geometry-Based Layout Generation with Hyper-Relations AMONG Objects

Abstract: Recent studies show increasing demands and interests in automatic layout generation, while there is still much room for improving the plausibility and robustness. In this paper, we present a data-driven layout generation framework without model formulation and loss term optimization. We achieve and organize priors directly based on samples from datasets instead of sampling probabilistic distributions. Therefore, our method enables expressing relations among three or more objects that are hard to be mathematically modeled. Subsequently, a non-learning geometric algorithm is proposed to arrange objects considering constraints such as positions of walls and windows. Experiments show that the proposed method outperforms the state-of-the-art and our generated layouts are competitive to those designed by professionals. 1

BPA-GAN: Human motion transfer using body-part-aware generative adversarial networks

Abstract: Human motion transfer has many applications in human behavior analysis, training data augmentation, and personalization in mixed reality. We propose a Body-Parts-Aware Generative Adversarial Network (BPA-GAN) for image-based human motion transfer. Our key idea is to take advantage of the human body with segmented parts instead of using the human skeleton like most of existing methods to encode the human motion information. As a result, we improve the reconstruction quality, the training efficiency, and the temporal consistency via training multiple GANs in a local-to-global manner and adding regularization on the source motion. Extensive experiments show that our method outperforms the baseline and the state-of-the-art techniques in preserving the details of body parts.

Orthogonality of isometries in the conformal model of the 3D space

Abstract: Motivated by questions on orthogonality of isometries, we present a new construction of the conformal model of the 3D space using just elementary linear algebra. In addition to pictures that can help the readers to understand the conformal model, our approach allows to obtain matrix representation of isometries that can be useful, for example, in applications of computational geometry, including computer graphics, robotics, and molecular geometry.

Adaptive geometric sound propagation based on A-weighting variance measure

Abstract: We introduce an A-weighting variance measurement, an objective estimation of the sound quality generated by geometric acoustic methods. Unlike the previous measurement, which applies to the impulse response, our measurement establishes the relationship between the impulse response and the auralized sound that the user hears. We also develop interactive methods to evaluate the measurement at run time and an adaptive algorithm that balances quality and performance based on the measurement. Experiments show that our method is more efficient in a wide variety of scene geometry, input sound, reverberation, and path tracing strategies.

Hybrid function representation for heterogeneous objects

Abstract: Heterogeneous object modelling is an emerging area where geometric shapes are considered in concert with their internal physically-based attributes. This paper describes a novel theoretical and practical framework for modelling volumetric heterogeneous objects on the basis of a novel unifying functionally-based hybrid representation called HFRep. This new representation allows for obtaining a continuous smooth distance field in Euclidean space and preserves the advantages of the conventional representations based on scalar fields of different kinds without their drawbacks. We systematically describe the mathematical and algorithmic basics of HFRep. The steps of the basic algorithm are presented in detail for both geometry and attributes. To solve some problematic issues, we have suggested several practical solutions, including a new algorithm for solving the eikonal equation on hierarchical grids. Finally, we show the practicality of the approach by modelling several representative heterogeneous objects, including those of a time-variant nature.

Multi-scale and multi-level shape descriptor learning via a hybrid fusion network

Abstract: Discriminative and informative 3D shape descriptors are of fundamental significance to computer graphics applications, especially in the fields of geometry modeling and shape analysis. 3D shape descriptors, which reveal extrinsic/intrinsic properties of 3D shapes, have been well studied for decades and proved to be useful and effective in various analysis and synthesis tasks. Nonetheless, existing descriptors are mainly founded upon certain local differential attributes or global shape spectra, and certain combinations of both types. Conventional descriptors are typically customized for specific tasks with priori domain knowledge, which severely prevents their applications from widespread use. Recently, neural networks, benefiting from their powerful data-driven capability for general feature extraction from raw data without any domain knowledge, have achieved great success in many areas including shape analysis. In this paper, we present a novel hybrid fusion network (HFN) that learns multi-scale and multi-level shape representations via uniformly integrating a traditional region-based descriptor with modern neural networks. On one hand, we exploit the spectral graph wavelets (SGWs) to extract the shapes’ local-to-global features. On the other hand, the shapes are fed into a convolutional neural network to generate multi-level features simultaneously. Then a hierarchical fusion network learns a general and unified representation from these two different types of features which capture multi-scale and multi-level properties of the underlying shapes. Extensive experiments and comprehensive comparisons demonstrate our HFN can achieve better performance in common shape analysis tasks, such as shape retrieval and recognition, and the learned hybrid descriptor is robust, informative, and discriminative with more potential for widespread applications.

Bas-relief generation from point clouds based on normal space compression with real-time adjustment on CPU

Abstract: This paper presents a bas-relief generation algorithm from scattered point cloud directly. Compared with the popular gradient domain methods for mesh surface, this algorithm takes normal vectors as the operation object, making it independent of topology connection, thus more suitable for point clouds and easier to implement. By constructing linear equations of the bas-relief height and using the solution strategy of the subspace, this algorithm can adjustment the bas-relief effect in real-time relying on the computing power of a consumer CPU only. In addition, we also propose an iterative solution to generate a bas-relief model of a specified height. The experimental results indicate that our algorithm provides a unified solution for generating different types of bas-relief with good saturation and rich details.

An optimal triangle projector with prescribed area and orientation, application to position-based dynamics

Abstract: The vast majority of mesh-based modelling applications iteratively transform the mesh vertices under prescribed geometric conditions. This occurs in particular in methods cycling through the constraint set such as Position-Based Dynamics (PBD). A common case is the approximate local area preservation of triangular 2D meshes under external editing constraints. At the constraint level, this yields the nonconvex optimal triangle projection under prescribed area problem, for which there does not currently exist a direct solution method. In current PBD implementations, the area preservation constraint is linearised. The solution comes out through the iterations, without a guarantee of optimality, and the process may fail for degenerate inputs where the vertices are colinear or colocated. We propose a closed-form solution method and its numerically robust algebraic implementation. Our method handles degenerate inputs through a two-case analysis of the problem’s generic ambiguities. We show in a series of experiments in area-based 2D mesh editing that using optimal projection in place of area constraint linearisation in PBD speeds up and stabilises convergence.

Normal manipulation for bas-relief modeling

Abstract: We introduce a normal-based modeling framework for bas-relief generation and stylization which is motivated by the recent advancement in this topic. Creating bas-relief from normal images has successfully facilitated bas-relief modeling in image space. However, the use of normal images in previous work is restricted to the cut-and-paste or blending operations of layers. These operations simply treat a normal vector as a pixel of a general color image. This paper is intended to extend normal-based methods by processing the normal image from a geometric perspective. Our method can not only generate a new normal image by combining various frequencies of existing normal images and details transferring, but also build bas-reliefs from a single RGB image and its edge-based sketch lines. In addition, we introduce an auxiliary function to represent a smooth base surface or generate a layered global shape. To integrate above considerations into our framework, we formulate the bas-relief generation as a variational problem which can be solved by a screened Poisson equation. One important advantage of our method is that it can generate more styles than previous methods and thus it expands the bas-relief shape space. We experimented our method on a range of normal images and it compares favorably to other popular classic and state-of-the-art methods.

Wrinkle and curl distortion of leaves using plant dynamic

Abstract: An algorithm was proposed to simulate the withering deformation of plant leaves by wrinkle and curl due to dehydration, based on cell dynamics and time-varying external force. First, a leaf boundary expansion algorithm was proposed to locate the feature points on the tip of the vein and construct the primary vein using a discrete geodesic path. Second, a novel mass-spring system by cell dynamics and a non-uniform mass distribution was defined to accelerate the movement of the boundary cells. Third, the cell swelling force was defined and adjusted to generate wrinkle deformation along with dehydration. Fourth, the time-varying external force on the feature points was defined to generate the curl deformation by adjusting the initial value of the external force and multiple iterative parameters. The implicit midpoint method was used to solve the equation of motion. The experimental results showed that our algorithm could simulate the wrinkle and curl deformation caused by dehydration and withering of leaves with high authenticity.