Showing papers in "The Visual Computer in 2011"

Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes

Abstract: With the increasing amount of 3D data and the ability of capture devices to produce low-cost multimedia data, the capability to select relevant information has become an interesting research field. In 3D objects, the aim is to detect a few salient structures which can be used, instead of the whole object, for applications like object registration, retrieval, and mesh simplification. In this paper, we present an interest points detector for 3D objects based on Harris operator, which has been used with good results in computer vision applications. We propose an adaptive technique to determine the neighborhood of a vertex, over which the Harris response on that vertex is calculated. Our method is robust to several transformations, which can be seen in the high repeatability values obtained using the SHREC feature detection and description benchmark. In addition, we show that Harris 3D outperforms the results obtained by recent effective techniques such as Heat Kernel Signatures.

3D facial expression recognition using SIFT descriptors of automatically detected keypoints

Abstract: Methods to recognize humans’ facial expressions have been proposed mainly focusing on 2D still images and videos. In this paper, the problem of person-independent facial expression recognition is addressed using the 3D geometry information extracted from the 3D shape of the face. To this end, a completely automatic approach is proposed that relies on identifying a set of facial keypoints, computing SIFT feature descriptors of depth images of the face around sample points defined starting from the facial keypoints, and selecting the subset of features with maximum relevance. Training a Support Vector Machine (SVM) for each facial expression to be recognized, and combining them to form a multi-class classifier, an average recognition rate of 78.43% on the BU-3DFE database has been obtained. Comparison with competitor approaches using a common experimental setting on the BU-3DFE database shows that our solution is capable of obtaining state of the art results. The same 3D face representation framework and testing database have been also used to perform 3D facial expression retrieval (i.e., retrieve 3D scans with the same facial expression as shown by a target subject), with results proving the viability of the proposed solution.

Fractal dimension based neurofeedback in serious games

Abstract: EEG-based technology is widely used in serious game design since more wireless headsets that meet consumer criteria for wearability, price, portability, and ease-of-use are coming to the market. Originally, such technologies were mostly used in different medical applications, Brain Computer Interfaces (BCI) and neurofeedback games. The algorithms adopted in such applications are mainly based on power spectrum analysis, which may not be fully revealing the nonlinear complexity of the brain activities. In this paper, we first review neurofeedback games, EEG processing methods, and algorithms, and then propose a new nonlinear fractal dimension based approach to neurofeedback implementation targeting EEG-based serious games design. Only one channel is used in the proposed concentration quantification algorithm. The developed method was compared with other methods used for the concentration level recognition in neurofeedback games. The result analysis demonstrated an efficiency of the proposed approach. We designed and implemented new EEG-based 2D and 3D neurofeedback games that make the process of brain training more enjoyable.

Reconstruction of high contrast images for dynamic scenes

Abstract: High Dynamic Range (HDR) imaging requires one to composite multiple, differently exposed images of a scene in the irradiance domain and perform tone mapping of the generated HDR image for displaying on Low Dynamic Range (LDR) devices. In the case of dynamic scenes, standard techniques may introduce artifacts called ghosts if the scene changes are not accounted for. In this paper, we consider the blind HDR problem for dynamic scenes. We develop a novel bottom-up segmentation algorithm through superpixel grouping which enables us to detect scene changes. We then employ a piecewise patch-based compositing methodology in the gradient domain to directly generate the ghost-free LDR image of the dynamic scene. Being a blind method, the primary advantage of our approach is that we do not assume any knowledge of camera response function and exposure settings while preserving the contrast even in the non-stationary regions of the scene. We compare the results of our approach for both static and dynamic scenes with that of the state-of-the-art techniques.

Video dehazing with spatial and temporal coherence

Abstract: This paper describes a new framework for video dehazing, the process of restoring the visibility of the videos taken under foggy scenes. The framework builds upon techniques in single image dehazing, optical flow estimation and Markov random field. It aims at improving the temporal and spatial coherence of the dehazed video. In this framework, we first extract the transmission map frame-by-frame using guided filter, then estimate the forward and backward optical flow between two neighboring frames to find the matched pixels. The flow fields are used to help us building an MRF model on the transmission map to improve the spatial and temporal coherence of the transmission. The proposed algorithm is verified in both real and synthetic videos. The results demonstrate that our algorithm can preserve the spatial and temporal coherence well. With more coherent transmission map, we get better refocusing effect. We also apply our framework on improving the video coherence on the application of video denoising.

Image-based rendering of intersecting surfaces for dynamic comparative visualization

Abstract: Nested or intersecting surfaces are proven techniques for visualizing shape differences between static 3D objects (Weigle and Taylor II, IEEE Visualization, Proceedings, pp. 503–510, 2005). In this paper we present an image-based formulation for these techniques that extends their use to dynamic scenarios, in which surfaces can be manipulated or even deformed interactively. The formulation is based on our new layered rendering pipeline, a generic image-based approach for rendering nested surfaces based on depth peeling and deferred shading. We use layered rendering to enhance the intersecting surfaces visualization. In addition to enabling interactive performance, our enhancements address several limitations of the original technique. Contours remove ambiguity regarding the shape of intersections. Local distances between the surfaces can be visualized at any point using either depth fogging or distance fields: Depth fogging is used as a cue for the distance between two surfaces in the viewing direction, whereas closest-point distance measures are visualized interactively by evaluating one surface’s distance field on the other surface. Furthermore, we use these measures to define a three-way surface segmentation, which visualizes regions of growth, shrinkage, and no change of a test surface compared with a reference surface. Finally, we demonstrate an application of our technique in the visualization of statistical shape models. We evaluate our technique based on feedback provided by medical image analysis researchers, who are experts in working with such models.

Parallel and efficient Boolean on polygonal solids

Abstract: We present a novel framework which can efficiently evaluate approximate Boolean set operations for B-rep models by highly parallel algorithms. This is achieved by taking axis-aligned surfels of Layered Depth Images (LDI) as a bridge and performing Boolean operations on the structured points. As compared with prior surfel-based approaches, this paper has much improvement. Firstly, we adopt key-data pairs to store LDI more compactly. Secondly, robust depth peeling is investigated to overcome the bottleneck of layer-complexity. Thirdly, an out-of-core tiling technique is presented to overcome the limitation of memory. Real-time feedback is provided by streaming the proposed pipeline on the many-core graphics hardware.

Robust interactive cutting based on an adaptive octree simulation mesh

Abstract: We present an adaptive octree based approach for interactive cutting of deformable objects. Our technique relies on efficient refine- and node split-operations. These are sufficient to robustly represent cuts in the mechanical simulation mesh. A high-resolution surface embedded into the octree is employed to represent a cut visually. Model modification is performed in the rest state of the object, which is accomplished by back-transformation of the blade geometry. This results in an improved robustness of our approach. Further, an efficient update of the correspondences between simulation elements and surface vertices is proposed. The robustness and efficiency of our approach is underlined in test examples as well as by integrating it into a prototype surgical simulator.

Saliency-driven scaling optimization for image retargeting

Abstract: This paper proposes a saliency-weighted scaling factor energy for image retargeting. Considering that salient objects should be scaled with a larger scaling factor with respect to nonsalient regions, we define a quadric energy to establish the relation between the scaling factor of a local region and its saliency. The quadric energy is the weighted sum of the square of scaling factors, where the weight of each scaling factor is inversely proportional to its corresponding saliency. Furthermore, a triangle similarity quadric energy is introduced to prevent salient regions from distortion. Compared to previous methods, our approach not only preserves the shapes of salient objects and the integrity of the whole image well, but also reserves more resolution to salient objects in target image even when the aspect ratio is unchanged.

Separating semantics from rendering: a scene graph based architecture for graphics applications

Abstract: A large number of rendering and graphics applications developed in research and industry are based on scene graphs. Traditionally, scene graphs encapsulate the hierarchical structure of a complete 3D scene, and combine both semantic and rendering aspects. In this paper, we propose a clean separation of the semantic and rendering parts of the scene graph. This leads to a generally applicable architecture for graphics applications that is loosely based on the well-known Model-View-Controller (MVC) design pattern for separating the user interface and computation parts of an application. We explore the benefits of this new design for various rendering and modeling tasks, such as rendering dynamic scenes, out-of-core rendering of large scenes, generation of geometry for trees and vegetation, and multi-view rendering. Finally, we show some of the implementation details that have been solved in the process of using this software architecture in a large framework for rapid development of visualization and rendering applications.

Evaluation of a geometry-based knee joint compared to a planar knee joint

Abstract: Today neuromuscular simulations are used in several fields, such as diagnostics and planing of surgery, to get a deeper understanding of the musculoskeletal system. During the last year, new models and datasets have been presented which can provide us with more in-depth simulations and results. The same kind of development has occurred in the field of studying the human knee joint using complex three dimensional finite element models and simulations. In the field of musculoskeletal simulations, no such knee joints can be used. Instead the most common knee joint description is an idealized knee joint with limited accuracy or a planar knee joint which only describes the knee motion in a plane. In this paper, a new knee joint based on both equations and geometry is introduced and compared to a common clinical planar knee joint. The two kinematical models are analyzed using a gait motion, and are evaluated using the muscle activation and joint reaction forces which are compared to in-vivo measured forces. We show that we are able to predict the lateral, anterior and longitudinal moments, and that we are able to predict better knee and hip joint reaction forces.

YaDiV—an open platform for 3D visualization and 3D segmentation of medical data

Abstract: In this work, we present the concept, design and implementation of a new software to visualize and segment 3-dimensional medical data. The main goal was to create a platform that would allow trying out new approaches and ideas while staying independent from hardware and operating system, being especially useful for interdisciplinary research groups. A special focus will be given on fast and interactive volume visualization, and a survey on the use of Virtual Reality (VR) and especially haptic/force feedback in medical applications will be provided. The software will be published as Open Source and therefore be available as a rapid prototyping platform for own ideas and plugins for all members of the scientific community.

Landmark-free posture invariant human shape correspondence

Abstract: We consider the problem of computing accurate point-to-point correspondences among a set of human bodies in varying postures using a landmark-free approach. The approach learns the locations of the anthropometric landmarks present in a database of human models in strongly varying postures and uses this knowledge to automatically predict the locations of these anthropometric landmarks on a newly available scan. The predicted landmarks are then used to compute point-to-point correspondences between a rigged template model and the newly available scan.

Painterly rendering with content-dependent natural paint strokes

Abstract: We present a new painterly rendering method that simulates artists’ content-dependent painting process and the natural variation of hand-painted strokes. First, a new stroke layout strategy is proposed to enhance the contrast between large and small paint strokes, which is an important characteristic of hand-painted paintings. Specifically, the input image is partitioned into nonuniform grids according to its importance map, and determined by the grid size, an individually constructed paint stroke is applied in each grid. Second, an anisotropic digital brush is designed to simulate a real paint brush. In particular, each bristle of the digital brush has an individual color, so that strokes rendered by the new brush can have multiple colors and naturally varied textures. Finally, we present a novel method to add lighting effects to the canvas. This lighting imitation method is robust and very easy to implement, and it can significantly improve the quality of rendering. Comparing with traditional painterly rendering approaches, the new method simulates more closely the real painting procedure, and our experimental results show that it can produce vivid paintings with fewer artifacts.

Grid-based SAH BVH construction on a GPU

Abstract: We present an efficient algorithm for building an adaptive bounding volume hierarchy (BVH) in linear time on commodity graphics hardware using CUDA. BVHs are widely used as an acceleration data structure to quickly ray trace animated polygonal scenes. We accelerate the construction process with auxiliary grids that help us build high quality BVHs with SAH in O(k∗n). We partition scene triangles and build a temporary grid structure only once. We also handle non-uniformly tessellated and long/thin triangles that we split into several triangle references with tight bounding box approximations. We make no assumptions on the type of geometry or animation motion. However, our algorithm takes advantage of coherent geometry layout and coherent frame-by-frame motion. We demonstrate the performance and quality of resulting BVHs that are built quickly with good spatial partitioning.

Realistic simulation of mixing fluids

Abstract: Recently, simulation of mixing fluids, for which wide applications can be found in multimedia, computer games, special effects, virtual reality, etc., is attracting more and more attention. Most previous methods focus separately on binary immiscible mixing fluids or binary miscible mixing fluids. Until now, little attention has been paid to realistic simulation of multiple mixing fluids. In this paper, based on the solution principles in physics, we present a unified framework for realistic simulation of liquid–liquid mixing with different solubility, which is called LLSPH. In our method, the mixing process of miscible fluids is modeled by a heat-conduction-based Smooth Particle Hydrodynamics method. A special self-diffusion coefficient is designed to simulate the interactions between miscible fluids. For immiscible fluids, marching-cube-based method is adopted to trace the interfaces between different types of fluids efficiently. Then, an optimized spatial hashing method is adopted for simulation of boundary-free mixing fluids such as the marine oil spill. Finally, various realistic scenes of mixing fluids are rendered using our method.

Joint reversible watermarking and progressive compression of 3D meshes

Abstract: A new reversible 3D mesh watermarking scheme is proposed in conjunction with progressive compression. Progressive 3D mesh compression permits a progressive refinement of the model from a coarse to a fine representation by using different levels of detail (LoDs). A reversible watermark is embedded into all refinement levels such that (1) the refinement levels are copyright protected, and (2) an authorized user is able to reconstruct the original 3D model after watermark extraction, hence reversible. The progressive compression considers a connectivity-driven algorithm to choose the vertices that are to be refined for each LoD. The proposed watermarking algorithm modifies the geometry information of these vertices based on histogram bin shifting technique. An authorized user can extract the watermark in each LoD and recover the original 3D mesh, while an unauthorized user which has access to the decompression algorithm can only reconstruct a distorted version of the 3D model. Experimental results show that the proposed method is robust to several attack scenarios while maintaining a good compression ratio.

Part-in-whole 3D shape matching and docking

Abstract: A new algorithmic framework is proposed to efficiently recognize instances of template shapes within target 3D models or scenes. The new framework provides an efficient solution of the part-in-whole matching problem and, with simple adaptations, it can also be exploited to quickly select sites in the target which properly fit with the template. Therefore, the method proposed potentially offers a new approach to all applications where complementarity has to be analysed quickly such as, for instance, docking. By assuming that the template is small when compared to the target, the proposed approach distinguishes from the previous literature because the part-in-whole matching is obtained by extracting offline only the shape descriptor of the template, while the description of the target is dynamically and adaptively extracted during the matching process. This novel framework, called the Fast Reject schema, exploits the incremental nature of a class of local shape descriptors to significantly reduce the part-in-whole matching time, without any expensive processing of the models for the extraction of the shape descriptors. The schema has been tested on three different descriptors and results are discussed in detail. Experiments show that the gain in computational performances does not compromise the accuracy of the matching results. An additional descriptor is introduced to compute parts of the target having a complementary shape with respect to the template. Results of such a shape complementarity detection are shown in domains such as cultural heritage and drug design.

Interpolated eigenfunctions for volumetric shape processing

Abstract: This paper introduces a set of volumetric functions suitable for geometric processing of volumes. We start with Laplace–Beltrami eigenfunctions on the bounding surface and interpolate them into the interior using barycentric coordinates. The interpolated eigenfunctions: (1) can be computed efficiently by using the boundary mesh only; (2) can be seen as a shape-aware generalization of barycentric coordinates; (3) can be used for efficiently representing volumetric functions; (4) can be naturally plugged into existing spectral embedding constructions such as the diffusion embedding to provide their volumetric counterparts. Using the interior diffusion embedding, we define the interior Heat Kernel Signature (iHKS) and examine its performance for the task of volumetric point correspondence. We show that the three main qualities of the surface Heat Kernel Signature—being informative, multiscale, and insensitive to pose—are inherited by this volumetric construction. Next, we construct a bag of features based shape descriptor that aggregates the iHKS signatures over the volume of a shape, and evaluate its performance on a public shape retrieval benchmark. We find that while, theoretically, strict isometry invariance requires concentrating on the intrinsic surface properties alone, yet, practically, pose insensitive shape retrieval can be achieved using volumetric information.

A GPU framework for parallel segmentation of volumetric images using discrete deformable models

Abstract: Despite the ability of current GPU processors to treat heavy parallel computation tasks, its use for solving medical image segmentation problems is still not fully exploited and remains challenging. A lot of difficulties may arise related to, for example, the different image modalities, noise and artifacts of source images, or the shape and appearance variability of the structures to segment. Motivated by practical problems of image segmentation in the medical field, we present in this paper a GPU framework based on explicit discrete deformable models, implemented over the NVidia CUDA architecture, aimed for the segmentation of volumetric images. The framework supports the segmentation in parallel of different volumetric structures as well as interaction during the segmentation process and real-time visualization of the intermediate results. Promising results in terms of accuracy and speed on a real segmentation experiment have demonstrated the usability of the system.

Maintaining frame rate perception in interactive environments by exploiting audio-visual cross-modal interaction

Abstract: The entertainment industry, primarily the video games industry, continues to dictate the development and performance requirements of graphics hardware and computer graphics algorithms. However, despite the enormous progress in the last few years, it is still not possible to achieve some of industry’s demands, in particular high-fidelity rendering of complex scenes in real-time, on a single desktop machine. A realisation that sound/music and other senses are important to entertainment led to an investigation of alternative methods, such as cross-modal interaction in order to try and achieve the goal of “realism in real-time”. In this paper we investigate the cross-modal interaction between vision and audition for reducing the amount of computation required to compute visuals by introducing movement related sound effects. Additionally, we look at the effect of camera movement speed on temporal visual perception. Our results indicate that slow animations are perceived as smoother than fast animations. Furthermore, introducing the sound effect of footsteps to walking animations further increased the animation smoothness perception. This has the consequence that for certain conditions, the number of frames that need to be rendered each second can be reduced, saving valuable computation time, without the viewer being aware of this reduction. The results presented are another step towards the full understanding of the auditory-visual cross-modal interaction and its importance for helping achieve “realism int real-time”.

Environment-Sensitive cloning in images

Abstract: We present an Environment-Sensitive image cloning technique which improves the previous gradient-based methods by taking into account the content of target scene. We create a reference image to represent the global feature of target image which could be further diffused into the cloned patch, and modify the diffusion process to ensure that the cloning result is seamless and natural. Specifically, we figure out an efficient solution based on Mean-Value Coordinates (MVC) to deal with the hybrid boundary, and construct a general model based on MVC to implement our image cloning, which is further applied to video cloning. Experimental results demonstrate the effectiveness of our Environment-Sensitive cloning.

An interactive design system for pop-up cards with a physical simulation

Abstract: We present an interactive system that allows users to design original pop-up cards. A pop-up card is an interesting form of papercraft consisting of folded paper that forms a three-dimensional structure when opened. However, it is very difficult for the average person to design pop-up cards from scratch because it is necessary to understand the mechanism and determine the positions of objects so that pop-up parts do not collide with each other or protrude from the card. In the proposed system, the user interactively sets and edits primitives that are predefined in the system. The system simulates folding and opening of the pop-up card using a mass–spring model that can simply simulate the physical movement of the card. This simulation detects collisions and protrusions and illustrates the movement of the pop-up card. The results of the present study reveal that the user can design a wide range of pop-up cards using the proposed system.

Interactive deformable models with quadratic bases in Bernstein–Bézier-form

Abstract: We present a physically based interactive simulation technique for de formable objects Our method models the geometry as well as the displacements using quadratic basis functions in Bernstein–Bezier form on a tetrahedral finite element mesh The Bernstein–Bezier formulation yields significant advantages compared to approaches using the monomial form The implementation is simplified, as spatial derivatives and integrals of the displacement field are obtained analytically avoiding the need for numerical evaluations of the elements’ stiffness matrices We introduce a novel traversal accounting for adjacency in order to accelerate the reconstruction of the global matrices We show that our proposed method can compensate the additional effort introduced by the co-rotational formulation to a large extent We validate our approach on several models and demonstrate new levels of accuracy and performance in comparison to current state-of-the-art

3D model alignment based on minimum projection area

Abstract: 3D model alignment is an important step for applications such as 3D model retrieval and 3D model recognition. In this paper, we propose a novel Minimum Projection Area-based (MPA) alignment method for pose normalization. Our method finds three principal axes to align a model: the first principal axis gives the minimum projection area when we perform an orthographic projection of the model in the direction parallel to this axis, the second axis is perpendicular to the first axis and gives the minimum projection area, and the third axis is the cross product of the first two axes. We devise an optimization method based on Particle Swarm Optimization to efficiently find the axis with minimum projection area. For application in retrieval, we further perform axis ordering and orientation in order to align similar models in similar poses. We have tested MPA on several standard databases which include rigid/non-rigid and open/watertight models. Experimental results demonstrate that MPA has a good performance in finding alignment axes which are parallel to the ideal canonical coordinate frame of models and aligning similar models in similar poses under different conditions such as model variations, noise, and initial poses. In addition, it achieves a better 3D model retrieval performance than several commonly used approaches such as CPCA, NPCA, and PCA.

Skeleton-aware size variations in digital mannequins

Abstract: The general trend in character modeling is toward the personalization of models with higher levels of visual realism. This becomes possible with the development of commodity computation resources that are capable of processing massive data in parallel across multiple processors. On the other hand, there is always a trade-off between the quantity of the model features that are simulated and the plausibility of the visual realism because of the limited computation resources. Also, to keep the resources’ to be used efficiently within the other modeling approaches such as skin reflectance, aging, animation, etc., one must consider the efficiency of the method being used in the simulation. In this paper, we present an efficient method to customize the size of a human body model to personalize it with industry standard parameters. One of the major contributions of this method is that it is possible to generate a range of different size body models by using anthropometry surveys. This process is not limited by data-driven mesh deformation but also adapts the skeleton and motion to keep the consistency between different body layers.

Modeling with blocks

Abstract: This paper presents a simple and general modeling primitive, called a block, based on a generalized cuboid shape. Blocks are laid out and connected together to constitute the base shape of complex objects, from which is extracted a control mesh that can contain both smooth and sharp edges. The volumetric nature of the blocks allows for easy topology specification, as well as CSG operations between blocks. The surface parameterization inherited from the block faces provides support for texturing and displacement functions to apply surface details. A variety of examples illustrate the generality of our blocks in both interactive and procedural modeling contexts.

Building look & feel concept models from color combinations: With applications in image classification, retrieval, and color transfer

Abstract: In this paper, we tackle the problem of associating combinations of colors to abstract concepts (e.g. capricious, classic, cool, delicate, etc.). Since such concepts are difficult to represent using single colors, we consider combinations of colors or color palettes. We leverage two novel databases for color palettes, and learn categorization models using both low and high level descriptors. It is shown that the Bag of Colors and Fisher Vectors are the most rewarding descriptors for palettes categorization and retrieval. A simple but novel and efficient method for cleaning weakly annotated data, whilst preserving the visual coherence of categories is also given. Finally, we demonstrate that abstract category models learned on color palettes can be used in different applications such as image personalization, concept-based palette, and image retrieval and color transfer.

Rapid scalar value classification and volume clipping for interactive 3D medical image visualization

Abstract: In many clinical scenarios, medical data visualization and interaction are important to physicians for exploring inner anatomical structures and extracting meaningful diagnostic information. Real-time high-quality volume rendering, artifact-free clipping, and rapid scalar value classification are important techniques employed in this process. Unfortunately, in practice, it is still difficult to achieve an optimal balance. In this paper, we present some strategies to address this issue, which are based on the calculation of segment-based post color attenuation and dynamic ray–plane intersection (RPI) respectively. When implemented within our visualization system, the new classification algorithm can deliver real-time performance while avoiding the “color over-accumulation” artifacts suffered by the commonly used acceleration algorithms that employ pre-integrated classification. Our new strategy can achieve an optimized balance between image quality and classification speed. Next, the RPI algorithm is used with opacity adjustment technique to effectively remove the “striping” artifacts on the clipping plane caused by the nonuniform integration length. Furthermore, we present techniques for multiple transfer function (TF) based anatomical feature enhancement and “keyhole” based endoscopic inner structure view. Finally, the algorithms are evaluated subjectively by radiologists and quantitatively compared using image power spectrum analysis.

Performance-based control interfaces using mixture of factor analyzers

Abstract: This paper introduces an approach to performance animation that employs a small number of inertial measurement sensors to create an easy-to-use system for an interactive control of a full-body human character. Our key idea is to construct a global model from a prerecorded motion database and utilize them to construct full-body human motion in a maximum a posteriori framework (MAP). We have demonstrated the effectiveness of our system by controlling a variety of human actions, such as boxing, golf swinging, and table tennis, in real time. One unique property of our system is its ability to learn priors from a large and heterogeneous motion capture database and use them to generate a wide range of natural poses, a capacity that has not been demonstrated in previous data-driven character posing systems.