Research over the last decade has built a solid mathematical foundation for representation and analysis of 3D meshes in graphics and geometric modeling. Much of this work however does not explicitly incorporate models of low-level human visual attention. In this paper we introduce the idea of mesh saliency as a measure of regional importance for graphics meshes. Our notion of saliency is inspired by low-level human visual system cues. We define mesh saliency in a scale-dependent manner using a center-surround operator on Gaussian-weighted mean curvatures. We observe that such a definition of mesh saliency is able to capture what most would classify as visually interesting regions on a mesh. The human-perception-inspired importance measure computed by our mesh saliency operator results in more visually pleasing results in processing and viewing of 3D meshes. compared to using a purely geometric measure of shape. such as curvature. We discuss how mesh saliency can be incorporated in graphics applications such as mesh simplification and viewpoint selection and present examples that show visually appealing results from using mesh saliency.

Mesh saliency

We present the first end-to-end solution to create high-quality free-viewpoint video encoded as a compact data stream. Our system records performances using a dense set of RGB and IR video cameras, generates dynamic textured surfaces, and compresses these to a streamable 3D video format. Four technical advances contribute to high fidelity and robustness: multimodal multi-view stereo fusing RGB, IR, and silhouette information; adaptive meshing guided by automatic detection of perceptually salient areas; mesh tracking to create temporally coherent subsequences; and encoding of tracked textured meshes as an MPEG video stream. Quantitative experiments demonstrate geometric accuracy, texture fidelity, and encoding efficiency. We release several datasets with calibrated inputs and processed results to foster future research.

/pdf/high-quality-streamable-free-viewpoint-video-9b60auvkj6.pdf

High-quality streamable free-viewpoint video

3D object recognition from local features is robust to occlusions and clutter However, local features must be extracted from a small set of feature rich keypoints to avoid computational complexity and ambiguous features We present an algorithm for the detection of such keypoints on 3D models and partial views of objects The keypoints are highly repeatable between partial views of an object and its complete 3D model We also propose a quality measure to rank the keypoints and select the best ones for extracting local features Keypoints are identified at locations where a unique local 3D coordinate basis can be derived from the underlying surface in order to extract invariant features We also propose an automatic scale selection technique for extracting multi-scale and scale invariant features to match objects at different unknown scales Features are projected to a PCA subspace and matched to find correspondences between a database and query object Each pair of matching features gives a transformation that aligns the query and database object These transformations are clustered and the biggest cluster is used to identify the query object Experiments on a public database revealed that the proposed quality measure relates correctly to the repeatability of keypoints and the multi-scale features have a recognition rate of over 95% for up to 80% occluded objects

/pdf/on-the-repeatability-and-quality-of-keypoints-for-local-19x01qtbbv.pdf

On the Repeatability and Quality of Keypoints for Local Feature-based 3D Object Retrieval from Cluttered Scenes

With the increasing amount of 3D data and the ability of capture devices to produce low-cost multimedia data, the capability to select relevant information has become an interesting research field. In 3D objects, the aim is to detect a few salient structures which can be used, instead of the whole object, for applications like object registration, retrieval, and mesh simplification. In this paper, we present an interest points detector for 3D objects based on Harris operator, which has been used with good results in computer vision applications. We propose an adaptive technique to determine the neighborhood of a vertex, over which the Harris response on that vertex is calculated. Our method is robust to several transformations, which can be seen in the high repeatability values obtained using the SHREC feature detection and description benchmark. In addition, we show that Harris 3D outperforms the results obtained by recent effective techniques such as Heat Kernel Signatures.

/pdf/harris-3d-a-robust-extension-of-the-harris-operator-for-1qovow3prr.pdf

Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes

In this paper we revisit local feature detectors/descriptors developed for 2D images and extend them to the more general framework of scalar fields defined on 2D manifolds. We provide methods and tools to detect and describe features on surfaces equiped with scalar functions, such as photometric information. This is motivated by the growing need for matching and tracking photometric surfaces over temporal sequences, due to recent advancements in multiple camera 3D reconstruction. We propose a 3D feature detector (MeshDOG) and a 3D feature descriptor (MeshHOG) for uniformly triangulated meshes, invariant to changes in rotation, translation, and scale. The descriptor is able to capture the local geometric and/or photometric properties in a succinct fashion. Moreover, the method is defined generically for any scalar function, e.g., local curvature. Results with matching rigid and non-rigid meshes demonstrate the interest of the proposed framework.

/pdf/surface-feature-detection-and-description-with-applications-4sykxbz5qy.pdf

Surface feature detection and description with applications to mesh matching

In this paper, we introduce geometry-dependent lighting that allows lighting parameters to be defined independently and possibly discrepantly over an object or scene based on the local geometry. We present and discuss light collages, a lighting design system with geometry-dependent lights for effective feature-enhanced visualization. Our algorithm segments the objects into local surface patches and places lights that are locally consistent but globally discrepant to enhance the perception of shape. We use spherical harmonics for efficiently storing and computing light placement and assignment. We also outline a method to find the minimal number of light sources sufficient to illuminate an object well with our globally discrepant lighting approach.

/pdf/geometry-dependent-lighting-3y5v20fyau.pdf

Geometry-dependent lighting

We introduce Light Collages - a lighting design system for effective visualization based on principles of human perception. Artists and illustrators enhance perception of features with lighting that is locally consistent and globally inconsistent. Inspired by these techniques, we design the placement of light sources to convey a greater sense of realism and better perception of shape with globally inconsistent lighting. Our algorithm segments the objects into local surface patches and uses a number of perceptual heuristics, such as highlights, shadows, and silhouettes, to enhance the perception of shape. We show our results on scientific and sculptured datasets.

/pdf/light-collages-lighting-design-for-effective-visualization-572w48g4li.pdf

Light Collages: Lighting Design for Effective Visualization

http://www.kanungo.com/pubs/pr03-trueviz.pdf

The architecture of TrueViz: a groundTRUth/metadata editing and VIsualiZing ToolKit

The smooth molecular surface is defined as the surface which an external probe sphere touches as it is rolled over the spherical atoms of a molecule. The previous methods to compute smooth molecular surface assumed that each atom in a molecule has a fixed position without thermal motion or uncertainty. In real world, the position of an atom in a molecule is fuzzy because of its uncertainty in protein structure determination and thermal energy of the atom. In this paper, we propose a method to compute smooth molecular surface for fuzzy atoms. The Gaussian distribution is used for modeling the fuzziness of each atom, and an extended-radius p-probability sphere is computed for each atom with a certain confidence level. An extended-radius p-probability sphere is defined for atom i as the smallest sphere containing the atom i with a probability p. The fuzzy molecular surface is defined as a collection of molecular surfaces constructed from extended-radius p-probability spheres for each probability p. We have implemented a program for visualizing three-dimensional molecular structures including the fuzzy molecular surface using multi-layered transparent surfaces, where the surface of each layer has a different confidence level and the transparency associated with the confidence level.© (2002) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

/pdf/representing-thermal-vibrations-and-uncertainty-in-molecular-8fb4s8ph7l.pdf

Chang Ha Lee

Papers

Mesh saliency

Geometry-dependent lighting

Light Collages: Lighting Design for Effective Visualization

The architecture of TrueViz: a groundTRUth/metadata editing and VIsualiZing ToolKit

Representing thermal vibrations and uncertainty in molecular surfaces