Unsupervised image-to-image translation is an important and challenging problem in computer vision. Given an image in the source domain, the goal is to learn the conditional distribution of corresponding images in the target domain, without seeing any examples of corresponding image pairs. While this conditional distribution is inherently multimodal, existing approaches make an overly simplified assumption, modeling it as a deterministic one-to-one mapping. As a result, they fail to generate diverse outputs from a given source domain image. To address this limitation, we propose a Multimodal Unsupervised Image-to-image \(\text{ Translation } \text{(MUNIT) }\) framework. We assume that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties. To translate an image to another domain, we recombine its content code with a random style code sampled from the style space of the target domain. We analyze the proposed framework and establish several theoretical results. Extensive experiments with comparisons to state-of-the-art approaches further demonstrate the advantage of the proposed framework. Moreover, our framework allows users to control the style of translation outputs by providing an example style image. Code and pretrained models are available at https://github.com/nvlabs/MUNIT.

/pdf/multimodal-unsupervised-image-to-image-translation-29ozx7vi5e.pdf

Multimodal Unsupervised Image-to-Image Translation

Unsupervised image-to-image translation is an important and challenging problem in computer vision. Given an image in the source domain, the goal is to learn the conditional distribution of corresponding images in the target domain, without seeing any pairs of corresponding images. While this conditional distribution is inherently multimodal, existing approaches make an overly simplified assumption, modeling it as a deterministic one-to-one mapping. As a result, they fail to generate diverse outputs from a given source domain image. To address this limitation, we propose a Multimodal Unsupervised Image-to-image Translation (MUNIT) framework. We assume that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties. To translate an image to another domain, we recombine its content code with a random style code sampled from the style space of the target domain. We analyze the proposed framework and establish several theoretical results. Extensive experiments with comparisons to the state-of-the-art approaches further demonstrates the advantage of the proposed framework. Moreover, our framework allows users to control the style of translation outputs by providing an example style image. Code and pretrained models are available at this https URL

Here, a combination of biophysical measurement, modelling, and genetic and experimental manipulation of cell contractile components is used to analyse the formation of the inner cell mass in the early mouse embryo. How cells in mouse blastocyst sort themselves out to generate the inner cell mass, and how the embryos respond to manipulation during early development remain unexplained. Previous studies have indicated the importance of differential cell adhesion or oriented cell division along an apical–basal axis in the sorting phenomenon. Jean-Leon Maitre et al. use a combination of biophysical measurement, modelling and both genetic and experimental manipulation of contractile components to analyse inner cell mass formation in the early mouse embryo. They suggest that cell polarization generates cells of different contractilities, which trigger their sorting to inner and outer position. The contractile forces are shown to modulate the sub-cellular localization of Yap, a transcriptional regulator known to influence cell fate. During pre-implantation development, the mammalian embryo self-organizes into the blastocyst, which consists of an epithelial layer encapsulating the inner-cell mass (ICM) giving rise to all embryonic tissues1. In mice, oriented cell division, apicobasal polarity and actomyosin contractility are thought to contribute to the formation of the ICM2,3,4,5. However, how these processes work together remains unclear. Here we show that asymmetric segregation of the apical domain generates blastomeres with different contractilities, which triggers their sorting into inner and outer positions. Three-dimensional physical modelling of embryo morphogenesis reveals that cells internalize only when differences in surface contractility exceed a predictable threshold. We validate this prediction using biophysical measurements, and successfully redirect cell sorting within the developing blastocyst using maternal myosin (Myh9)-knockout chimaeric embryos. Finally, we find that loss of contractility causes blastomeres to show ICM-like markers, regardless of their position. In particular, contractility controls Yap subcellular localization6, raising the possibility that mechanosensing occurs during blastocyst lineage specification. We conclude that contractility couples the positioning and fate specification of blastomeres. We propose that this ensures the robust self-organization of blastomeres into the blastocyst, which confers remarkable regulative capacities to mammalian embryos.

/pdf/asymmetric-division-of-contractile-domains-couples-cell-7xrz6m4yt4.pdf

Asymmetric division of contractile domains couples cell positioning and fate specification

The construction of networks of maps among shapes in a collection enables a variety of applications in data-driven geometry processing. A key task in network construction is to make the maps consistent with each other. This consistency constraint, when properly defined, leads not only to a concise representation of such networks, but more importantly, it serves as a strong regularizer for correcting and improving noisy initial maps computed between pairs of shapes in isolation. Up-to-now, however, the consistency constraint has only been fully formulated for point-based maps or for shape collections that are fully similar. In this paper, we introduce a framework for computing consistent functional maps within heterogeneous shape collections. In such collections not all shapes share the same structure --- different types of shared structure may be present within different (but possibly overlapping) sub-collections. Unlike point-based maps, functional maps can encode similarities at multiple levels of detail (points or parts), and thus are particularly suitable for coping with such diversity within a shape collection. We show how to rigorously formulate the consistency constraint in the functional map setting. The formulation leads to a powerful tool for computing consistent functional maps, and also for discovering shared structures, such as meaningful shape parts. We also show how to adapt the procedure for handling very large-scale shape collections. Experimental results on benchmark datasets show that the proposed framework significantly improves upon state-of-the-art data-driven techniques. We demonstrate the usefulness of the framework in shape co-segmentation and various shape exploration tasks.

/pdf/functional-map-networks-for-analyzing-and-exploring-large-2g3on5vbac.pdf

Functional map networks for analyzing and exploring large shape collections

Realistic handling of fluid-air and fluid-solid interfaces in SPH is a challenging problem. The main reason is that some important physical phenomena such as surface tension and adhesion emerge as a result of inter-molecular forces in a microscopic scale. This is different from scalar fields such as fluid pressure, which can be plausibly evaluated on a macroscopic scale using particles. Although there exist techniques to address this problem for some specific simulation scenarios, there does not yet exist a general approach to reproduce the variety of effects that emerge in reality from fluid-air and fluid-solid interactions. In order to address this problem, we present a new surface tension force and a new adhesion force. Different from the existing work, our surface tension force can handle large surface tensions in a realistic way. This property lets our approach handle challenging real scenarios, such as water crown formation, various types of fluid-solid interactions, and even droplet simulations. Furthermore, it prevents particle clustering at the free surface where inter-particle pressure forces are incorrect. Our adhesion force allows plausible two-way attraction of fluids and solids and can be used to model different wetting conditions. By using our forces, modeling surface tension and adhesion effects do not require involved techniques such as generating a ghost air phase or surface tracking. The forces are applied to the neighboring fluid-fluid and fluid-boundary particle pairs in a symmetric way, which satisfies momentum conservation. We demonstrate that combining both forces allows simulating a variety of interesting effects in a plausible way.

/pdf/versatile-surface-tension-and-adhesion-for-sph-fluids-elv8y95hyw.pdf

Versatile surface tension and adhesion for SPH fluids

We consider the simulation of dense foams composed of microscopic bubbles, such as shaving cream and whipped cream. We represent foam not as a collection of discrete bubbles, but instead as a continuum. We employ the material point method (MPM) to discretize a hyperelastic constitutive relation augmented with the Herschel-Bulkley model of non-Newtonian viscoplastic flow, which is known to closely approximate foam behavior. Since large shearing flows in foam can produce poor distributions of material points, a typical MPM implementation can produce non-physical internal holes in the continuum. To address these artifacts, we introduce a particle resampling method for MPM. In addition, we introduce an explicit tearing model to prevent regions from shearing into artificially thin, honey-like threads. We evaluate our method's efficacy by simulating a number of dense foams, and we validate our method by comparing to real-world footage of foam.

/pdf/continuum-foam-a-material-point-method-for-shear-dependent-3l0i5ck8gd.pdf

Continuum Foam: A Material Point Method for Shear-Dependent Flows

We present the first reduced-dimensional technique to simulate the dynamics of thin sheets of viscous incompressible liquid in three dimensions. Beginning from a discrete Lagrangian model for elastic thin shells, we apply the Stokes-Rayleigh analogy to derive a simple yet consistent model for viscous forces. We incorporate nonlinear surface tension forces with a formulation based on minimizing discrete surface area, and preserve the quality of triangular mesh elements through local remeshing operations. Simultaneously, we track and evolve the thickness of each triangle to exactly conserve liquid volume. This approach enables the simulation of extremely thin sheets of viscous liquids, which are difficult to animate with existing volumetric approaches. We demonstrate our method with examples of several characteristic viscous sheet behaviors, including stretching, buckling, sagging, and wrinkling.

/pdf/discrete-viscous-sheets-39wxeqaxfe.pdf

Discrete viscous sheets

We present a triangle mesh-based technique for tracking the evolution of three-dimensional multimaterial interfaces undergoing complex deformations It is the first non-manifold triangle mesh tracking method to simultaneously maintain intersection-free meshes and support the proposed broad set of multimaterial remeshing and topological operations We represent the interface as a non-manifold triangle mesh with material labels assigned to each half-face to distinguish volumetric regions Starting from proposed application-dependent vertex velocities, we deform the mesh, seeking a non-intersecting, watertight solution This goal necessitates development of various collision-safe, label-aware non-manifold mesh operations: multimaterial mesh improvement; T1 and T2 processes, topological transitions arising in foam dynamics and multiphase flows; and multimaterial merging, in which a new interface is created between colliding materials We demonstrate the robustness and effectiveness of our approach on a range of scenarios including geometric flows and multiphase fluid animation

/pdf/multimaterial-mesh-based-surface-tracking-wztn6reqbe.pdf

Multimaterial mesh-based surface tracking

We propose a novel surface-only technique for simulating incompressible, inviscid and uniform-density liquids with surface tension in three dimensions. The liquid surface is captured by a triangle mesh on which a Lagrangian velocity field is stored. Because advection of the velocity field may violate the incompressibility condition, we devise an orthogonal projection technique to remove the divergence while requiring the evaluation of only two boundary integrals. The forces of surface tension, gravity, and solid contact are all treated by a boundary element solve, allowing us to perform detailed simulations of a wide range of liquid phenomena, including waterbells, droplet and jet collisions, fluid chains, and crown splashes.

Surface-only liquids

Simulating the delightful dynamics of soap films, bubbles, and foams has traditionally required the use of a fully three-dimensional many-phase Navier-Stokes solver, even though their visual appearance is completely dominated by the thin liquid surface. We depart from earlier work on soap bubbles and foams by noting that their dynamics are naturally described by a Lagrangian vortex sheet model in which circulation is the primary variable. This leads us to derive a novel circulation-preserving surface-only discretization of foam dynamics driven by surface tension on a non-manifold triangle mesh. We represent the surface using a mesh-based multimaterial surface tracker which supports complex bubble topology changes, and evolve the surface according to the ambient air flow induced by a scalar circulation field stored on the mesh. Surface tension forces give rise to a simple update rule for circulation, even at non-manifold Plateau borders, based on a discrete measure of signed scalar mean curvature. We further incorporate vertex constraints to enable the interaction of soap films with wires. The result is a method that is at once simple, robust, and efficient, yet able to capture an array of soap films behaviors including foam rearrangement, catenoid collapse, blowing bubbles, and double bubbles being pulled apart.

/pdf/double-bubbles-sans-toil-and-trouble-discrete-circulation-3dx98d63ic.pdf

Christopher Batty

Papers

Continuum Foam: A Material Point Method for Shear-Dependent Flows

Discrete viscous sheets

Multimaterial mesh-based surface tracking

Surface-only liquids

Double bubbles sans toil and trouble: discrete circulation-preserving vortex sheets for soap films and foams