Showing papers on "Representation (systemics) published in 2021"
TL;DR: The High-Resolution Network (HRNet) as mentioned in this paper maintains high-resolution representations through the whole process by connecting the high-to-low resolution convolution streams in parallel and repeatedly exchanging the information across resolutions.
Abstract: High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions in series (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams in parallel and (ii) repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems. All the codes are available at https://github.com/HRNet .
1,162 citations
TL;DR: Interoception refers to the representation of the internal states of an organism, and includes the processes by which it senses, interprets, integrates, and regulates signals from within itself as discussed by the authors.
174 citations
01 Jun 2021
TL;DR: LiFeng et al. as discussed by the authors proposed Local Implicit Image Function (LIIF), which takes an image coordinate and the 2D deep features around the coordinate as inputs, predicts the RGB value at a given coordinate as an output.
Abstract: How to represent an image? While the visual world is presented in a continuous manner, machines store and see the images in a discrete way with 2D arrays of pixels. In this paper, we seek to learn a continuous representation for images. Inspired by the recent progress in 3D reconstruction with implicit neural representation, we propose Local Implicit Image Function (LIIF), which takes an image coordinate and the 2D deep features around the coordinate as inputs, predicts the RGB value at a given coordinate as an output. Since the coordinates are continuous, LIIF can be presented in arbitrary resolution. To generate the continuous representation for images, we train an encoder with LIIF representation via a self-supervised task with superresolution. The learned continuous representation can be presented in arbitrary resolution even extrapolate to ×30 higher resolution, where the training tasks are not provided. We further show that LIIF representation builds a bridge between discrete and continuous representation in 2D, it naturally supports the learning tasks with size-varied image ground-truths and significantly outperforms the method with resizing the ground-truths. Our project page with code is at https://yinboc.github.io/liif/.
124 citations
TL;DR: The model using semantic representation as input verifies that more accurate results can be obtained by introducing a high-level semantic representation, and shows that it is feasible and effective to introduce high- level and abstract forms of knowledge representation into deep learning tasks.
Abstract: In visual reasoning, the achievement of deep learning significantly improved the accuracy of results. Image features are primarily used as input to get answers. However, the image features are too redundant to learn accurate characterizations within a limited complexity and time. While in the process of human reasoning, abstract description of an image is usually to avoid irrelevant details. Inspired by this, a higher-level representation named semantic representation is introduced. In this paper, a detailed visual reasoning model is proposed. This new model contains an image understanding model based on semantic representation, feature extraction and process model refined with watershed and u-distance method, a feature vector learning model using pyramidal pooling and residual network, and a question understanding model combining problem embedding coding method and machine translation decoding method. The feature vector could better represent the whole image instead of overly focused on specific characteristics. The model using semantic representation as input verifies that more accurate results can be obtained by introducing a high-level semantic representation. The result also shows that it is feasible and effective to introduce high-level and abstract forms of knowledge representation into deep learning tasks. This study lays a theoretical and experimental foundation for introducing different levels of knowledge representation into deep learning in the future.
116 citations
TL;DR: A common object detection pipeline and taxonomy is introduced to facilitate a thorough comparison between different techniques and a comparison between performance results of the different models is included, alongside with some future research challenges.
112 citations
01 Jun 2021
TL;DR: In this paper, a scene representation network is combined with a low-dimensional morphable model which provides explicit control over pose and expressions for modeling the appearance and dynamics of a human face.
Abstract: We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face1. Digitally modeling and reconstructing a talking human is a key building-block for a variety of applications. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoint or headposes is required. In contrast to state-of-the-art approaches that model the geometry and material properties explicitly, or are purely image-based, we introduce an implicit representation of the head based on scene representation networks. To handle the dynamics of the face, we combine our scene representation network with a low-dimensional morphable model which provides explicit control over pose and expressions. We use volumetric rendering to generate images from this hybrid representation and demonstrate that such a dynamic neural scene representation can be learned from monocular input data only, without the need of a specialized capture setup. In our experiments, we show that this learned volumetric representation allows for photorealistic image generation that surpasses the quality of state-of-the-art video-based reenactment methods.
99 citations
Energy Institute1, Aalto University2, Potsdam Institute for Climate Impact Research3, University of Geneva4, Netherlands Environmental Assessment Agency5, European Institute6, École Normale Supérieure7, Utrecht University8, University of Paris9, Centre national de la recherche scientifique10, University of Exeter11, University of Cambridge12, Federal University of Rio de Janeiro13, International Institute for Applied Systems Analysis14
TL;DR: The IAM landscape is summarized, six prominent critiques emerging from the recent literature are discussed, and ways forward are suggested to reflect and respond to them in the light of IAM diversity and ongoing work.
Abstract: Integrated Assessment Models (IAMs) have emerged as key tools for building and assessing long term climate mitigation scenarios. Due to their central role in the recent IPCC assessments, and international climate policy analyses more generally, and the high uncertainties related to future projections, IAMs have been critically assessed by scholars from different fields receiving various critiques ranging from adequacy of their methods to how their results are used and communicated. Although IAMs are conceptually diverse and evolved in very different directions, they tend to be criticized under the umbrella of "IAMs". Here we first briefly summarise the IAM landscape and how models differ from each other. We then proceed to discuss six prominent critiques emerging from the recent literature, reflect and respond to them in the light of IAM diversity and ongoing work and suggest ways forward. The six critiques relate to (1) representation of heterogeneous actors in the models, (2) modelling of technology diffusion and dynamics, (3) representation of capital markets, (4) energy-economy feedbacks, (5) policy scenarios, and (6) interpretation and use of model results.
73 citations
TL;DR: This review focuses on an important property of the control representation's neural code: its representational dimensionality, which balances a basic separability/generalizability trade-off in neural computation.
Abstract: Cognitive control allows us to think and behave flexibly based on our context and goals. At the heart of theories of cognitive control is a control representation that enables the same input to produce different outputs contingent on contextual factors. In this review, we focus on an important property of the control representation's neural code: its representational dimensionality. Dimensionality of a neural representation balances a basic separability/generalizability trade-off in neural computation. We will discuss the implications of this trade-off for cognitive control. We will then briefly review current neuroscience findings regarding the dimensionality of control representations in the brain, particularly the prefrontal cortex. We conclude by highlighting open questions and crucial directions for future research.
65 citations
TL;DR: In this article, the authors examine the extent to which the appointment of employees to the board of directors influences market perceptions of environmental, social and governance (ESG) performance using a...
Abstract: In this paper, we examine the extent to which the appointment of employees to the board of directors influences market perceptions of environmental, social and governance (ESG) performance. Using a...
58 citations
TL;DR: The optimal selection of scene objects, semantic augmentation through the combination of various visual variables and dynamic augmented representation are discussed in detail and a debris flow that occurred Shuimo town is selected for experiment analysis, showing that most people are unaware of the risks posed by debris flow disasters.
Abstract: Virtual scenes can present rich and clear disaster information, which can significantly improve the level of public disaster perception. However, existing methods for constructing scenes of debris ...
57 citations
11 Mar 2021
TL;DR: Zhang et al. as discussed by the authors proposed an image-based local structured implicit network to improve the object shape estimation and refine the 3D object pose and scene layout via a novel implicit scene graph neural network that exploits the implicit local object features.
Abstract: We present a new pipeline for holistic 3D scene understanding from a single image, which could predict object shapes, object poses, and scene layout. As it is a highly ill-posed problem, existing methods usually suffer from inaccurate estimation of both shapes and layout especially for the cluttered scene due to the heavy occlusion between objects. We propose to utilize the latest deep implicit representation to solve this challenge. We not only propose an image-based local structured implicit network to improve the object shape estimation, but also refine the 3D object pose and scene layout via a novel implicit scene graph neural network that exploits the implicit local object features. A novel physical violation loss is also proposed to avoid incorrect context between objects. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods in terms of object shape, scene layout estimation, and 3D object detection.
TL;DR: In this paper, the authors present de novo approaches according to the coarseness of their molecular representation: that is, whether molecular design is modeled on an atom-based, fragment-based or reaction-based paradigm.
TL;DR: This article reviews recent progress in SLAM, focusing on advances in the expressive capacity of the environmental models used inSLAM systems (representation) and the performance of the algorithms used to estimate these models from data (inference).
Abstract: Simultaneous localization and mapping (SLAM) is the process of constructing a global model of an environment from local observations of it; this is a foundational capability for mobile robots, supp...
30 Jun 2021
TL;DR: For a survey of word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS), see as mentioned in this paper.
Abstract: Word representation has always been an important research area in the history of natural language processing (NLP). Understanding such complex text data is imperative, given that it is rich in information and can be used widely across various applications. In this survey, we explore different word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS). We describe a variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs. These models can transform large volumes of text into effective vector representations capturing the same semantic information. Further, such representations can be utilized by various machine learning (ML) algorithms for a variety of NLP-related tasks. In the end, this survey briefly discusses the commonly used ML- and DL-based classifiers, evaluation metrics, and the applications of these word embeddings in different NLP tasks.
TL;DR: A taxonomy of heterogeneous network representation learning algorithms according to different approaches of capturing semantic information in heterogeneous networks, including path based algorithms and semantic unit based algorithms is proposed.
TL;DR: A semi-supervised multi-view deep discriminant representation learning approach that incorporates the orthogonality and adversarial similarity constraints to reduce the redundancy of learned representations and to exploit the information contained in unlabeled data is proposed.
Abstract: Learning an expressive representation from multi-view data is a key step in various real-world applications. In this paper, we propose a semi-supervised multi-view deep discriminant representation learning (SMDDRL) approach. Unlike existing joint or alignment multi-view representation learning methods that cannot simultaneously utilize the consensus and complementary properties of multi-view data to learn inter-view shared and intra-view specific representations, SMDDRL comprehensively exploits the consensus and complementary properties as well as learns both shared and specific representations by employing the shared and specific representation learning network. Unlike existing shared and specific multi-view representation learning methods that ignore the redundancy problem in representation learning, SMDDRL incorporates the orthogonality and adversarial similarity constraints to reduce the redundancy of learned representations. Moreover, to exploit the information contained in unlabeled data, we design a semi-supervised learning framework by combining deep metric learning and density clustering. Experimental results on three typical multi-view learning tasks, i.e., webpage classification, image classification, and document classification demonstrate the effectiveness of the proposed approach.
TL;DR: In this article, the authors argue that information about events may be retained in multiple forms (e.g., event-specific sensory-near episodic memory, eventspecific gist information, event-general schematic information, or abstract semantic memory).
TL;DR: In this paper, a neural network is used to regress finger motions from input trajectories of wrists and objects and then predicts a new finger pose for the next frame as an autoregressive model.
Abstract: Natural hand manipulations exhibit complex finger maneuvers adaptive to object shapes and the tasks at hand. Learning dexterous manipulation from data in a brute force way would require a prohibitive amount of examples to effectively cover the combinatorial space of 3D shapes and activities. In this paper, we propose a hand-object spatial representation that can achieve generalization from limited data. Our representation combines the global object shape as voxel occupancies with local geometric details as samples of closest distances. This representation is used by a neural network to regress finger motions from input trajectories of wrists and objects. Specifically, we provide the network with the current finger pose, past and future trajectories, and the spatial representations extracted from these trajectories. The network then predicts a new finger pose for the next frame as an autoregressive model. With a carefully chosen hand-centric coordinate system, we can handle single-handed and two-handed motions in a unified framework. Learning from a small number of primitive shapes and kitchenware objects, the network is able to synthesize a variety of finger gaits for grasping, in-hand manipulation, and bimanual object handling on a rich set of novel shapes and functional tasks. We also demonstrate a live demo of manipulating virtual objects in real-time using a simple physical prop. Our system is useful for offline animation or real-time applications forgiving to a small delay.
TL;DR: An intercultural and intergenerational model highlights the complexity and diversity of the studied field, providing a reference framework for future studies and discussion of those findings of this study that are inconsistent with commonplace assumptions and conclusions present in the academic literature.
Abstract: Reflecting on the thousands of diverse research studies of social media representation and digital privacy, this article presents a comprehensive summary of online personal strategies. First, the e...
TL;DR: This work clarifies some of the aspects of minimal length models, with particular reference to the representation of the position operator, in relation to the commutation relation between position and momentum.
Abstract: Quantum mechanical models with a minimal length are often described by modifying the commutation relation between position and momentum. Although this represents a small complication when described in momentum space, at least formally, the (quasi-)position representation acquires numerous issues, source of misunderstandings. In this work, we review these issues, clarifying some of the aspects of minimal length models, with particular reference to the representation of the position operator.
17 Oct 2021
TL;DR: Zhang et al. as mentioned in this paper proposed an Adaptive Normalized Representation Learning (ANRL) framework, which adaptively selects feature normalization methods according to the inputs, aiming to learn domain-agnostic and discriminative representation.
Abstract: With various face presentation attacks arising under unseen scenarios, face anti-spoofing (FAS) based on domain generalization (DG) has drawn growing attention due to its robustness. Most existing methods utilize DG frameworks to align the features to seek a compact and generalized feature space. However, little attention has been paid to the feature extraction process for the FAS task, especially the influence of normalization, which also has a great impact on the generalization of the learned representation. To address this issue, we propose a novel perspective of face anti-spoofing that focuses on the normalization selection in the feature extraction process. Concretely, an Adaptive Normalized Representation Learning (ANRL) framework is devised, which adaptively selects feature normalization methods according to the inputs, aiming to learn domain-agnostic and discriminative representation. Moreover, to facilitate the representation learning, Dual Calibration Constraints are designed, including Inter-Domain Compatible loss and Inter-Class Separable loss, which provide a better optimization direction for generalizable representation. Extensive experiments and visualizations are presented to demonstrate the effectiveness of our method against the SOTA competitors.
25 Feb 2021
TL;DR: In this article, a framework for closed-loop robotic manipulation that automatically handles a category of objects, despite potentially unseen object instances and significant intra-category variations in shape, size and appearance is presented.
Abstract: In this letter, we explore generalizable, perception-to-action robotic manipulation for precise, contact-rich tasks. In particular, we contribute a framework for closed-loop robotic manipulation that automatically handles a category of objects, despite potentially unseen object instances and significant intra-category variations in shape, size and appearance. Previous approaches typically build a feedback loop on top of a realtime 6-DOF pose estimator. However, representing an object with a parameterized transformation from a fixed geometric template does not capture large intra-category shape variation. Hence we adopt the keypoint-based object representation proposed in [13] for category-level pick-and-place, and extend it to closed-loop manipulation policies with contact-rich tasks. We first augment keypoints with local orientation information. Using the oriented keypoints, we propose a novel object-centric action representation in terms of regulating the linear/angular velocity or force/torque of these oriented keypoints. This formulation is surprisingly versatile – we demonstrate that it can accomplish contact-rich manipulation tasks that require precision and dexterity for a category of objects with different shapes, sizes and appearances, such as peg-hole insertion for pegs and holes with significant shape variation and tight clearance. With the proposed object and action representation, our framework is also agnostic to the robot grasp pose and initial object configuration, making it flexible for integration and deployment. Video demonstration, source code and supplemental materials are available on https://sites.google.com/view/kpam2/home .
Book•
28 Jan 2021
TL;DR: This chapter introduces the core data model of the Semantic Web and linked data, that is the Resource Description Framework, RDF, and briefly introduces the Web Ontology Language (OWL) as a vocabulary to describe ontological and terminological knowledge and SPARQL, the query language for RDF andlinked data.
Abstract: This chapter introduces preliminaries that are essential to follow the content in the remainder of this book. First of all, we introduce the core data model of the Semantic Web and linked data, that is the Resource Description Framework, RDF. This format was designed in the 1990s and its core purpose is to represent data and knowledge in a Web-compatible fashion, taking into account that the Web can be regarded as a network of linked sites. RDF allows one to define networks of connected ‘things’ rather than a network of connected documents. We briefly introduce the semantics of RDF and also introduce the most popular serialization formats for RDF, that is N-Triples, Turtle, XML and JSON-LD. Glossing over many details, we briefly introduce the Web Ontology Language (OWL) as a vocabulary to describe ontological and terminological knowledge and SPARQL, the query language for RDF and linked data. Finally, we briefly discuss aspects of publishing linked data.
TL;DR: A new human action recognition method, skeleton edge motion networks (SEMN), to further explore the motion information of human body parts by using the angle changes of skeleton edge and the movement of the corresponding body joints, and proposes a new progressive ranking loss to help this method maintain temporal order information in a self-supervised manner.
TL;DR: The authors argue that the parliamentary representation of the radical right normalizes radical right support, and that stigmatized political preferences become normalized through the representation of right-wing extremists in the media.
Abstract: How do stigmatized political preferences become normalized? I argue that the parliamentary representation of the radical right normalizes radical right support. Radical right politicians breach est...