Santa Fe Institute

About: Santa Fe Institute is a nonprofit organization based out in Santa Fe, New Mexico, United States. It is known for research contribution in the topics: Population & Context (language use). The organization has 558 authors who have published 4558 publications receiving 396015 citations. The organization is also known as: SFI.

Where Next for Microbiome Research

Abstract: The development of high-throughput sequencing technologies has transformed our capacity to investigate the composition and dynamics of the microbial communities that populate diverse habitats. Over the past decade, these advances have yielded an avalanche of metagenomic data. The current stage of “van Leeuwenhoek”–like cataloguing, as well as functional analyses, will likely accelerate as DNA and RNA sequencing, plus protein and metabolic profiling capacities and computational tools, continue to improve. However, it is time to consider: what’s next for microbiome research? The short pieces included here briefly consider the challenges and opportunities awaiting microbiome research.

On the information bottleneck theory of deep learning

Abstract: The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior. In this work, we study the information bottleneck (IB) theory of deep learning, which makes three specific claims: first, that deep networks undergo two distinct phases consisting of an initial fitting phase and a subsequent compression phase; second, that the compression phase is causally related to the excellent generalization performance of deep networks; and third, that the compression phase occurs due to the diffusion-like behavior of stochastic gradient descent. Here we show that none of these claims hold true in the general case. Through a combination of analytical results and simulation, we demonstrate that the information plane trajectory is predominantly a function of the neural nonlinearity employed: double-sided saturating nonlinearities like tanh yield a compression phase as neural activations enter the saturation regime, but linear activation functions and single-sided saturating nonlinearities like the widely used ReLU in fact do not. Moreover, we find that there is no evident causal connection between compression and generalization: networks that do not compress are still capable of generalization, and vice versa. Next, we show that the compression phase, when it exists, does not arise from stochasticity in training by demonstrating that we can replicate the IB findings using full batch gradient descent rather than stochastic gradient descent. Finally, we show that when an input domain consists of a subset of task-relevant and task-irrelevant information, hidden representations do compress the task-irrelevant information, although the overall information about the input may monotonically increase with training time, and that this compression happens concurrently with the fitting process rather than during a subsequent compression period.

Automatic detection of conserved RNA structure elements in complete RNA virus genomes

Abstract: We propose a new method for detecting conserved RNA secondary structures in a family of related RNA sequences. Our method is based on a combination of thermodynamic structure prediction and phylogenetic comparison. In contrast to purely phylogenetic methods, our algorithm can be used for small data sets of approximately 10 sequences, efficiently exploiting the information contained in the sequence variability. The procedure constructs a prediction only for those parts of sequences that are consistent with a single conserved structure. Our implementation produces reasonable consensus structures without user interference. As an example we have analysed the complete HIV-1 and hepatitis C virus (HCV) genomes as well as the small segment of hantavirus. Our method confirms the known structures in HIV-1 and predicts previously unknown conserved RNA secondary structures in HCV.

Collective Dynamic Choice: The Necessity of Time Inconsistency†

Abstract: We study collective decisions by time-discounting individuals choosing a common consumption stream. We show that with any heterogeneity in time preferences, every Pareto efficient and non-dictatorial method of aggregating utility functions must be time-inconsistent. We also show that decisions made via non-dictatorial voting methods are intransitive.