The ground truth about metadata and community detection in networks
Reads0
Chats0
TLDR
It is proved that no algorithm can uniquely solve community detection, and a general No Free Lunch theorem for community detection is proved, which implies that there can be no algorithm that is optimal for all possible community detection tasks.Abstract:
Across many scientific domains, there is a common need to automatically extract a simplified view or coarse-graining of how a complex system's components interact. This general task is called community detection in networks and is analogous to searching for clusters in independent vector data. It is common to evaluate the performance of community detection algorithms by their ability to find so-called ground truth communities. This works well in synthetic networks with planted communities because these networks' links are formed explicitly based on those known communities. However, there are no planted communities in real-world networks. Instead, it is standard practice to treat some observed discrete-valued node attributes, or metadata, as ground truth. We show that metadata are not the same as ground truth and that treating them as such induces severe theoretical and practical problems. We prove that no algorithm can uniquely solve community detection, and we prove a general No Free Lunch theorem for community detection, which implies that there can be no algorithm that is optimal for all possible community detection tasks. However, community detection remains a powerful tool and node metadata still have value, so a careful exploration of their relationship with network structure can yield insights of genuine worth. We illustrate this point by introducing two statistical techniques that can quantify the relationship between metadata and community structure for a broad class of models. We demonstrate these techniques using both synthetic and real-world networks, and for multiple types of metadata and community structures.read more
Citations
More filters
Journal ArticleDOI
Community Discovery in Dynamic Networks: A Survey
Giulio Rossetti,Rémy Cazabet +1 more
TL;DR: In this article, the authors present the distinctive features and challenges of dynamic community discovery and propose a classification of published approaches, which can be used to identify the set of approaches that best fit their needs.
Journal ArticleDOI
Global energy flows embodied in international trade: A combination of environmentally extended input–output analysis and complex network analysis
TL;DR: In this paper, the authors apply a variety of complex network analysis tools to uncover the structure of embodied energy flow network (EEFN) at global, regional and national level, based on environmentally extended input-output analysis (EEIOA).
Journal ArticleDOI
Metrics for Community Analysis: A Survey
TL;DR: A survey of the metrics used for community detection and evaluation can be found in this paper, where the authors also conduct experiments on synthetic and real networks to present a comparative analysis of these metrics in measuring the goodness of the underlying community structure.
Journal ArticleDOI
The modular organization of human anatomical brain networks: Accounting for the cost of wiring
Richard F. Betzel,John D. Medaglia,Lia Papadopoulos,Graham L. Baum,Ruben C. Gur,Raquel E. Gur,David R. Roalf,Theodore D. Satterthwaite,Danielle S. Bassett +8 more
TL;DR: A modification of an existing module detection algorithm that allowed it to focus on connections that are unexpected under a cost-reduction wiring rule and to identify modules from among these connections, which support the hypothesis that brain networks are composed of modules and provide additional insight into the function of those modules.
Journal ArticleDOI
Community detection in node-attributed social networks: A survey
TL;DR: An exhaustive search of known methods is performed and a classification of them based on when and how structure and attributes are fused is proposed, which pays attention to available information which methods outperform others and which datasets and quality measures are used for their evaluation.
References
More filters
Journal ArticleDOI
Uncovering the community structure associated with the diffusion dynamics on networks
Xueqi Cheng,Huawei Shen +1 more
TL;DR: Wang et al. as discussed by the authors demonstrate that the intrinsic community structure of networks can be revealed by the stable local equilibrium states of the diffusion process, and they show that such community structure can be directly identified through the optimization of the network, which measures how easily the diffusion among different communities occurs.
Journal ArticleDOI
Network structure, metadata and the prediction of missing nodes.
TL;DR: This work constructs a joint generative model for the data and metadata, and a nonparametric Bayesian framework to infer its parameters from annotated datasets, and assess the quality of the metadata not according to its direct alignment with the network communities, but rather in its capacity to predict the placement of edges in the network.
Proceedings ArticleDOI
Computer science fields as ground-truth communities: their impact, rise and fall
TL;DR: This paper systematically studies the temporal interaction patterns of communities using a large scale citation network (directed and unweighted) of computer science to understand how each individual community grows/shrinks, becomes important over time.
Journal ArticleDOI
Testing and Modeling Dependencies Between a Network and Nodal Attributes.
Bailey K. Fosdick,Peter D. Hoff +1 more
TL;DR: In this article, a latent variable model is used to obtain a low-dimensional representation of the network in terms of node-specific network factors, and a novel testing procedure is introduced to determine if dependencies exist between the network factors.
The first steps.
TL;DR: $ mvn -version Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T10:41:47-06:00)