GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods

TLDR: A novel and comprehensive method for in silico benchmark generation and performance profiling of network inference methods available to the community as an open-source software called GNW, which provides a network motif analysis that reveals systematic prediction errors, thereby indicating potential ways of improving inference methods.

Abstract: Motivation: Over the last decade, numerous methods have been developed for inference of regulatory networks from gene expression data. However, accurate and systematic evaluation of these methods is hampered by the difficulty of constructing adequate benchmarks and the lack of tools for a differentiated analysis of network predictions on such benchmarks. Results: Here, we describe a novel and comprehensive method for in silico benchmark generation and performance profiling of network inference methods available to the community as an open-source software called GeneNetWeaver (GNW). In addition to the generation of detailed dynamical models of gene regulatory networks to be used as benchmarks, GNW provides a network motif analysis that reveals systematic prediction errors, thereby indicating potential ways of improving inference methods. The accuracy of network inference methods is evaluated using standard metrics such as precision-recall and receiver operating characteristic curves. We show how GNW can be used to assess the performance and identify the strengths and weaknesses of six inference methods. Furthermore, we used GNW to provide the international Dialogue for Reverse Engineering Assessments and Methods (DREAM) competition with three network inference challenges (DREAM3, DREAM4 and DREAM5). Availability: GNW is available at http://gnw.sourceforge.net along with its Java source code, user manual and supporting data. Supplementary information: Supplementary data are available at Bioinformatics online.