To evaluate binary classifications and their confusion matrices, scientific researchers can employ several statistical rates, accordingly to the goal of the experiment they are investigating. Despite being a crucial issue in machine learning, no widespread consensus has been reached on a unified elective chosen measure yet. Accuracy and F1 score computed on confusion matrices have been (and still are) among the most popular adopted metrics in binary classification tasks. However, these statistical measures can dangerously show overoptimistic inflated results, especially on imbalanced datasets. The Matthews correlation coefficient (MCC), instead, is a more reliable statistical rate which produces a high score only if the prediction obtained good results in all of the four confusion matrix categories (true positives, false negatives, true negatives, and false positives), proportionally both to the size of positive elements and the size of negative elements in the dataset. In this article, we show how MCC produces a more informative and truthful score in evaluating binary classifications than accuracy and F1 score, by first explaining the mathematical properties, and then the asset of MCC in six synthetic use cases and in a real genomics scenario. We believe that the Matthews correlation coefficient should be preferred to accuracy and F1 score in evaluating binary classification tasks by all scientific communities.

/pdf/the-advantages-of-the-matthews-correlation-coefficient-mcc-hn3s2s0jsg.pdf

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

Root turnover is a critical component of ecosystem nutrient dynamics and carbon sequestration and is also an important sink for plant primary productivity. We tested global controls on root turnover across climatic gradients and for plant functional groups by using a database of 190 published studies. Root turnover rates increased exponentially with mean annual temperature for fine roots of grasslands (r2 = 0.48) and forests (r2 = 0.17) and for total root biomass in shrublands (r2 = 0.55). On the basis of the best-fit exponential model, the Q10 for root turnover was 1.4 for forest small diameter roots (5 mm or less), 1.6 for grassland fine roots, and 1.9 for shrublands. Surprisingly, after accounting for temperature, there was no such global relationship between precipitation and root turnover. The slowest average turnover rates were observed for entire tree root systems (10% annually), followed by 34% for shrubland total roots, 53% for grassland fine roots, 55% for wetland fine roots, and 56% for forest fine roots. Root turnover decreased from tropical to high-latitude systems for all plant functional groups. To test whether global relationships can be used to predict interannual variability in root turnover, we evaluated 14 yr of published root turnover data from a shortgrass steppe site in northeastern Colorado, USA. At this site there was no correlation between interannual variability in mean annual temperature and root turnover. Rather, turnover was positively correlated with the ratio of growing season precipitation and maximum monthly temperature (r2 = 0.61). We conclude that there are global patterns in rates of root turnover between plant groups and across climatic gradients but that these patterns cannot always be used for the successful prediction of the relationship of root turnover to climate change at a particular site.

https://nph.onlinelibrary.wiley.com/doi/pdf/10.1046/j.1469-8137.2000.00681.x

Global patterns of root turnover for terrestrial ecosystems

In order to investigate the annual variation of soil respiration and its components in relation to seasonal changes in soil temperature and soil moisture in a Mediterranean mixed oak forest ecosystem, we set up a series of experimental treatments in May 1999 where litter (no litter), roots (no roots, by trenching) or both were excluded from plots of 4 m2. Subsequently, we measured soil respiration, soil temperature and soil moisture in each plot over a year after the forest was coppiced. The treatments did not significantly affect soil temperature or soil moisture measured over 0–10 cm depth.

Soil respiration varied markedly during the year with high rates in spring and autumn and low rates in summer, coinciding with summer drought, and in winter, with the lowest temperatures. Very high respiration rates, however, were observed during the summer immediately after rainfall events. The mean annual rate of soil respiration was 2.9 µmol m−2 s−1, ranging from 1.35 to 7.03 µmol m−2 s−1.

Soil respiration was highly correlated with temperature during winter and during spring and autumn whenever volumetric soil water content was above 20%. Below this threshold value, there was no correlation between soil respiration and soil temperature, but soil moisture was a good predictor of soil respiration. A simple empirical model that predicted soil respiration during the year, using both soil temperature and soil moisture accounted for more than 91% of the observed annual variation in soil respiration.

All the components of soil respiration followed a similar seasonal trend and were affected by summer drought. The Q10 value for soil respiration was 2.32, which is in agreement with other studies in forest ecosystems. However, we found a Q10 value for root respiration of 2.20, which is lower than recent values reported for forest sites. The fact that the seasonal variation in root growth with temperature in Mediterranean ecosystems differs from that in temperate regions may explain this difference. In temperate regions, increases in size of root populations during the growing season, coinciding with high temperatures, may yield higher apparent Q10 values than in Mediterranean regions where root growth is suppressed by summer drought.

The decomposition of organic matter and belowground litter were the major components of soil respiration, accounting for almost 55% of the total soil respiration flux. This proportion is higher than has been reported for mature boreal and temperate forest and is probably the result of a short-term C loss following recent logging at the site.

The relationship proposed for soil respiration with soil temperature and soil moisture is useful for understanding and predicting potential changes in Mediterranean forest ecosystems in response to forest management and climate change.

Annual variation in soil respiration and its components in a coppice oak forest in Central Italy

Vaccine adjuvants revisited

Evaluating binary classifications is a pivotal task in statistics and machine learning, because it can influence decisions in multiple areas, including for example prognosis or therapies of patients in critical conditions The scientific community has not agreed on a general-purpose statistical indicator for evaluating two-class confusion matrices (having true positives, true negatives, false positives, and false negatives) yet, even if advantages of the Matthews correlation coefficient (MCC) over accuracy and F1 score have already been shownIn this manuscript, we reaffirm that MCC is a robust metric that summarizes the classifier performance in a single value, if positive and negative cases are of equal importance We compare MCC to other metrics which value positive and negative cases equally: balanced accuracy (BA), bookmaker informedness (BM), and markedness (MK) We explain the mathematical relationships between MCC and these indicators, then show some use cases and a bioinformatics scenario where these metrics disagree and where MCC generates a more informative responseAdditionally, we describe three exceptions where BM can be more appropriate: analyzing classifications where dataset prevalence is unrepresentative, comparing classifiers on different datasets, and assessing the random guessing level of a classifier Except in these cases, we believe that MCC is the most informative among the single metrics discussed, and suggest it as standard measure for scientists of all fields A Matthews correlation coefficient close to +1, in fact, means having high values for all the other confusion matrix metrics The same cannot be said for balanced accuracy, markedness, bookmaker informedness, accuracy and F1 score

/pdf/the-matthews-correlation-coefficient-mcc-is-more-reliable-2otx9u016u.pdf

The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation

The impact of class imbalance in classification performance metrics based on the binary confusion matrix

Belowground structure and annual production in an open Mediterranean scrub were studied on a sandy substrate which had not been harvested or exposed to fire during the last 30–40 years. Estimated belowground biomass (1328 ± 93.4 g m-2) and production (548 ± 246.9 g m-2 yr-1) were lower than in other Mediterranean scrubs. However, the energy investment in belowground structures was high (root biomass/ shoot biomass = 2.7; annual belowground production/ annual aboveground production = 7.1), which is associated with a high density of fine roots in the top soil (the average distance between the fine roots = 0.76 cm). A very simple model based on nutrient diffusion was considered to analyse the resource constrains of the community. The results underline the importance of nutrients (and more specifically phosphorus) rather than water, as possible determinants of the structure and dynamics of the root system, as well as for the primary production of the community during the vegetative period.

Belowground structure and production in a Mediterranean sand dune shrub community

In response to epidemic levels of serogroup B meningococcal disease in Cuba during the 1980s, the VA-MENGOC-BC vaccine was developed and introduced into the National Infant Immunization Program in 1991. Since then the incidence of meningococcal disease in Cuba has returned to the low levels recorded before the epidemic. A total of 420 Neisseria meningitidis strains collected between 1983 and 2005 in Cuba were analyzed by multilocus sequence typing (MLST). The set of strains comprised 167 isolated from disease cases and 253 obtained from healthy carriers. By MLST analysis, 63 sequence types (STs) were identified, and 32 of these were reported to be a new ST. The Cuban isolates were associated with 12 clonal complexes; and the most common were ST-32 (246 isolates), ST-53 (86 isolates), and ST-41/44 (36 isolates). This study also showed that the application of VA-MENGOC-BC, the Cuban serogroup B and C vaccine, reduced the frequency and diversity of hypervirulent clonal complexes ST-32 (vaccine serogroup B type-strain) and ST-41/44 and also affected other lineages. Lineages ST-8 and ST-11 were no longer found during the postvaccination period. The vaccine also affected the genetic composition of the carrier-associated meningococcal isolates. The number of carrier isolates belonging to hypervirulent lineages decreased significantly after vaccination, and ST-53, a sequence type common in carriers, became the predominant ST.

Clonal Distribution of Disease-Associated and Healthy Carrier Isolates of Neisseria meningitidis between 1983 and 2005 in Cuba

Situación epidemiológica actual y resistencia de los patógenos respiratorios en Espan̄a

P64k is a meningococcal protein from Neisseria meningitidis that has been obtained by recombinant DNA technology. Recombinant P64k has been extensively characterized by physicochemical and immunological methods. Lately this protein has been found to act as a versatile immunological carrier for weak antigens in mice. In the present work, a Phase I clinical trial was carried out in healthy volunteers who received three inoculations of either placebo or recombinant P64k (20 or 50 microg). No severe adverse events occurred during the trial. Only mild adverse events in ten volunteers were observed. At 1 month after the third dose, 15 out of 18 volunteers (83.3%) who received the recombinant antigen had a P64k-specific antibody titre > or =1:100, as detected by ELISA. A fourth dose, given 9 months after the third one, elicited a potent booster immune response in P64k vaccinees. Accordingly, these P64k formulations were considered safe and immunogenic in healthy human volunteers.

Alejandro Martín

Papers

The impact of class imbalance in classification performance metrics based on the binary confusion matrix

Belowground structure and production in a Mediterranean sand dune shrub community

Clonal Distribution of Disease-Associated and Healthy Carrier Isolates of Neisseria meningitidis between 1983 and 2005 in Cuba

Situación epidemiológica actual y resistencia de los patógenos respiratorios en Espan̄a

Safety and preliminary immunogenicity of the recombinant outer membrane protein P64k of Neisseria meningitidis in human volunteers