To evaluate binary classifications and their confusion matrices, scientific researchers can employ several statistical rates, accordingly to the goal of the experiment they are investigating. Despite being a crucial issue in machine learning, no widespread consensus has been reached on a unified elective chosen measure yet. Accuracy and F1 score computed on confusion matrices have been (and still are) among the most popular adopted metrics in binary classification tasks. However, these statistical measures can dangerously show overoptimistic inflated results, especially on imbalanced datasets. The Matthews correlation coefficient (MCC), instead, is a more reliable statistical rate which produces a high score only if the prediction obtained good results in all of the four confusion matrix categories (true positives, false negatives, true negatives, and false positives), proportionally both to the size of positive elements and the size of negative elements in the dataset. In this article, we show how MCC produces a more informative and truthful score in evaluating binary classifications than accuracy and F1 score, by first explaining the mathematical properties, and then the asset of MCC in six synthetic use cases and in a real genomics scenario. We believe that the Matthews correlation coefficient should be preferred to accuracy and F1 score in evaluating binary classification tasks by all scientific communities.

/pdf/the-advantages-of-the-matthews-correlation-coefficient-mcc-hn3s2s0jsg.pdf

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

Classic IR (information retrieval) is inherently predicated on users searching for information, the so-called "information need". But the need behind a web search is often not informational -- it might be navigational (give me the url of the site I want to reach) or transactional (show me sites where I can perform a certain transaction, e.g. shop, download a file, or find a map). We explore this taxonomy of web searches and discuss how global search engines evolved to deal with web-specific needs.

/pdf/a-taxonomy-of-web-search-1c1wfauugs.pdf

A taxonomy of web search

Many real world data mining applications involve learning from imbalanced data sets. Learning from data sets that contain very few instances of the minority (or interesting) class usually produces biased classifiers that have a higher predictive accuracy over the majority class(es), but poorer predictive accuracy over the minority class. SMOTE (Synthetic Minority Over-sampling TEchnique) is specifically designed for learning from imbalanced data sets. This paper presents a novel approach for learning from imbalanced data sets, based on a combination of the SMOTE algorithm and the boosting procedure. Unlike standard boosting where all misclassified examples are given equal weights, SMOTEBoost creates synthetic examples from the rare or minority class, thus indirectly changing the updating weights and compensating for skewed distributions. SMOTEBoost applied to several highly and moderately imbalanced data sets shows improvement in prediction performance on the minority class and overall improved F-values.

/pdf/smoteboost-improving-prediction-of-the-minority-class-in-2fuvkmwltk.pdf

SMOTEBoost: Improving Prediction of the Minority Class in Boosting

Abbreviated Contents Figures and Tables Preface Introduction and Examples Concepts Relevant to Information Behavior Models, Paradigms, and Theories in the Study of Information Behavior Methods for Studying Information Behavior Research Results and Reflections Appendix: Glossary Appendix: Questions for Discussion and Application References Index

Looking for Information: A Survey of Research on Information Seeking, Needs and Behavior

No wonder you activities are, reading will be always needed. It is not only to fulfil the duties that you need to finish in deadline time. Reading will encourage your mind and thoughts. Of course, reading will greatly develop your experiences about everything. Reading invisible colleges diffusion of knowledge in scientific communities is also a way as one of the collective books that gives many advantages. The advantages are not only for you, but for the other peoples with those meaningful benefits.

Invisible colleges : diffusion of knowledge in scientific communities

Three meanings of “information” are distinguished: “Information-as-process”; “information-as-knowledge”; and “information-as-thing,” the attributive use of “information” to denote things regarded as informative. The nature and characteristics of “information-as-thing” are discussed, using an indirect approach (“What things are informative?”). Varieties of “information-as-thing” include data, text, documents, objects, and events. On this view “information” includes but extends beyond communication. Whatever information storage and retrieval systems store and retrieve is necessarily “information-as-thing.”

These three meanings of “information,” along with “information processing,” offer a basis for classifying disparate information-related activities (e.g., rhetoric, bibliographic retrieval, statistical analysis) and, thereby, suggest a topography for “information science.” © 1991 John Wiley & Sons, Inc.

Information as Thing

Empirical studies of retrieval performance have shown a tendency for Precision to decline as Recall increases. This article examines the nature of the relationship between Precision and Recall. The relationships between Recall and the number of documents retrieved, between Precision and the number of documents retrieved, and between Precision and Recall are described in the context of different assumptions about retrieval performance. It is demonstrated that a tradeoff between Recall and Precision is unavoidable whenever retrieval performance is consistently better than retrieval at random. More generally, for the Precision–Recall trade-off to be avoided as the total number of documents retrieved increases, retrieval performance must be equal to or better than overall retrieval performance up to that point. Examination of the mathematical relationship between Precision and Recall shows that a quadratic Recall curve can resemble empirical Recall–Precision behavior if transformed into a tangent parabola. With very large databases and/or systems with limited retrieval capabilities there can be advantages to retrieval in two stages: Initial retrieval emphasizing high Recall, followed by more detailed searching of the initially retrieved set, can be used to improve both Recall and Precision simultaneously. Even so, a tradeoff between Precision and Recall remains. © 1994 John Wiley & Sons, Inc.

The relationship between recall and precision

Ordinarily the word “document” denotes a textual record. Increasingly sophisticated attempts to provide access to the rapidly growing quantity of available documents raised questions about what should be considered a “document.” The answer is important for any definition of the scope of Information Science. Paul Otlet and others developed a functional view of “document” and discussed whether, for example, sculpture, museum objects, and live animals, could be considered “documents.” Suzanne Briet equated “document” with organized physical evidence. These ideas appear to resemble notions of “material culture” in cultural anthropology and “object-as-sign” in semiotics. Others, especially in the U.S.A. (e.g., Jesse Shera and Louis Shores) took a narrower view. New digital technology renews old questions and also old confusions between medium, message, and meaning. © 1997 John Wiley & Sons, Inc.

What is a “document”?

From the Publisher:
Buckland offers an examination of information systems that is comparative rather than narrowly technical in approach Using explicitly defined terms, he interprets the nature of retrieval-based information systems such as archives, libraries, databases, and museums, and their relationships to their social context Since the primary focus of the work is on unusual examples of information systems, the discussion yields interesting and thoughtful conclusions on the nature of information systems in general and constructive consideration of existing problems in this rapidly expanding field

Information and information systems

Feel lonely? What about reading books? Book is one of the greatest friends to accompany while in your lonely time. When you have no friends and activities somewhere and sometimes, reading book can be a great choice. This is not only for spending the time, it will increase the knowledge. Of course the b=benefits to take will relate to what kind of book that you are reading. And now, we will concern you to try reading annual review of information science and technology as one of the reading material to finish quickly.

Michael K. Buckland

Papers

Information as Thing

The relationship between recall and precision

What is a “document”?

Information and information systems

Annual Review of Information Science and Technology