Authorship identification remains a highly topical research problem in computational text analysis with many relevant applications in contemporary society and industry. For this edition of PAN, we focused on authorship verification, where the task is to assess whether a pair of documents has been authored by the same individual. Like in previous editions, we continued to work with (English-language) fanfiction, written by non-professional authors. As a novelty, we substantially increased the size of the provided dataset to enable more datahungry approaches. In total, thirteen systems (from ten participating teams) have been submitted, which are substantially more diverse than the submissions from previous years. We provide a detailed comparison of these approaches and two generic baselines. Our findings suggest that the increased scale of the training data boosts the state of the art in the field, but we also confirm the conventional issue that the field struggles with an overreliance on topic-related information.

/pdf/overview-of-the-cross-domain-authorship-verification-task-at-4kdqzw1p1r.pdf

Overview of the Cross-Domain Authorship Verification Task at PAN 2020.

Mortality prediction for intensive care unit (ICU) patients is crucial for improving outcomes and efficient utilization of resources. Accessibility of electronic health records (EHR) has enabled data-driven predictive modeling using machine learning. However, very few studies rely solely on unstructured clinical notes from the EHR for mortality prediction. In this work, we propose a framework to predict short, mid, and long-term mortality in adult ICU patients using unstructured clinical notes from the MIMIC III database, natural language processing (NLP), and machine learning (ML) models. Depending on the statistical description of the patients’ length of stay, we define the short-term as 48-hour and 4-day period, the mid-term as 7-day and 10-day period, and the long-term as 15-day and 30-day period after admission. We found that by only using clinical notes within the 24 hours of admission, our framework can achieve a high area under the receiver operating characteristics (AU-ROC) score for short, mid and long-term mortality prediction tasks. The test AU-ROC scores are 0.87, 0.83, 0.83, 0.82, 0.82, and 0.82 for 48-hour, 4-day, 7-day, 10-day, 15-day, and 30-day period mortality prediction, respectively. We also provide a comparative study among three types of feature extraction techniques from NLP: frequency-based technique, fixed embedding-based technique, and dynamic embedding-based technique. Lastly, we provide an interpretation of the NLP-based predictive models using feature-importance scores.

/pdf/unstructured-clinical-notes-within-the-24-hours-since-2zhtst50.pdf

Unstructured clinical notes within the 24 hours since admission predict short, mid & long-term mortality in adult ICU patients

The article explores approaches to determining the author of a natural language text and the advantages and disadvantages of these approaches. The importance of the considered problem is due to the active digitalization of society and reassignment of most parts of the life activities online. Text authorship methods are particularly useful for information security and forensics. For example, such methods can be used to identify authors of suicide notes, and other texts are subjected to forensic examinations. Another area of application is plagiarism detection. Plagiarism detection is a relevant issue both for the field of intellectual property protection in the digital space and for the educational process. The article describes identifying the author of the Russian-language text using support vector machine (SVM) and deep neural network architectures (long short-term memory (LSTM), convolutional neural networks (CNN) with attention, Transformer). The results show that all the considered algorithms are suitable for solving the authorship identification problem, but SVM shows the best accuracy. The average accuracy of SVM reaches 96%. This is due to thoroughly chosen parameters and feature space, which includes statistical and semantic features (including those extracted as a result of an aspect analysis). Deep neural networks are inferior to SVM in accuracy and reach only 93%. The study also includes an evaluation of the impact of attacks on the method on models’ accuracy. Experiments show that the SVM-based methods are unstable to deliberate text anonymization. In comparison, the loss in accuracy of deep neural networks does not exceed 20%. Transformer architecture is the most effective for anonymized texts and allows 81% accuracy to be achieved.

Authorship Identification of a Russian-Language Text Using Support Vector Machine and Deep Neural Networks

Authorship attribution attempts to identify the authors behind texts and has important applications mainly in digital forensics, cyber-security, digital humanities, and social media analytics. A challenging yet realistic scenario is cross-domain attribution where texts of known authorship (training set) differ from texts of disputed authorship (test set) in, for example, topic or genre. In this paper, we propose the use of transfer learning based on pre-trained neural network language models and a multi-headed classifier. A series of experiments is reported to compare the effectiveness of our approach on cross-topic, cross-genre, and cross-fandom conditions with state-of-the-art methods. We also demonstrate the crucial effect of the normalization corpus (an unlabeled corpus used to adjust the output of our classifier) in cross-domain attribution and the usefulness of shallower layers of pre-trained models.

A transfer learning approach to cross-domain authorship attribution

In computational authorship attribution (AA) – the task of identifying the author of a given text based on a set of possible candidates – existing differences across domains, languages or input settings may require using knowledge from multiple sources, ranging from surface character patterns to deeper semantics and others. Moreover, since increasing the model complexity may easily lead to overfitting, sources of this kind have to be selected judiciously according to each particular input. Based on these observations, this article introduces a novel approach to AA consisting of stacked classifiers built from multiple knowledge sources - words, characters, part-of-speech n-grams, syntactic dependencies, word embeddings and more - that are dynamically included in the AA model according to the relevant input. In doing so, we would like to show that a stacking approach not only outperforms previous work in the field, but also that dynamic model selection outperforms the use of any of the individual components alone. The current model - called DynAA - is evaluated in a number of AA scenarios covering multiple languages, domains and input sizes, and is shown to generally outperform a number of baseline alternatives, including convolutional neural networks, BERT and others.

Stacked authorship attribution of digital texts

Authorship attribution attempts to identify the authors behind texts and has important applications mainly in cyber-security, digital humanities and social media analytics. An especially challenging but very realistic scenario is cross-domain attribution where texts of known authorship (training set) differ from texts of disputed authorship (test set) in topic or genre. In this paper, we modify a successful authorship verification approach based on a multi-headed neural network language model and combine it with pre-trained language models. Based on experiments on a controlled corpus covering several text genres where topic and genre is specifically controlled, we demonstrate that the proposed approach achieves very promising results. We also demonstrate the crucial effect of the normalization corpus in cross-domain attribution.

/pdf/cross-domain-authorship-attribution-using-pre-trained-3t07l34o3f.pdf

Georgios Barlas

Papers

Cross-Domain Authorship Attribution Using Pre-trained Language Models

A transfer learning approach to cross-domain authorship attribution