scispace - formally typeset
Search or ask a question

Showing papers in "Entropy in 2020"


Journal ArticleDOI
25 Dec 2020-Entropy
TL;DR: In this paper, a literature review and taxonomy of machine learning interpretability methods are presented, as well as links to their programming implementations, in the hope that this survey would serve as a reference point for both theorists and practitioners.
Abstract: Recent advances in artificial intelligence (AI) have led to its widespread industrial adoption, with machine learning systems demonstrating superhuman performance in a significant number of tasks. However, this surge in performance, has often been achieved through increased model complexity, turning such systems into “black box” approaches and causing uncertainty regarding the way they operate and, ultimately, the way that they come to decisions. This ambiguity has made it problematic for machine learning systems to be adopted in sensitive yet critical domains, where their value could be immense, such as healthcare. As a result, scientific interest in the field of Explainable Artificial Intelligence (XAI), a field that is concerned with the development of new methods that explain and interpret machine learning models, has been tremendously reignited over recent years. This study focuses on machine learning interpretability methods; more specifically, a literature review and taxonomy of these methods are presented, as well as links to their programming implementations, in the hope that this survey would serve as a reference point for both theorists and practitioners.

543 citations


Journal ArticleDOI
30 Jul 2020-Entropy
TL;DR: In this paper, the authors used decision tree, bagging, random forest, adaptive boosting (Adaboost), gradient boosting, and eXtreme gradient boosting (XGBoost), and artificial neural networks (ANN), recurrent neural network (RNN) and long short-term memory (LSTM).
Abstract: The prediction of stock groups values has always been attractive and challenging for shareholders due to its inherent dynamics, non-linearity, and complex nature. This paper concentrates on the future prediction of stock market groups. Four groups named diversified financials, petroleum, non-metallic minerals, and basic metals from Tehran stock exchange were chosen for experimental evaluations. Data were collected for the groups based on 10 years of historical records. The value predictions are created for 1, 2, 5, 10, 15, 20, and 30 days in advance. Various machine learning algorithms were utilized for prediction of future values of stock market groups. We employed decision tree, bagging, random forest, adaptive boosting (Adaboost), gradient boosting, and eXtreme gradient boosting (XGBoost), and artificial neural networks (ANN), recurrent neural network (RNN) and long short-term memory (LSTM). Ten technical indicators were selected as the inputs into each of the prediction models. Finally, the results of the predictions were presented for each technique based on four metrics. Among all algorithms used in this paper, LSTM shows more accurate results with the highest model fitting ability. In addition, for tree-based models, there is often an intense competition between Adaboost, Gradient Boosting, and XGBoost.

154 citations


Journal ArticleDOI
01 May 2020-Entropy
TL;DR: This study presents the combination of deep learning of extracted features with the Q-deformed entropy handcrafted features for discriminating between COVID-19 coronavirus, pneumonia and healthy computed tomography (CT) lung scans.
Abstract: Many health systems over the world have collapsed due to limited capacity and a dramatic increase of suspected COVID-19 cases. What has emerged is the need for finding an efficient, quick and accurate method to mitigate the overloading of radiologists' efforts to diagnose the suspected cases. This study presents the combination of deep learning of extracted features with the Q-deformed entropy handcrafted features for discriminating between COVID-19 coronavirus, pneumonia and healthy computed tomography (CT) lung scans. In this study, pre-processing is used to reduce the effect of intensity variations between CT slices. Then histogram thresholding is used to isolate the background of the CT lung scan. Each CT lung scan undergoes a feature extraction which involves deep learning and a Q-deformed entropy algorithm. The obtained features are classified using a long short-term memory (LSTM) neural network classifier. Subsequently, combining all extracted features significantly improves the performance of the LSTM network to precisely discriminate between COVID-19, pneumonia and healthy cases. The maximum achieved accuracy for classifying the collected dataset comprising 321 patients is 99.68%.

110 citations


Journal ArticleDOI
24 Feb 2020-Entropy
TL;DR: This paper combines entropy weight, analytic hierarchy process (AHP) weight, and the technique for order preference by similarity to an ideal solution (TOPSIS) method into a suitable multi-criteria decision making (MCDM) solution.
Abstract: The type of criterion weight can be distinguished according to different decision methods. Subjective weights are given by decision makers based on their knowledge, experience, expertise, and other factors. Objective weights are obtained through multi-step calculations of the evaluation matrix constructed from the actual information about the evaluation criteria of the alternatives. A single consideration of these two types of weights often results in biased results. In addition, in order to build an effective supply chain source, buyers must find suitable quality products and/or service providers in the process of supplier selection. Based on the above reasons, it is difficult to accurately select the appropriate alternative. The main contribution of this paper is to combine entropy weight, analytic hierarchy process (AHP) weight, and the technique for order preference by similarity to an ideal solution (TOPSIS) method into a suitable multi-criteria decision making (MCDM) solution. The TOPSIS method is extended with entropy-AHP weights, and entropy-AHP weights are used instead of subjective weights. A novel decision-making model of TOPSIS integrated entropy-AHP weights is proposed to select the appropriate supplier. Finally, we take the selection of building material suppliers as an example and use sensitivity analysis to show that the combination of the TOPSIS method based on entropy-AHP weights can effectively select the appropriate supplier.

106 citations


Journal ArticleDOI
24 Sep 2020-Entropy
TL;DR: The present analysis deals with the entropy analysis of the blood flow through an anisotropically tapered arteries under the suspension of magnetic Zinc-oxide (ZnO) nanoparticles (NPs) and the present outcomes are enriched to give valuable information for the research scientists in the field biomedical science.
Abstract: The present analysis deals with the entropy analysis of the blood flow through an anisotropically tapered arteries under the suspension of magnetic Zinc-oxide (ZnO) nanoparticles (NPs). The Jeffrey fluid model is contemplated as blood that is electrically conducting and incompressible. The lubrication approach is used for the mathematical modeling. The second law of thermodynamics is used to examine the entropy generation. The exact solutions are obtained against velocity and temperature profile with the use of computational software. The results for Entropy, Velocity, Bejan number, temperature profile, and impedance profile are discussed by plotting the graphs. ZnO-NPs have promising applications in biomedical engineering due to its low toxicity, economically reliable, and excellent biocompatibility. ZnO-NPs also emerged in medicine i.e., antibacterial and anticancer activity, and also beneficial in antidiabetic treatment. The monitoring of the blood temperature in the case of the tapered artery has supreme importance in controlling the temperature of blood in the living environment. The presence of a magnetic field is advantageous to manage and control the blood motion at different temperatures. The present outcomes are enriched to give valuable information for the research scientists in the field biomedical science, who are looking to examine the blood flow with stenosis conditions and also beneficial in treating multiple diseases.

103 citations


Journal ArticleDOI
21 Mar 2020-Entropy
TL;DR: A systematic literature review about variants and improvements of the Particle Swarm Optimisation algorithm and its extensions to other classes of optimisation problems, taking into consideration the most important ones is made.
Abstract: The Particle Swarm Optimisation (PSO) algorithm was inspired by the social and biological behaviour of bird flocks searching for food sources. In this nature-based algorithm, individuals are referred to as particles and fly through the search space seeking for the global best position that minimises (or maximises) a given problem. Today, PSO is one of the most well-known and widely used swarm intelligence algorithms and metaheuristic techniques, because of its simplicity and ability to be used in a wide range of applications. However, in-depth studies of the algorithm have led to the detection and identification of a number of problems with it, especially convergence problems and performance issues. Consequently, a myriad of variants, enhancements and extensions to the original version of the algorithm, developed and introduced in the mid-1990s, have been proposed, especially in the last two decades. In this article, a systematic literature review about those variants and improvements is made, which also covers the hybridisation and parallelisation of the algorithm and its extensions to other classes of optimisation problems, taking into consideration the most important ones. These approaches and improvements are appropriately summarised, organised and presented, in order to allow and facilitate the identification of the most appropriate PSO variant for a particular application.

94 citations


Journal ArticleDOI
23 Apr 2020-Entropy
TL;DR: A novel procedure is proposed and justify a novel procedure that should be used in adjusting the evaluation metrics for imbalanced datasets that are common for different kinds of skin lesions.
Abstract: In this paper, a new Computer-Aided Detection (CAD) system for the detection and classification of dangerous skin lesions (melanoma type) is presented, through a fusion of handcraft features related to the medical algorithm ABCD rule (Asymmetry Borders-Colors-Dermatoscopic Structures) and deep learning features employing Mutual Information (MI) measurements. The steps of a CAD system can be summarized as preprocessing, feature extraction, feature fusion, and classification. During the preprocessing step, a lesion image is enhanced, filtered, and segmented, with the aim to obtain the Region of Interest (ROI); in the next step, the feature extraction is performed. Handcraft features such as shape, color, and texture are used as the representation of the ABCD rule, and deep learning features are extracted using a Convolutional Neural Network (CNN) architecture, which is pre-trained on Imagenet (an ILSVRC Imagenet task). MI measurement is used as a fusion rule, gathering the most important information from both types of features. Finally, at the Classification step, several methods are employed such as Linear Regression (LR), Support Vector Machines (SVMs), and Relevant Vector Machines (RVMs). The designed framework was tested using the ISIC 2018 public dataset. The proposed framework appears to demonstrate an improved performance in comparison with other state-of-the-art methods in terms of the accuracy, specificity, and sensibility obtained in the training and test stages. Additionally, we propose and justify a novel procedure that should be used in adjusting the evaluation metrics for imbalanced datasets that are common for different kinds of skin lesions.

94 citations


Journal ArticleDOI
30 Apr 2020-Entropy
TL;DR: A key distinction in the physical sciences that may provide a foundation for the distinction between mind and matter, and between sentient and intentional systems is described.
Abstract: This essay addresses Cartesian duality and how its implicit dialectic might be repaired using physics and information theory. Our agenda is to describe a key distinction in the physical sciences that may provide a foundation for the distinction between mind and matter, and between sentient and intentional systems. From this perspective, it becomes tenable to talk about the physics of sentience and 'forces' that underwrite our beliefs (in the sense of probability distributions represented by our internal states), which may ground our mental states and consciousness. We will refer to this view as Markovian monism, which entails two claims: (1) fundamentally, there is only one type of thing and only one type of irreducible property (hence monism). (2) All systems possessing a Markov blanket have properties that are relevant for understanding the mind and consciousness: if such systems have mental properties, then they have them partly by virtue of possessing a Markov blanket (hence Markovian). Markovian monism rests upon the information geometry of random dynamic systems. In brief, the information geometry induced in any system-whose internal states can be distinguished from external states-must acquire a dual aspect. This dual aspect concerns the (intrinsic) information geometry of the probabilistic evolution of internal states and a separate (extrinsic) information geometry of probabilistic beliefs about external states that are parameterised by internal states. We call these intrinsic (i.e., mechanical, or state-based) and extrinsic (i.e., Markovian, or belief-based) information geometries, respectively. Although these mathematical notions may sound complicated, they are fairly straightforward to handle, and may offer a means through which to frame the origins of consciousness.

92 citations


Journal ArticleDOI
28 Feb 2020-Entropy
TL;DR: This paper proposes a novel system that is computationally less expensive and provided a higher level of security in chaotic-based encryption schemes based on a shuffling process with fractals key along with three-dimensional Lorenz chaotic map.
Abstract: Chaos-based encryption schemes have attracted many researchers around the world in the digital image security domain. Digital images can be secured using existing chaotic maps, multiple chaotic maps, and several other hybrid dynamic systems that enhance the non-linearity of digital images. The combined property of confusion and diffusion was introduced by Claude Shannon which can be employed for digital image security. In this paper, we proposed a novel system that is computationally less expensive and provided a higher level of security. The system is based on a shuffling process with fractals key along with three-dimensional Lorenz chaotic map. The shuffling process added the confusion property and the pixels of the standard image is shuffled. Three-dimensional Lorenz chaotic map is used for a diffusion process which distorted all pixels of the image. In the statistical security test, means square error (MSE) evaluated error value was greater than the average value of 10000 for all standard images. The value of peak signal to noise (PSNR) was 7.69(dB) for the test image. Moreover, the calculated correlation coefficient values for each direction of the encrypted images was less than zero with a number of pixel change rate (NPCR) higher than 99%. During the security test, the entropy values were more than 7.9 for each grey channel which is almost equal to the ideal value of 8 for an 8-bit system. Numerous security tests and low computational complexity tests validate the security, robustness, and real-time implementation of the presented scheme.

89 citations


Journal ArticleDOI
04 Feb 2020-Entropy
TL;DR: This work proposes a permissioned private blockchain-based solution to secure the image while encrypting it, ensuring the privacy and security of the image data on the blockchain.
Abstract: Smart cameras and image sensors are widely used in industrial processes, from the designing to the quality checking of the final product. Images generated by these sensors are at continuous risk of disclosure and privacy breach in the industrial Internet of Things (IIoT). Traditional solutions to secure sensitive data fade in IIoT environments because of the involvement of third parties. Blockchain technology is the modern-day solution for trust issues and eliminating or minimizing the role of the third party. In the context of the IIoT, we propose a permissioned private blockchain-based solution to secure the image while encrypting it. In this scheme, the cryptographic pixel values of an image are stored on the blockchain, ensuring the privacy and security of the image data. Based on the number of pixels change rate (NPCR), the unified averaged changed intensity (UACI), and information entropy analysis, we evaluate the strength of proposed image encryption algorithm ciphers with respect to differential attacks. We obtained entropy values near to an ideal value of 8, which is considered to be safe from brute force attack. Encrypted results show that the proposed scheme is highly effective for data leakage prevention and security.

82 citations


Journal ArticleDOI
27 Jan 2020-Entropy
TL;DR: This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects.
Abstract: This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate connections of this setting to remote source-coding under logarithmic loss distortion measure, information combining, common reconstruction, the Wyner–Ahlswede–Korner problem, the efficiency of investment information, as well as, generalization, variational inference, representation learning, autoencoders, and others are highlighted. We discuss its extension to the distributed information bottleneck problem with emphasis on the Gaussian model and highlight the basic connections to the uplink Cloud Radio Access Networks (CRAN) with oblivious processing. For this model, the optimal trade-offs between relevance (i.e., information) and complexity (i.e., rates) in the discrete and vector Gaussian frameworks is determined. In the concluding outlook, some interesting problems are mentioned such as the characterization of the optimal inputs (“features”) distributions under power limitations maximizing the “relevance” for the Gaussian information bottleneck, under “complexity” constraints.

Journal ArticleDOI
13 Jan 2020-Entropy
TL;DR: A novel Blockchain-driven design to reshape the business logic of SCF to develop an efficient and reliable financing platform for small and medium enterprises (SMEs) in the auto retail industry to decrease the cost of financing and speed up the cash flows is proposed.
Abstract: In this paper, a Blockchain-driven platform for supply chain finance, BCautoSCF (Zhi-lian-che-rong in Chinese), is introduced. It is successfully established as a reliable and efficient financing platform for the auto retail industry. Due to the Blockchain built-in trust mechanism, participants in the supply chain (SC) networks work extensively and transparently to run a reliable, convenient, and traceable business. Likewise, the traditional supply chain finance (SCF), partial automation of SCF workflows with fewer human errors and disruptions was achieved through smart contract in BCautoSCF. Such open and secure features suggest the feasibility of BCautoSCF in SCF. As the first Blockchain-driven SCF application for the auto retail industry in China, our contribution lies in studying these pain points existing in traditional SCF and proposing a novel Blockchain-driven design to reshape the business logic of SCF to develop an efficient and reliable financing platform for small and medium enterprises (SMEs) in the auto retail industry to decrease the cost of financing and speed up the cash flows. Currently, there are over 600 active enterprise users that adopt BCautoSCF to run their financing business. Up to October 2019, the BCautoSCF provides services to 449 online/offline auto retailors, three B2B asset exchange platforms, nine fund providers, and 78 logistic services across 21 provinces in China. There are 3296 financing transactions successfully completed in BCautoSCF, and the amount of financing is ¥566,784,802.18. In the future, we will work towards supporting a full automation of SCF workflow by smart contracts, so that the efficiency of transaction will be further improved.

Journal ArticleDOI
Chungu Guo, Liangwei Yang, Xiao Chen, Duanbing Chen, Hui Gao, Jing Ma1 
21 Feb 2020-Entropy
TL;DR: The proposed EnRenew algorithm measures the importance of nodes based on information entropy and selects a group of important nodes through dynamic update strategy and shed light on new method of node mining in complex networks for information spreading and epidemic prevention.
Abstract: Identifying a set of influential nodes is an important topic in complex networks which plays a crucial role in many applications, such as market advertising, rumor controlling, and predicting valuable scientific publications. In regard to this, researchers have developed algorithms from simple degree methods to all kinds of sophisticated approaches. However, a more robust and practical algorithm is required for the task. In this paper, we propose the EnRenew algorithm aimed to identify a set of influential nodes via information entropy. Firstly, the information entropy of each node is calculated as initial spreading ability. Then, select the node with the largest information entropy and renovate its l-length reachable nodes' spreading ability by an attenuation factor, repeat this process until specific number of influential nodes are selected. Compared with the best state-of-the-art benchmark methods, the performance of proposed algorithm improved by 21.1%, 7.0%, 30.0%, 5.0%, 2.5%, and 9.0% in final affected scale on CEnew, Email, Hamster, Router, Condmat, and Amazon network, respectively, under the Susceptible-Infected-Recovered (SIR) simulation model. The proposed algorithm measures the importance of nodes based on information entropy and selects a group of important nodes through dynamic update strategy. The impressive results on the SIR simulation model shed light on new method of node mining in complex networks for information spreading and epidemic prevention.

Journal ArticleDOI
26 Jul 2020-Entropy
TL;DR: A novel features extraction method which incorporates robust entropy optimization and an efficient Maximum Entropy Markov Model (MEMM) for HIR via multiple vision sensors is proposed, which will be applicable to a wide variety of man–machine interfaces.
Abstract: Automatic identification of human interaction is a challenging task especially in dynamic environments with cluttered backgrounds from video sequences. Advancements in computer vision sensor technologies provide powerful effects in human interaction recognition (HIR) during routine daily life. In this paper, we propose a novel features extraction method which incorporates robust entropy optimization and an efficient Maximum Entropy Markov Model (MEMM) for HIR via multiple vision sensors. The main objectives of proposed methodology are: (1) to propose a hybrid of four novel features—i.e., spatio-temporal features, energy-based features, shape based angular and geometric features—and a motion-orthogonal histogram of oriented gradient (MO-HOG); (2) to encode hybrid feature descriptors using a codebook, a Gaussian mixture model (GMM) and fisher encoding; (3) to optimize the encoded feature using a cross entropy optimization function; (4) to apply a MEMM classification algorithm to examine empirical expectations and highest entropy, which measure pattern variances to achieve outperformed HIR accuracy results. Our system is tested over three well-known datasets: SBU Kinect interaction; UoL 3D social activity; UT-interaction datasets. Through wide experimentations, the proposed features extraction algorithm, along with cross entropy optimization, has achieved the average accuracy rate of 91.25% with SBU, 90.4% with UoL and 87.4% with UT-Interaction datasets. The proposed HIR system will be applicable to a wide variety of man–machine interfaces, such as public-place surveillance, future medical applications, virtual reality, fitness exercises and 3D interactive gaming.

Journal ArticleDOI
20 May 2020-Entropy
TL;DR: A human activity recognition model that acquires signal data from motion node sensors including inertial sensors, i.e., gyroscopes and accelerometers is presented, which outperformed existing well-known statistical state-of-the-art methods by achieving an improved recognition accuracy.
Abstract: Advancements in wearable sensors technologies provide prominent effects in the daily life activities of humans. These wearable sensors are gaining more awareness in healthcare for the elderly to ensure their independent living and to improve their comfort. In this paper, we present a human activity recognition model that acquires signal data from motion node sensors including inertial sensors, i.e., gyroscopes and accelerometers. First, the inertial data is processed via multiple filters such as Savitzky-Golay, median and hampel filters to examine lower/upper cutoff frequency behaviors. Second, it extracts a multifused model for statistical, wavelet and binary features to maximize the occurrence of optimal feature values. Then, adaptive moment estimation (Adam) and AdaDelta are introduced in a feature optimization phase to adopt learning rate patterns. These optimized patterns are further processed by the maximum entropy Markov model (MEMM) for empirical expectation and highest entropy, which measure signal variances for outperformed accuracy results. Our model was experimentally evaluated on University of Southern California Human Activity Dataset (USC-HAD) as a benchmark dataset and on an Intelligent Mediasporting behavior (IMSB), which is a new self-annotated sports dataset. For evaluation, we used the "leave-one-out" cross validation scheme and the results outperformed existing well-known statistical state-of-the-art methods by achieving an improved recognition accuracy of 91.25%, 93.66% and 90.91% when compared with USC-HAD, IMSB, and Mhealth datasets, respectively. The proposed system should be applicable to man-machine interface domains, such as health exercises, robot learning, interactive games and pattern-based surveillance.

Journal ArticleDOI
13 Aug 2020-Entropy
TL;DR: It is argued that the model of adaptive phenotypes under the free-energy principle can be used to furnish a formal semantics, enabling us to assign semantic content to specific phenotypic states (the internal states of a Markovian system that exists far from equilibrium).
Abstract: The aim of this paper is twofold: (1) to assess whether the construct of neural representations plays an explanatory role under the variational free-energy principle and its corollary process theory, active inference; and (2) if so, to assess which philosophical stance-in relation to the ontological and epistemological status of representations-is most appropriate. We focus on non-realist (deflationary and fictionalist-instrumentalist) approaches. We consider a deflationary account of mental representation, according to which the explanatorily relevant contents of neural representations are mathematical, rather than cognitive, and a fictionalist or instrumentalist account, according to which representations are scientifically useful fictions that serve explanatory (and other) aims. After reviewing the free-energy principle and active inference, we argue that the model of adaptive phenotypes under the free-energy principle can be used to furnish a formal semantics, enabling us to assign semantic content to specific phenotypic states (the internal states of a Markovian system that exists far from equilibrium). We propose a modified fictionalist account-an organism-centered fictionalism or instrumentalism. We argue that, under the free-energy principle, pursuing even a deflationary account of the content of neural representations licenses the appeal to the kind of semantic content involved in the 'aboutness' or intentionality of cognitive systems; our position is thus coherent with, but rests on distinct assumptions from, the realist position. We argue that the free-energy principle thereby explains the aboutness or intentionality in living systems and hence their capacity to parse their sensory stream using an ontology or set of semantic factors.

Journal ArticleDOI
07 May 2020-Entropy
TL;DR: It is shown that some of these indices, like those derived from recurrence plots, from entropy or fractal analysis, can detect MMF efficiently and that more work remains to be done to compare the complexity indices in terms of reliability and sensibility.
Abstract: The surface electromyography (sEMG) records the electrical activity of muscle fibers during contraction: one of its uses is to assess changes taking place within muscles in the course of a fatiguing contraction to provide insights into our understanding of muscle fatigue in training protocols and rehabilitation medicine. Until recently, these myoelectric manifestations of muscle fatigue (MMF) have been assessed essentially by linear sEMG analyses. However, sEMG shows a complex behavior, due to many concurrent factors. Therefore, in the last years, complexity-based methods have been tentatively applied to the sEMG signal to better individuate the MMF onset during sustained contractions. In this review, after describing concisely the traditional linear methods employed to assess MMF we present the complexity methods used for sEMG analysis based on an extensive literature search. We show that some of these indices, like those derived from recurrence plots, from entropy or fractal analysis, can detect MMF efficiently. However, we also show that more work remains to be done to compare the complexity indices in terms of reliability and sensibility; to optimize the choice of embedding dimension, time delay and threshold distance in reconstructing the phase space; and to elucidate the relationship between complexity estimators and the physiologic phenomena underlying the onset of MMF in exercising muscles.

Journal ArticleDOI
09 Mar 2020-Entropy
TL;DR: This paper reviews the nonlinear methods most used to assess heart-rate dynamics based on concepts of chaos, fractality, and complexity: Poincaré plot, recurrence plot analysis, fractal dimension, detrended fluctuation analysis, Hurst exponent, Lyapunov exponent entropies, and symbolic dynamics.
Abstract: The heart-rate dynamics are one of the most analyzed physiological interactions. Many mathematical methods were proposed to evaluate heart-rate variability. These methods have been successfully applied in research to expand knowledge concerning the cardiovascular dynamics in healthy as well as in pathological conditions. Notwithstanding, they are still far from clinical practice. In this paper, we aim to review the nonlinear methods most used to assess heart-rate dynamics. We focused on methods based on concepts of chaos, fractality, and complexity: Poincare plot, recurrence plot analysis, fractal dimension (and the correlation dimension), detrended fluctuation analysis, Hurst exponent, Lyapunov exponent entropies (Shannon, conditional, approximate, sample entropy, and multiscale entropy), and symbolic dynamics. We present the description of the methods along with their most notable applications.

Journal ArticleDOI
18 Aug 2020-Entropy
TL;DR: The first paper published as Finite-Time Thermodynamics is from 1977, and the first book on finite-time thermodynamics is published in 2016.
Abstract: The first paper published as Finite-Time Thermodynamics is from 1977 [...].

Journal ArticleDOI
Ian Fischer1
08 Sep 2020-Entropy
TL;DR: This article proposed the Minimum Necessary Information (MNI) criterion for evaluating the quality of a model, which extends the traditional measure of generalization as accuracy or related metrics on a held-out set.
Abstract: Much of the field of Machine Learning exhibits a prominent set of failure modes, including vulnerability to adversarial examples, poor out-of-distribution (OoD) detection, miscalibration, and willingness to memorize random labelings of datasets. We characterize these as failures of robust generalization, which extends the traditional measure of generalization as accuracy or related metrics on a held-out set. We hypothesize that these failures to robustly generalize are due to the learning systems retaining too much information about the training data. To test this hypothesis, we propose the Minimum Necessary Information (MNI) criterion for evaluating the quality of a model. In order to train models that perform well with respect to the MNI criterion, we present a new objective function, the Conditional Entropy Bottleneck (CEB), which is closely related to the Information Bottleneck (IB). We experimentally test our hypothesis by comparing the performance of CEB models with deterministic models and Variational Information Bottleneck (VIB) models on a variety of different datasets and robustness challenges. We find strong empirical evidence supporting our hypothesis that MNI models improve on these problems of robust generalization.

Journal ArticleDOI
24 Jan 2020-Entropy
TL;DR: The extensive experimental results indicated that the proposed CEEMD-XGBoost can significantly enhance the detection performance of epileptic seizures in terms of sensitivity, specificity, and accuracy.
Abstract: Epilepsy is a common nervous system disease that is characterized by recurrent seizures. An electroencephalogram (EEG) records neural activity, and it is commonly used for the diagnosis of epilepsy. To achieve accurate detection of epileptic seizures, an automatic detection approach of epileptic seizures, integrating complementary ensemble empirical mode decomposition (CEEMD) and extreme gradient boosting (XGBoost), named CEEMD-XGBoost, is proposed. Firstly, the decomposition method, CEEMD, which is capable of effectively reducing the influence of mode mixing and end effects, was utilized to divide raw EEG signals into a set of intrinsic mode functions (IMFs) and residues. Secondly, the multi-domain features were extracted from raw signals and the decomposed components, and they were further selected according to the importance scores of the extracted features. Finally, XGBoost was applied to develop the epileptic seizure detection model. Experiments were conducted on two benchmark epilepsy EEG datasets, named the Bonn dataset and the CHB-MIT (Children's Hospital Boston and Massachusetts Institute of Technology) dataset, to evaluate the performance of our proposed CEEMD-XGBoost. The extensive experimental results indicated that, compared with some previous EEG classification models, CEEMD-XGBoost can significantly enhance the detection performance of epileptic seizures in terms of sensitivity, specificity, and accuracy.

Journal ArticleDOI
23 Apr 2020-Entropy
TL;DR: An interdependent network model, where individuals in each layer follow different evolutionary games, and where each player is considered as a mobile agent that can move locally inside its own layer to improve its fitness is considered.
Abstract: Evolutionary game theory in the realm of network science appeals to a lot of research communities, as it constitutes a popular theoretical framework for studying the evolution of cooperation in social dilemmas. Recent research has shown that cooperation is markedly more resistant in interdependent networks, where traditional network reciprocity can be further enhanced due to various forms of interdependence between different network layers. However, the role of mobility in interdependent networks is yet to gain its well-deserved attention. Here we consider an interdependent network model, where individuals in each layer follow different evolutionary games, and where each player is considered as a mobile agent that can move locally inside its own layer to improve its fitness. Probabilistically, we also consider an imitation possibility from a neighbor on the other layer. We show that, by considering migration and stochastic imitation, further fascinating gateways to cooperation on interdependent networks can be observed. Notably, cooperation can be promoted on both layers, even if cooperation without interdependence would be improbable on one of the layers due to adverse conditions. Our results provide a rationale for engineering better social systems at the interface of networks and human decision making under testing dilemmas.

Journal ArticleDOI
18 Sep 2020-Entropy
TL;DR: In this paper, the authors studied the dynamics of the cryptocurrency exchange rates to other cryptocurrencies and fiat currencies and analyzed the coupling and decoupling of the cryptocurrencies and the conventional assets with fractal analysis.
Abstract: Social systems are characterized by an enormous network of connections and factors that can influence the structure and dynamics of these systems. Among them the whole economical sphere of human activity seems to be the most interrelated and complex. All financial markets, including the youngest one, the cryptocurrency market, belong to this sphere. The complexity of the cryptocurrency market can be studied from different perspectives. First, the dynamics of the cryptocurrency exchange rates to other cryptocurrencies and fiat currencies can be studied and quantified by means of multifractal formalism. Second, coupling and decoupling of the cryptocurrencies and the conventional assets can be investigated with the advanced cross-correlation analyses based on fractal analysis. Third, an internal structure of the cryptocurrency market can also be a subject of analysis that exploits, for example, a network representation of the market. In this work, we approach the subject from all three perspectives based on data from a recent time interval between January 2019 and June 2020. This period includes the peculiar time of the Covid-19 pandemic; therefore, we pay particular attention to this event and investigate how strong its impact on the structure and dynamics of the market was. Besides, the studied data covers a few other significant events like double bull and bear phases in 2019. We show that, throughout the considered interval, the exchange rate returns were multifractal with intermittent signatures of bifractality that can be associated with the most volatile periods of the market dynamics like a bull market onset in April 2019 and the Covid-19 outburst in March 2020. The topology of a minimal spanning tree representation of the market also used to alter during these events from a distributed type without any dominant node to a highly centralized type with a dominating hub of USDT. However, the MST topology during the pandemic differs in some details from other volatile periods.

Journal ArticleDOI
21 Aug 2020-Entropy
TL;DR: The present study identifies that the complexity, as quantified by entropy, is a fundamental property of conscious experience, which also plays a vital role in the brain’s capacity for adaptation and therefore whose loss by ageing constitutes a basis for diseases and disorders.
Abstract: Entropy is a powerful tool for quantification of the brain function and its information processing capacity. This is evident in its broad domain of applications that range from functional interactivity between the brain regions to quantification of the state of consciousness. A number of previous reviews summarized the use of entropic measures in neuroscience. However, these studies either focused on the overall use of nonlinear analytical methodologies for quantification of the brain activity or their contents pertained to a particular area of neuroscientific research. The present study aims at complementing these previous reviews in two ways. First, by covering the literature that specifically makes use of entropy for studying the brain function. Second, by highlighting the three fields of research in which the use of entropy has yielded highly promising results: the (altered) state of consciousness, the ageing brain, and the quantification of the brain networks’ information processing. In so doing, the present overview identifies that the use of entropic measures for the study of consciousness and its (altered) states led the field to substantially advance the previous findings. Moreover, it realizes that the use of these measures for the study of the ageing brain resulted in significant insights on various ways that the process of ageing may affect the dynamics and information processing capacity of the brain. It further reveals that their utilization for analysis of the brain regional interactivity formed a bridge between the previous two research areas, thereby providing further evidence in support of their results. It concludes by highlighting some potential considerations that may help future research to refine the use of entropic measures for the study of brain complexity and its function. The present study helps realize that (despite their seemingly differing lines of inquiry) the study of consciousness, the ageing brain, and the brain networks’ information processing are highly interrelated. Specifically, it identifies that the complexity, as quantified by entropy, is a fundamental property of conscious experience, which also plays a vital role in the brain’s capacity for adaptation and therefore whose loss by ageing constitutes a basis for diseases and disorders. Interestingly, these two perspectives neatly come together through the association of entropy and the brain capacity for information processing.

Journal ArticleDOI
21 Sep 2020-Entropy
TL;DR: This study shows a comparison among the text feature extraction techniques, and machine learning algorithms to the problem of requirements engineer classification to answer the two major questions “ which works best” and “Which Machine Learning Algorithm provides the best performance for the requirements classification task”.
Abstract: The correct classification of requirements has become an essential task within software engineering. This study shows a comparison among the text feature extraction techniques, and machine learning algorithms to the problem of requirements engineer classification to answer the two major questions “Which works best (Bag of Words (BoW) vs. Term Frequency–Inverse Document Frequency (TF-IDF) vs. Chi Squared (CHI2)) for classifying Software Requirements into Functional Requirements (FR) and Non-Functional Requirements (NF), and the sub-classes of Non-Functional Requirements?” and “Which Machine Learning Algorithm provides the best performance for the requirements classification task?”. The data used to perform the research was the PROMISE_exp, a recently made dataset that expands the already known PROMISE repository, a repository that contains labeled software requirements. All the documents from the database were cleaned with a set of normalization steps and the two feature extractions, and feature selection techniques used were BoW, TF-IDF and CHI2 respectively. The algorithms used for classification were Logist Regression (LR), Support Vector Machine (SVM), Multinomial Naive Bayes (MNB) and k-Nearest Neighbors (kNN). The novelty of our work is the data used to perform the experiment, the details of the steps used to reproduce the classification, and the comparison between BoW, TF-IDF and CHI2 for this repository not having been covered by other studies. This work will serve as a reference for the software engineering community and will help other researchers to understand the requirement classification process. We noticed that the use of TF-IDF followed by the use of LR had a better classification result to differentiate requirements, with an F-measure of 0.91 in binary classification (tying with SVM in that case), 0.74 in NF classification and 0.78 in general classification. As future work we intend to compare more algorithms and new forms to improve the precision of our models.

Journal ArticleDOI
20 Feb 2020-Entropy
TL;DR: The current review helps to reveal the patterns of dysfunction in the brains of patients with AD and to investigate whether signal complexity can be used as a biomarker to accurately respond to the functional lesion in AD.
Abstract: Alzheimer's disease (AD) is a degenerative brain disease with a high and irreversible incidence. In recent years, because brain signals have complex nonlinear dynamics, there has been growing interest in studying complex changes in the time series of brain signals in patients with AD. We reviewed studies of complexity analyses of single-channel time series from electroencephalogram (EEG), magnetoencephalogram (MEG), and functional magnetic resonance imaging (fMRI) in AD and determined future research directions. A systematic literature search for 2000-2019 was performed in the Web of Science and PubMed databases, resulting in 126 identified studies. Compared to healthy individuals, the signals from AD patients have less complexity and more predictable oscillations, which are found mainly in the left parietal, occipital, right frontal, and temporal regions. This complexity is considered a potential biomarker for accurately responding to the functional lesion in AD. The current review helps to reveal the patterns of dysfunction in the brains of patients with AD and to investigate whether signal complexity can be used as a biomarker to accurately respond to the functional lesion in AD. We proposed further studies in the signal complexities of AD patients, including investigating the reliability of complexity algorithms and the spatial patterns of signal complexity. In conclusion, the current review helps to better understand the complexity of abnormalities in the AD brain and provide useful information for AD diagnosis.

Journal ArticleDOI
15 Jul 2020-Entropy
TL;DR: The experimental results and security analysis show that the algorithm has some advantages of large key space, no obvious statistical characteristics of ciphertext, sensitivity to plaintext and key and able to resist differential attacks and chosen plaintext attack.
Abstract: In this paper, we construct a five dimensional continuous hyperchaotic system and propose an image encryption scheme based on the hyperchaotic system, which adopts DNA dynamic coding mechanism and classical scrambling diffusion encryption structure. In the diffusion stage, two rounds of diffusion are adopted and the rules of DNA encoding (DNA decoding) are dynamically changed according to the pixel value of the plaintext image, that is, the rules of DNA encoding (DNA decoding) used to encrypt different images are different, which makes the algorithm can resist chosen-plaintext attack. The encryption (decryption) key is only the initial value of the chaotic system, which overcomes the difficulty of key management in the "one time pad" encryption system. The experimental results and security analysis show that the algorithm has some advantages of large key space, no obvious statistical characteristics of ciphertext, sensitivity to plaintext and key and able to resist differential attacks and chosen plaintext attack. It has good application prospects.

Journal ArticleDOI
20 Jan 2020-Entropy
TL;DR: The Standardized Project Gutenberg Corpus (SPGC), an open science approach to a curated version of the complete PG data containing more than 50,000 books and more than 3×109 word-tokens, is presented, providing a reproducible, pre-processed, full-size version of Project Gutenberg as a new scientific resource for corpus linguistics, natural language processing, and information retrieval.
Abstract: The use of Project Gutenberg (PG) as a text corpus has been extremely popular in statistical analysis of language for more than 25 years. However, in contrast to other major linguistic datasets of similar importance, no consensual full version of PG exists to date. In fact, most PG studies so far either consider only a small number of manually selected books, leading to potential biased subsets, or employ vastly different pre-processing strategies (often specified in insufficient details), raising concerns regarding the reproducibility of published results. In order to address these shortcomings, here we present the Standardized Project Gutenberg Corpus (SPGC), an open science approach to a curated version of the complete PG data containing more than 50,000 books and more than 3 × 10 9 word-tokens. Using different sources of annotated metadata, we not only provide a broad characterization of the content of PG, but also show different examples highlighting the potential of SPGC for investigating language variability across time, subjects, and authors. We publish our methodology in detail, the code to download and process the data, as well as the obtained corpus itself on three different levels of granularity (raw text, timeseries of word tokens, and counts of words). In this way, we provide a reproducible, pre-processed, full-size version of Project Gutenberg as a new scientific resource for corpus linguistics, natural language processing, and information retrieval.

Journal ArticleDOI
20 Jun 2020-Entropy
TL;DR: The proposed ensemble system achieves satisfactory results on the DCC at the AVEC-2016 in comparison with a reference system based on Support Vector Machines and hand-crafted features, with a CNN+LSTM-based system called DepAudionet, and with the case of a single CNN-based classifier.
Abstract: This paper proposes a speech-based method for automatic depression classification The system is based on ensemble learning for Convolutional Neural Networks (CNNs) and is evaluated using the data and the experimental protocol provided in the Depression Classification Sub-Challenge (DCC) at the 2016 Audio-Visual Emotion Challenge (AVEC-2016) In the pre-processing phase, speech files are represented as a sequence of log-spectrograms and randomly sampled to balance positive and negative samples For the classification task itself, first, a more suitable architecture for this task, based on One-Dimensional Convolutional Neural Networks, is built Secondly, several of these CNN-based models are trained with different initializations and then the corresponding individual predictions are fused by using an Ensemble Averaging algorithm and combined per speaker to get an appropriate final decision The proposed ensemble system achieves satisfactory results on the DCC at the AVEC-2016 in comparison with a reference system based on Support Vector Machines and hand-crafted features, with a CNN+LSTM-based system called DepAudionet, and with the case of a single CNN-based classifier

Journal ArticleDOI
24 Sep 2020-Entropy
TL;DR: A pedagogical introduction to the notion of thermodynamic length is introduced, and a geometric lower bound on entropy production in finite-time is presented, which represents a quantum generalisation of the original classical bound.
Abstract: Differential geometry offers a powerful framework for optimising and characterising finite-time thermodynamic processes, both classical and quantum. Here, we start by a pedagogical introduction to the notion of thermodynamic length. We review and connect different frameworks where it emerges in the quantum regime: adiabatically driven closed systems, time-dependent Lindblad master equations, and discrete processes. A geometric lower bound on entropy production in finite-time is then presented, which represents a quantum generalisation of the original classical bound. Following this, we review and develop some general principles for the optimisation of thermodynamic processes in the linear-response regime. These include constant speed of control variation according to the thermodynamic metric, absence of quantum coherence, and optimality of small cycles around the point of maximal ratio between heat capacity and relaxation time for Carnot engines.