The screening of advanced materials coupled with the modeling of their quantitative structural-activity relationships has recently become one of the hot and trending topics in energy materials due to the diverse challenges, including low success probabilities, high time consumption, and high computational cost associated with the traditional methods of developing energy materials. Following this, new research concepts and technologies to promote the research and development of energy materials become necessary. The latest advancements in artificial intelligence and machine learning have therefore increased the expectation that data-driven materials science would revolutionize scientific discoveries towards providing new paradigms for the development of energy materials. Furthermore, the current advances in data-driven materials engineering also demonstrate that the application of machine learning technology would not only significantly facilitate the design and development of advanced energy materials but also enhance their discovery and deployment. In this article, the importance and necessity of developing new energy materials towards contributing to the global carbon neutrality are presented. A comprehensive introduction to the fundamentals of machine learning is also provided, including open-source databases, feature engineering, machine learning algorithms, and analysis of machine learning model. Afterwards, the latest progress in data-driven materials science and engineering, including alkaline ion battery materials, photovoltaic materials, catalytic materials, and carbon dioxide capture materials, is discussed. Finally, relevant clues to the successful applications of machine learning and the remaining challenges towards the development of advanced energy materials are highlighted.

Machine learning for advanced energy materials

In this paper we propose an efficient hardware architecture that implements the Q-Learning algorithm, suitable for real-time applications. Its main features are low-power, high throughput and limited hardware resources. We also propose a technique based on approximated multipliers to reduce the hardware complexity of the algorithm. We implemented the design on a Xilinx Zynq Ultrascale+ MPSoC ZCU106 Evaluation Kit. The implementation results are evaluated in terms of hardware resources, throughput and power consumption. The architecture is compared to the state of the art of Q-Learning hardware accelerators presented in the literature obtaining better results in speed, power and hardware resources. Experiments using different sizes for the Q-Matrix and different wordlengths for the fixed point arithmetic are presented. With a Q-Matrix of size $8\times4$ (8 bit data) we achieved a throughput of 222 MSPS (Mega Samples Per Second) and a dynamic power consumption of 37 mW, while with a Q-Matrix of size $256\times16$ (32 bit data) we achieved a throughput of 93 MSPS and a power consumption 611 mW. Due to the small amount of hardware resources required by the accelerator, our system is suitable for multi-agent IoT applications. Moreover, the architecture can be used to implement the SARSA (State-Action-Reward-State-Action) Reinforcement Learning algorithm with minor modifications.

/pdf/an-efficient-hardware-implementation-of-reinforcement-21vddvh0j7.pdf

An Efficient Hardware Implementation of Reinforcement Learning: The Q-Learning Algorithm

This paper presents a comprehensive survey of the meta-heuristic optimization algorithms on the text clustering applications and highlights its main procedures. These Artificial Intelligence (AI) algorithms are recognized as promising swarm intelligence methods due to their successful ability to solve machine learning problems, especially text clustering problems. This paper reviews all of the relevant literature on meta-heuristic-based text clustering applications, including many variants, such as basic, modified, hybridized, and multi-objective methods. As well, the main procedures of text clustering and critical discussions are given. Hence, this review reports its advantages and disadvantages and recommends potential future research paths. The main keywords that have been considered in this paper are text, clustering, meta-heuristic, optimization, and algorithm.

/pdf/advances-in-meta-heuristic-optimization-algorithms-in-big-1enhg95wen.pdf

Advances in Meta-Heuristic Optimization Algorithms in Big Data Text Clustering

Wireless underground sensor networks (WUSNs) facilitate remote monitoring and control of various underground environments, which are suffering from a significant reliability problem. To address this problem and relieve the ongoing networking challenges, we propose a new concept, called the magnetic induction (MI)-assisted wireless powered underground sensor network (MI-WPUSN), which integrates the advantages of MI communication techniques with those of wireless power transfer mechanisms. MI-WPUSN offers a unique platform consisting of seven envisioned devices and four distinct communication modes to provide significant reliability potential, which is constrained by its complex and challenging data collection. To unlock the potential of MI-WPUSN, we provide a systematic research roadmap for MI-WPUSN data collection, spreading from sensor deployment to multiple channel access control and to frequency-selective routing establishment. Specifically, we first summarize the state-of-the-art research advances on sensor deployment, based on which we propose a bidirectional projection-based deployment topology that achieves high network reliability and discuss the sensor numerical determination problem resorting to weighted Voronoi diagrams. We then introduce the principles for multiple channel access control and review the existing research. Finally, after introducing kernel regression methods that can easily handle the frequency-selective property of MI-WPUSN, we develop a modified Q-learning-based routing protocol. In particular, we list and analyze the underlying challenges and related future issues in each direction.

Data Collection in MI-Assisted Wireless Powered Underground Sensor Networks: Directions, Recent Advances, and Challenges

Aim to improve the power efficiency of the dual-active-bridge (DAB) dc–dc converter, an efficiency optimization scheme with triple-phase-shift (TPS) modulation using reinforcement learning (RL) is proposed in this article. More specifically, the Q-learning algorithm, as a typical algorithm of the RL, is applied to train an agent offline to obtain an optimized modulation strategy, and then the trained agent provides control decisions online in a real-time manner for the DAB dc–dc converter according to the current operating environment. The main objective is to obtain the optimal phase-shift angles for the DAB dc–dc converter, which can achieve the maximum power efficiency by reducing the power losses. Moreover, all possible operation modes of the TPS modulation are considered during the offline training process of the Q-learning algorithm. Thus, the cumbersome process for selecting the optimal operation mode in the conventional schemes can be circumvented successfully. Based on these merits, the proposed efficiency optimization scheme using the RL can realize the excellent performances for the whole load conditions and voltage conversion ratios. Finally, a 1.2-KW prototyped is built, and the simulation and the experimental results demonstrate that the power efficiency can be improved by using the optimization scheme based on the RL.

Reinforcement Learning Based Efficiency Optimization Scheme for the DAB DC–DC Converter With Triple-Phase-Shift Modulation

Q-learning is arguably one of the most applied representative reinforcement learning approaches and one of the off-policy strategies. Since the emergence of Q-learning, many studies have described its uses in reinforcement learning and artificial intelligence problems. However, there is an information gap as to how these powerful algorithms can be leveraged and incorporated into general artificial intelligence workflow. Early Q-learning algorithms were unsatisfactory in several aspects and covered a narrow range of applications. It has also been observed that sometimes, this rather powerful algorithm learns unrealistically and overestimates the action values hence abating the overall performance. Recently with the general advances of machine learning, more variants of Q-learning like Deep Q-learning which combines basic Q learning with deep neural networks have been discovered and applied extensively. In this paper, we thoroughly explain how Q-learning evolved by unraveling the mathematical complexities behind it as well its flow from reinforcement learning family of algorithms. Improved variants are fully described, and we categorize Q-learning algorithms into single-agent and multi-agent approaches. Finally, we thoroughly investigate up-to-date research trends and key applications that leverage Q-learning algorithms.

/pdf/q-learning-algorithms-a-comprehensive-classification-and-4ont4s8shd.pdf

Q-Learning Algorithms: A Comprehensive Classification and Applications

There is a need to extract meaningful information from big data, classify it into different categories, and predict end-user behavior or emotions. Large amounts of data are generated from various sources such as social media and websites. Text classification is a representative research topic in the field of natural-language processing that categorizes unstructured text data into meaningful categorical classes. The long short-term memory (LSTM) model and the convolutional neural network for sentence classification produce accurate results and have been recently used in various natural-language processing (NLP) tasks. Convolutional neural network (CNN) models use convolutional layers and maximum pooling or max-overtime pooling layers to extract higher-level features, while LSTM models can capture long-term dependencies between word sequences hence are better used for text classification. However, even with the hybrid approach that leverages the powers of these two deep-learning models, the number of features to remember for classification remains huge, hence hindering the training process. In this study, we propose an attention-based Bi-LSTM+CNN hybrid model that capitalize on the advantages of LSTM and CNN with an additional attention mechanism. We trained the model using the Internet Movie Database (IMDB) movie review data to evaluate the performance of the proposed model, and the test results showed that the proposed hybrid attention Bi-LSTM+CNN model produces more accurate classification results, as well as higher recall and F1 scores, than individual multi-layer perceptron (MLP), CNN or LSTM models as well as the hybrid models.

Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism

Due to the prevalence of globalization and the surge in people’s traffic, diseases are spreading more rapidly than ever and the risks of sporadic contamination are becoming higher than before. Disease warnings continue to rely on censored data, but these warning systems have failed to cope with the speed of disease proliferation. Due to the risks associated with the problem, there have been many studies on disease outbreak surveillance systems, but existing systems have limitations in monitoring disease-related topics and internationalization. With the advent of online news, social media and search engines, social and web data contain rich unexplored data that can be leveraged to provide accurate, timely disease activities and risks. In this study, we develop an infectious disease surveillance system for extracting information related to emerging diseases from a variety of Internet-sourced data. We also propose an effective deep learning-based data filtering and ranking algorithm. This system provides nation-specific disease outbreak information, disease-related topic ranking, a number of reports per district and disease through various visualization techniques such as a map, graph, chart, correlation and coefficient, and word cloud. Our system provides an automated web-based service, and it is free for all users and live in operation.

EagleEye: A Worldwide Disease-Related Topic Extraction System Using a Deep Learning Based Ranking Algorithm and Internet-Sourced Data.

Although there have been many researches on the disease information system with the increased interest in disease, the existing systems have limitations in terms of emerging disease monitoring and internationalization. The purpose of this study is to develop a worldwide emerging disease-related information extraction system from news data, which provides nation-specific disease related information, disease-related topic ranking, map-based number of news articles per region, and various charts showing top disease regions and diseases. Our system is available on the web through http://www.epidemic.co.kr/worldwide.

Myeonghwi Kim

Papers

Q-Learning Algorithms: A Comprehensive Classification and Applications

Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism

EagleEye: A Worldwide Disease-Related Topic Extraction System Using a Deep Learning Based Ranking Algorithm and Internet-Sourced Data.

Worldwide emerging disease-related information extraction system from news data