Knowledge graph embedding plays an important role in knowledge representation, reasoning, and data mining applications. However, for multiple cross-domain knowledge graphs, state-of-the-art embedding models cannot make full use of the data from different knowledge domains while preserving the privacy of exchanged data. In addition, the centralized embedding model may not scale to the extensive real-world knowledge graphs. Therefore, we propose a novel decentralized scalable learning framework, \emph{Federated Knowledge Graphs Embedding} (FKGE), where embeddings from different knowledge graphs can be learnt in an asynchronous and peer-to-peer manner while being privacy-preserving. FKGE exploits adversarial generation between pairs of knowledge graphs to translate identical entities and relations of different domains into near embedding spaces. In order to protect the privacy of the training data, FKGE further implements a privacy-preserving neural network structure to guarantee no raw data leakage. We conduct extensive experiments to evaluate FKGE on 11 knowledge graphs, demonstrating a significant and consistent improvement in model quality with at most 17.85\% and 7.90\% increases in performance on triple classification and link prediction tasks.

Differentially Private Federated Knowledge Graphs Embedding

Dialogue systems are a popular natural language processing (NLP) task as it is promising in real-life applications. It is also a complicated task since many NLP tasks deserving study are involved. As a result, a multitude of novel works on this task are carried out, and most of them are deep learning based due to their outstanding performance. In this survey, we mainly focus on the deep learning based dialogue systems. We comprehensively review state-of-the-art research outcomes in dialogue systems and analyze them from two angles: model type and system type. Specifically, from the angle of model type, we discuss the principles, characteristics, and applications of different models that are widely used in dialogue systems. This will help researchers acquaint these models and see how they are applied in state-of-the-art frameworks, which is rather helpful when designing a new dialogue system. From the angle of system type, we discuss task-oriented and open-domain dialogue systems as two streams of research, providing insight into the hot topics related. Furthermore, we comprehensively review the evaluation methods and datasets for dialogue systems to pave the way for future research. Finally, some possible research trends are identified based on the recent research outcomes. To the best of our knowledge, this survey is the most comprehensive and up-to-date one at present for deep learning based dialogue systems, extensively covering the popular techniques. We speculate that this work is a good starting point for academics who are new to the dialogue systems or those who want to quickly grasp up-to-date techniques in this area. 

Recent advances in deep learning based dialogue systems: a systematic survey

In recent years, considerable research has been dedicated to the application of neural models in the field of natural language generation (NLG). The primary objective is to generate text that is both linguistically natural and human-like, while also exerting control over the generation process. This paper offers a comprehensive and task-agnostic survey of the recent advancements in neural text generation. These advancements have been facilitated through a multitude of developments, which we categorize into four key areas: data construction, neural frameworks, training and inference strategies, and evaluation metrics. By examining these different aspects, we aim to provide a holistic overview of the progress made in the field. Furthermore, we explore the future directions for the advancement of neural text generation, which encompass the utilization of neural pipelines and the incorporation of background knowledge. These avenues present promising opportunities to further enhance the capabilities of NLG systems. Overall, this survey serves to consolidate the current state of the art in neural text generation and highlights potential avenues for future research and development in this dynamic field.

Recent Advances in Neural Text Generation: A Task-Agnostic Survey

• Developed a novel neural network for CVD mortality prediction with clinical data. • Proposed a more time-efficient feature-based artificial neural network (ANN). • Investigated feature representation to improve ANN for practical guideline. • Boosted calibration of neural networks and enabled model to make less mistakes. • Made model updates in clinical context more accessible, flexible and reliable. Cardiovascular (CVD) is the leading cause of death worldwide and a significant public health concern. Therefore, its mortality prediction is crucial to many existing treatment guidelines. Medical claims data can be used to accurately foresee the health outcomes of patients contracting to a variety of diseases. Many machine learning algorithms, especially deep learning artificial neural networks, can predict mortality rate among patients with CVD using clinical data. Calibration of probabilistic prediction is essential for precise medical interventions as it indicates how well a model’s output matches the probability of the event. However, deep learning neural networks are poorly calibrated. Through experiments, we observe that feature representation is an important factor influencing calibration. This paper proposes a novel feature-based deep learning neural network framework to predict the mortality rate among patients with CVD. Our focus is to present a comprehensive study to achieve advanced performance calibration of mortality prediction on CVD in leveraging deep learning architecture and feature representations. Our study demonstrates that the proposed feature-based neural network framework integrated with Principal Component Analysis or Autoencoders significantly reduces training time and boosts calibration, making model updates in clinical context more flexible and decision-making in medical prevention more reliable. 

Advanced calibration of mortality prediction on cardiovascular disease using feature-based artificial neural network

The knowledge graph can make more accurate personalized recommendations for the recommendation system, but it is also interpretative and has traces to follow. The purpose of the recommendation system is to recommend a series of unobserved items for users. At present, recommendation systems based on knowledge graphs are mainly implemented in two ways: Embedding-based and path-based. Embedding methods usually directly use information from the knowledge graph to enrich the representation of an item or user. Still, it failed to introduce multi-hop relations, and it is challenging to use semantic network information. A path-based recommendation algorithm utilizes the knowledge graph to gain multi-hop knowledge and compare the similarity between users or items to improve the recommendation effect. This paper (1) Aiming at the problem of how the recommendation algorithm effectively utilizes the semantically related information of knowledge, a self-attention-based knowledge representation learning model is designed to learn the semantic information of the entity-relationship by using the overall triplet of the entity-relationship to achieve high-quality knowledge features, Which brings more and more helpful information to the recommendation. (2) Constructing a content recommendation model with unified, embedded behavior and knowledge features, using historical user preferences combined with knowledge graphs to dynamically learn knowledge features to bring users more accurate and diverse recommendations. (3) Aiming at the problem of knowledge feature representation learning, a self-attention-based knowledge representation learning model is proposed. Focusing on the difference in the importance of triples for determining entity semantics, the self-attention mechanism is used to learn semantics from triples to improve knowledge features. The quality of the representation provides high-quality auxiliary information for the recommendation system. The model’s performance is demonstrated through link prediction and triple classification experiments to prove the feasibility of the method proposed in this article.

/pdf/let-knowledge-make-recommendations-for-you-57n7bbo6e4.pdf

Let Knowledge Make Recommendations for You

Entity synonyms discovery is crucial for entity-leveraging applications. However, existing studies suffer from several critical issues: (1) the input mentions may be out-of-vocabulary (OOV) and may come from a different semantic space of the entities; (2) the connection between mentions and entities may be hidden and cannot be established by surface matching; and (3) some entities rarely appear due to the long-tail effect. To tackle these challenges, we facilitate knowledge graphs and propose a novel entity synonyms discovery framework, named KGSynNet. Specifically, we pre-train subword embeddings for mentions and entities using a large-scale domain-specific corpus while learning the knowledge embeddings of entities via a joint TransC-TransE model. More importantly, to obtain a comprehensive representation of entities, we employ a specifically designed fusion gate to adaptively absorb the entities’ knowledge information into their semantic features. We conduct extensive experiments to demonstrate the effectiveness of our KGSynNet in leveraging the knowledge graph. The experimental results show that the KGSynNet improves the state-of-the-art methods by 14.7% in terms of hits@3 in the offline evaluation and outperforms the BERT model by 8.3% in the positive feedback rate of an online A/B test on the entity linking module of a question answering system.

KGSynNet: A Novel Entity Synonyms Discovery Framework with Knowledge Graph.

We introduce a framework of Monte Carlo Tree Search with Double-q Dueling network (MCTS-DDU) for task-completion dialogue policy learning. Different from the previous deep model-based reinforcement learning methods, which uses background planning and may suffer from low-quality simulated experiences, MCTS-DDU performs decision-time planning based on dialogue state search trees built by Monte Carlo simulations and is robust to the simulation errors. Such idea arises naturally in human behaviors, e.g. predicting others' responses and then deciding our own actions. In the simulated movie-ticket booking task, our method outperforms the background planning approaches significantly. We demonstrate the effectiveness of MCTS and the dueling network in detailed ablation studies, and also compare the performance upper bounds of these two planning methods.

Task-Completion Dialogue Policy Learning via Monte Carlo Tree Search with Dueling Network

Neural network (NN) based data2text models achieve state-of-the-art (SOTA) performance in most metrics, but they sometimes drop or modify the information in the input, and it is hard to control the generation contents. Moreover, it requires paired training data that are usually expensive to collect. Template-based methods have good fidelity and controllability but require heavy human involvement. We propose a novel template-based data2text system powered by a text stitch model. It ensures fidelity and controllability by using templates to produce the main contents. In addition, it reduces human involvement in template design by using a text stitch model to automatically stitch adjacent template units, which is a step that usually requires careful template design and limits template reusability. The text stitch model can be trained in self-supervised fashion, which only requires free texts. The experiments on a benchmark dataset show that our system outperforms SOTA NN-based systems in fidelity and surpasses template-based systems in diversity and human involvement.

/pdf/make-templates-smarter-a-template-based-data2text-system-2btodh14lw.pdf

Make Templates Smarter: A Template Based Data2Text System Powered by Text Stitch Model

The invention relates to artificial intelligence, and provides an intelligent question-answering method and device based on a knowledge graph, and computer equipment, and the method comprises the steps: receiving a question text online; extracting a first entity in the question text; searching a first node corresponding to the first entity in a preset knowledge graph; utilizing a preset intentionrecognition rule engine to recognize a first intention of the question text, the intention recognition rule engine being an engine established according to a Trie tree; searching a first answer corresponding to the question text in the knowledge graph by taking the first node as a starting point according to the first intention; and feeding back the first answer according to a preset mode. When intelligent questioning and answering are realized, intention recognition can be quickly carried out through an intention recognition rule engine established based on a Trie tree, and the feedback speedof a questioning and answering system is increased. The natural language is mapped to the knowledge graph for structured query, the query speed is higher, and the result is more accurate.

Intelligent question and answer method and device based on knowledge graph and computer equipment

Entity synonyms discovery is crucial for entity-leveraging applications. However, existing studies suffer from several critical issues: (1) the input mentions may be out-of-vocabulary (OOV) and may come from a different semantic space of the entities; (2) the connection between mentions and entities may be hidden and cannot be established by surface matching; and (3) some entities rarely appear due to the long-tail effect. To tackle these challenges, we facilitate knowledge graphs and propose a novel entity synonyms discovery framework, named \emph{KGSynNet}. Specifically, we pre-train subword embeddings for mentions and entities using a large-scale domain-specific corpus while learning the knowledge embeddings of entities via a joint TransC-TransE model. More importantly, to obtain a comprehensive representation of entities, we employ a specifically designed \emph{fusion gate} to adaptively absorb the entities' knowledge information into their semantic features. We conduct extensive experiments to demonstrate the effectiveness of our \emph{KGSynNet} in leveraging the knowledge graph. The experimental results show that the \emph{KGSynNet} improves the state-of-the-art methods by 14.7\% in terms of hits@3 in the offline evaluation and outperforms the BERT model by 8.3\% in the positive feedback rate of an online A/B test on the entity linking module of a question answering system.

Kunfeng Lai

Papers

KGSynNet: A Novel Entity Synonyms Discovery Framework with Knowledge Graph.

Task-Completion Dialogue Policy Learning via Monte Carlo Tree Search with Dueling Network

Make Templates Smarter: A Template Based Data2Text System Powered by Text Stitch Model

Intelligent question and answer method and device based on knowledge graph and computer equipment

KGSynNet: A Novel Entity Synonyms Discovery Framework with Knowledge Graph