Showing papers by "Université de Montréal published in 2015"

Deep learning

Abstract: Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.

Neural Machine Translation by Jointly Learning to Align and Translate

Abstract: Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-length vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.

A global reference for human genetic variation.

Abstract: The 1000 Genomes Project set out to provide a comprehensive description of common human genetic variation by applying whole-genome sequencing to a diverse set of individuals from multiple populations. Here we report completion of the project, having reconstructed the genomes of 2,504 individuals from 26 populations using a combination of low-coverage whole-genome sequencing, deep exome sequencing, and dense microarray genotyping. We characterized a broad spectrum of genetic variation, in total over 88 million variants (84.7 million single nucleotide polymorphisms (SNPs), 3.6 million short insertions/deletions (indels), and 60,000 structural variants), all phased onto high-quality haplotypes. This resource includes >99% of SNP variants with a frequency of >1% for a variety of ancestries. We describe the distribution of genetic variation across the global sample, and discuss the implications for common disease studies.

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Abstract: Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe how we can train this model in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound. We also show through visualization how the model is able to automatically learn to fix its gaze on salient objects while generating the corresponding words in the output sequence. We validate the use of attention with state-of-the-art performance on three benchmark datasets: Flickr9k, Flickr30k and MS COCO.

Randomized Assessment of Rapid Endovascular Treatment of Ischemic Stroke

Abstract: Among patients with a proximal vessel occlusion in the anterior circulation, 60 to 80% of patients die within 90 days after stroke onset or do not regain functional independence despite alteplase treatment. We evaluated rapid endovascular treatment in addition to standard care in patients with acute ischemic stroke with a small infarct core, a proximal intracranial arterial occlusion, and moderate-to-good collateral circulation. Methods We randomly assigned participants to receive standard care (control group) or standard care plus endovascular treatment with the use of available thrombectomy devices (intervention group). Patients with a proximal intracranial occlusion in the anterior circulation were included up to 12 hours after symptom onset. Patients with a large infarct core or poor collateral circulation on computed tomography (CT) and CT angiography were excluded. Workflow times were measured against predetermined targets. The primary outcome was the score on the modified Rankin scale (range, 0 [no symptoms] to 6 [death]) at 90 days. A proportional odds model was used to calculate the common odds ratio as a measure of the likelihood that the intervention would lead to lower scores on the modified Rankin scale than would control care (shift analysis). Results The trial was stopped early because of efficacy. At 22 centers worldwide, 316 participants were enrolled, of whom 238 received intravenous alteplase (120 in the intervention group and 118 in the control group). In the intervention group, the median time from study CT of the head to first reperfusion was 84 minutes. The rate of functional independence (90-day modified Rankin score of 0 to 2) was increased with the intervention (53.0%, vs. 29.3% in the control group; P<0.001). The primary outcome favored the intervention (common odds ratio, 2.6; 95% confidence interval, 1.7 to 3.8; P<0.001), and the intervention was associated with reduced mortality (10.4%, vs. 19.0% in the control group; P = 0.04). Symptomatic intracerebral hemorrhage occurred in 3.6% of participants in intervention group and 2.7% of participants in control group (P = 0.75). Conclusions Among patients with acute ischemic stroke with a proximal vessel occlusion, a small infarct core, and moderate-to-good collateral circulation, rapid endovascular treatment improved functional outcomes and reduced mortality. (Funded by Covidien and others; ESCAPE ClinicalTrials.gov number, NCT01778335.)

The Blood–Brain Barrier

Abstract: Blood vessels are critical to deliver oxygen and nutrients to all of the tissues and organs throughout the body. The blood vessels that vascularize the central nervous system (CNS) possess unique properties, termed the blood-brain barrier, which allow these vessels to tightly regulate the movement of ions, molecules, and cells between the blood and the brain. This precise control of CNS homeostasis allows for proper neuronal function and also protects the neural tissue from toxins and pathogens, and alterations of these barrier properties are an important component of pathology and progression of different neurological diseases. The physiological barrier is coordinated by a series of physical, transport, and metabolic properties possessed by the endothelial cells (ECs) that form the walls of the blood vessels, and these properties are regulated by interactions with different vascular, immune, and neural cells. Understanding how these different cell populations interact to regulate the barrier properties is essential for understanding how the brain functions during health and disease.

Attention-based models for speech recognition

Abstract: Recurrent sequence generators conditioned on input data through an attention mechanism have recently shown very good performance on a range of tasks including machine translation, handwriting synthesis [1,2] and image caption generation [3]. We extend the attention-mechanism with features needed for speech recognition. We show that while an adaptation of the model used for machine translation in [2] reaches a competitive 18.7% phoneme error rate (PER) on the TIMET phoneme recognition task, it can only be applied to utterances which are roughly as long as the ones it was trained on. We offer a qualitative explanation of this failure and propose a novel and generic method of adding location-awareness to the attention mechanism to alleviate this issue. The new method yields a model that is robust to long inputs and achieves 18% PER in single utterances and 20% in 10-times longer (repeated) utterances. Finally, we propose a change to the attention mechanism that prevents it from concentrating too much on single frames, which further reduces PER to 17.6% level.

Combined Measurement of the Higgs Boson Mass in pp Collisions at √s=7 and 8 TeV with the ATLAS and CMS Experiments

Abstract: A measurement of the Higgs boson mass is presented based on the combined data samples of the ATLAS and CMS experiments at the CERN LHC in the H→γγ and H→ZZ→4l decay channels. The results are obtained from a simultaneous fit to the reconstructed invariant mass peaks in the two channels and for the two experiments. The measured masses from the individual channels and the two experiments are found to be consistent among themselves. The combined measured mass of the Higgs boson is mH=125.09±0.21 (stat)±0.11 (syst) GeV.

Long-Term Use of Ticagrelor in Patients with Prior Myocardial Infarction

Abstract: BACKGROUND The potential benefit of dual antiplatelet therapy beyond 1 year after a myocardial infarction has not been established. We investigated the efficacy and safety of ticagrelor, a P2Y 12 receptor antagonist with established efficacy after an acute coronary syndrome, in this context. METHODS We randomly assigned, in a double-blind 1:1:1 fashion, 21,162 patients who had had a myocardial infarction 1 to 3 years earlier to ticagrelor at a dose of 90 mg twice daily, ticagrelor at a dose of 60 mg twice daily, or placebo. All the patients were to receive low-dose aspirin and were followed for a median of 33 months. The primary efficacy end point was the composite of cardiovascular death, myocardial infarction, or stroke. The primary safety end point was Thrombolysis in Myocardial Infarction (TIMI) major bleeding. RESULTS The two ticagrelor doses each reduced, as compared with placebo, the rate of the primary efficacy end point, with Kaplan–Meier rates at 3 years of 7.85% in the group that received 90 mg of ticagrelor twice daily, 7.77% in the group that received 60 mg of ticagrelor twice daily, and 9.04% in the placebo group (hazard ratio for 90 mg of ticagrelor vs. placebo, 0.85; 95% confidence interval [CI], 0.75 to 0.96; P = 0.008; hazard ratio for 60 mg of ticagrelor vs. placebo, 0.84; 95% CI, 0.74 to 0.95; P = 0.004). Rates of TIMI major bleeding were higher with ticagrelor (2.60% with 90 mg and 2.30% with 60 mg) than with placebo (1.06%) (P<0.001 for each dose vs. placebo); the rates of intracranial hemorrhage or fatal bleeding in the three groups were 0.63%, 0.71%, and 0.60%, respectively. CONCLUSIONS In patients with a myocardial infarction more than 1 year previously, treatment with ticagrelor significantly reduced the risk of cardiovascular death, myocardial infarction, or stroke and increased the risk of major bleeding. (Funded by AstraZeneca; PEGASUS-TIMI 54 ClinicalTrials.gov number, NCT01225562.)

Attention-Based Models for Speech Recognition

Abstract: Recurrent sequence generators conditioned on input data through an attention mechanism have recently shown very good performance on a range of tasks in- cluding machine translation, handwriting synthesis and image caption gen- eration. We extend the attention-mechanism with features needed for speech recognition. We show that while an adaptation of the model used for machine translation in reaches a competitive 18.7% phoneme error rate (PER) on the TIMIT phoneme recognition task, it can only be applied to utterances which are roughly as long as the ones it was trained on. We offer a qualitative explanation of this failure and propose a novel and generic method of adding location-awareness to the attention mechanism to alleviate this issue. The new method yields a model that is robust to long inputs and achieves 18% PER in single utterances and 20% in 10-times longer (repeated) utterances. Finally, we propose a change to the at- tention mechanism that prevents it from concentrating too much on single frames, which further reduces PER to 17.6% level.

BinaryConnect: training deep neural networks with binary weights during propagations

Abstract: Deep Neural Networks (DNN) have achieved state-of-the-art results in a wide range of tasks, with the best results obtained with large training sets and large models. In the past, GPUs enabled these breakthroughs because of their greater computational speed. In the future, faster computation at both training and test time is likely to be crucial for further progress and for consumer applications on low-power devices. As a result, there is much interest in research and development of dedicated hardware for Deep Learning (DL). Binary weights, i.e., weights which are constrained to only two possible values (e.g. -1 or 1), would bring great benefits to specialized DL hardware by replacing many multiply-accumulate operations by simple accumulations, as multipliers are the most space and power-hungry components of the digital implementation of neural networks. We introduce BinaryConnect, a method which consists in training a DNN with binary weights during the forward and backward propagations, while retaining precision of the stored weights in which gradients are accumulated. Like other dropout schemes, we show that BinaryConnect acts as regularizer and we obtain near state-of-the-art results with BinaryConnect on the permutation-invariant MNIST, CIFAR-10 and SVHN.

On-Demand Preexposure Prophylaxis in Men at High Risk for HIV-1 Infection

Abstract: Background Antiretroviral preexposure prophylaxis has been shown to reduce the risk of human immunodeficiency virus type 1 (HIV-1) infection in some studies, but conflicting results have been reported among studies, probably due to challenges of adherence to a daily regimen. Methods We conducted a double-blind, randomized trial of antiretroviral therapy for preexposure HIV-1 prophylaxis among men who have unprotected anal sex with men. Participants were randomly assigned to take a combination of tenofovir disoproxil fumarate (TDF) and emtricitabine (FTC) or placebo before and after sexual activity. All participants received risk-reduction counseling and condoms and were regularly tested for HIV-1 and HIV-2 and other sexually transmitted infections. Results Of the 414 participants who underwent randomization, 400 who did not have HIV infection were enrolled (199 in the TDF-FTC group and 201 in the placebo group). All participants were followed for a median of 9.3 months (interquartile range, 4.9 to 20.6). A total of 16 HIV-1 infections occurred during follow-up, 2 in the TDF-FTC group (incidence, 0.91 per 100 person-years) and 14 in the placebo group (incidence, 6.60 per 100 person-years), a relative reduction in the TDF-FTC group of 86% (95% confidence interval, 40 to 98; P=0.002). Participants took a median of 15 pills of TDF-FTC or placebo per month (P=0.57). The rates of serious adverse events were similar in the two study groups. In the TDF-FTC group, as compared with the placebo group, there were higher rates of gastrointestinal adverse events (14% vs. 5%, P=0.002) and renal adverse events (18% vs. 10%, P=0.03). Conclusions The use of TDF-FTC before and after sexual activity provided protection against HIV-1 infection in men who have sex with men. The treatment was associated with increased rates of gastrointestinal and renal adverse events. (Funded by the National Agency of Research on AIDS and Viral Hepatitis [ ANRS] and others; ClinicalTrials.gov number, NCT01473472.)

SUDS, LID, BMPs, WSUD and more – The evolution and application of terminology surrounding urban drainage

Abstract: The management of urban stormwater has become increasingly complex over recent decades. Consequently, terminology describing the principles and practices of urban drainage has become increasingly diverse, increasing the potential for confusion and miscommunication. This paper documents the history, scope, application and underlying principles of terms used in urban drainage and provides recommendations for clear communication of these principles. Terminology evolves locally and thus has an important role in establishing awareness and credibility of new approaches and contains nuanced understandings of the principles that are applied locally to address specific problems. Despite the understandable desire to have a ‘uniform set of terminology’, such a concept is flawed, ignoring the fact that terms reflect locally shared understanding. The local development of terminology thus has an important role in advancing the profession, but authors should facilitate communication between disciplines and between regio...

Photooxidation and quantum confinement effects in exfoliated black phosphorus

Abstract: The degradation of exfoliated black phosphorus in ambient conditions may limit its use in electronic devices. The combined effects of light irradiation and exposure to oxygen on mono- and multilayers of this material are now investigated.

Describing Videos by Exploiting Temporal Structure

Abstract: Recent progress in using recurrent neural networks (RNNs) for image description has motivated the exploration of their application for video description. However, while images are static, working with videos requires modeling their dynamic temporal structure and then properly integrating that information into a natural language description model. In this context, we propose an approach that successfully takes into account both the local and global temporal structure of videos to produce descriptions. First, our approach incorporates a spatial temporal 3-D convolutional neural network (3-D CNN) representation of the short temporal dynamics. The 3-D CNN representation is trained on video action recognition tasks, so as to produce a representation that is tuned to human motion and behavior. Second we propose a temporal attention mechanism that allows to go beyond local temporal modeling and learns to automatically select the most relevant temporal segments given the text-generating RNN. Our approach exceeds the current state-of-art for both BLEU and METEOR metrics on the Youtube2Text dataset. We also present results on a new, larger and more challenging dataset of paired video and natural language descriptions.

Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

Abstract: We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models. Generative models produce system responses that are autonomously generated word-by-word, opening up the possibility for realistic, flexible interactions. In support of this goal, we extend the recently proposed hierarchical recurrent encoder-decoder neural network to the dialogue domain, and demonstrate that this model is competitive with state-of-the-art neural language models and back-off n-gram models. We investigate the limitations of this and similar approaches, and show how its performance can be improved by bootstrapping the learning from a larger question-answer pair corpus and from pretrained word embeddings.

Tenofovir-Based Preexposure Prophylaxis for HIV Infection among African Women

Abstract: BackgroundReproductive-age women need effective interventions to prevent the acquisition of human immunodeficiency virus type 1 (HIV-1) infection. MethodsWe conducted a randomized, placebo-controlled trial to assess daily treatment with oral tenofovir disoproxil fumarate (TDF), oral tenofovir–emtricitabine (TDF-FTC), or 1% tenofovir (TFV) vaginal gel as preexposure prophylaxis against HIV-1 infection in women in South Africa, Uganda, and Zimbabwe. HIV-1 testing was performed monthly, and plasma TFV levels were assessed quarterly. ResultsOf 12,320 women who were screened, 5029 were enrolled in the study. The rate of retention in the study was 91% during 5509 person-years of follow-up. A total of 312 HIV-1 infections occurred; the incidence of HIV-1 infection was 5.7 per 100 person-years. In the modified intention-to-treat analysis, the effectiveness was −49.0% with TDF (hazard ratio for infection, 1.49; 95% confidence interval [CI], 0.97 to 2.29), −4.4% with TDF-FTC (hazard ratio, 1.04; 95% CI, 0.73 to 1.4...

Abiraterone acetate plus prednisone versus placebo plus prednisone in chemotherapy-naive men with metastatic castration-resistant prostate cancer (COU-AA-302): final overall survival analysis of a randomised, double-blind, placebo-controlled phase 3 study

Abstract: Summary Background Abiraterone acetate plus prednisone significantly improved radiographic progression-free survival compared with placebo plus prednisone in men with chemotherapy-naive castration-resistant prostate cancer at the interim analyses of the COU-AA-302 trial. Here, we present the prespecified final analysis of the trial, assessing the effect of abiraterone acetate plus prednisone on overall survival, time to opiate use, and use of other subsequent therapies. Methods In this placebo-controlled, double-blind, randomised phase 3 study, 1088 asymptomatic or mildly symptomatic patients with chemotherapy-naive prostate cancer stratified by Eastern Cooperative Oncology performance status (0 vs 1) were randomly assigned with a permuted block allocation scheme via a web response system in a 1:1 ratio to receive either abiraterone acetate (1000 mg once daily) plus prednisone (5 mg twice daily; abiraterone acetate group) or placebo plus prednisone (placebo group). Coprimary endpoints were radiographic progression-free survival and overall survival analysed in the intention-to-treat population. The study is registered with ClinicalTrials.gov, number NCT00887198. Findings At a median follow-up of 49·2 months (IQR 47·0–51·8), 741 (96%) of the prespecified 773 death events for the final analysis had been observed: 354 (65%) of 546 patients in the abiraterone acetate group and 387 (71%) of 542 in the placebo group. 238 (44%) patients initially receiving prednisone alone subsequently received abiraterone acetate plus prednisone as crossover per protocol (93 patients) or as subsequent therapy (145 patients). Overall, 365 (67%) patients in the abiraterone acetate group and 435 (80%) in the placebo group received subsequent treatment with one or more approved agents. Median overall survival was significantly longer in the abiraterone acetate group than in the placebo group (34·7 months [95% CI 32·7–36·8] vs 30·3 months [28·7–33·3]; hazard ratio 0·81 [95% CI 0·70–0·93]; p=0·0033). The most common grade 3–4 adverse events of special interest were cardiac disorders (41 [8%] of 542 patients in the abiraterone acetate group vs 20 [4%] of 540 patients in the placebo group), increased alanine aminotransferase (32 [6%] vs four [ vs 17 [3%]). Interpretation In this randomised phase 3 trial with a median follow-up of more than 4 years, treatment with abiraterone acetate prolonged overall survival compared with prednisone alone by a margin that was both clinically and statistically significant. These results further support the favourable safety profile of abiraterone acetate in patients with chemotherapy-naive metastatic castration-resistant prostate cancer. Funding Janssen Research & Development.

Vascular Anomalies Classification: Recommendations From the International Society for the Study of Vascular Anomalies

Abstract: Vascular anomalies represent a spectrum of disorders from a simple "birthmark" to life- threatening entities. Incorrect nomenclature and misdiagnoses are commonly experienced by patients with these anomalies. Accurate diagnosis is crucial for appropriate evaluation and management, often requiring multidisciplinary specialists. Classification schemes provide a consistent terminology and serve as a guide for pathologists, clinicians, and researchers. One of the goals of the International Society for the Study of Vascular Anomalies (ISSVA) is to achieve a uniform classification. The last classification (1997) stratified vascular lesions into vascular malformations and proliferative vascular lesions (tumors). However, additional disease entities have since been identified that are complex and less easily classified by generic headings, such as capillary malformation, venous malformation, lymphatic malformation, etc. We hereby present the updated official ISSVA classification of vascular anomalies. The general biological scheme of the classification is retained. The section on tumors has been expanded and lists the main recognized vascular tumors, classified as benign, locally aggressive or borderline, and malignant. A list of well-defined diseases is included under each generic heading in the "Simple Vascular Malformations" section. A short definition is added for eponyms. Two new sections were created: one dealing with the malformations of individually named vessels (previously referred to as "truncular" malformations); the second groups lesions of uncertain or debated nature (tumor versus malformation). The known genetic defects underlying vascular anomalies are included in an appendix. This classification is meant to be a framework, acknowledging that it will require modification as new scientific information becomes available.

On Using Very Large Target Vocabulary for Neural Machine Translation

Abstract: Neural machine translation, a recently proposed approach to machine translation based purely on neural networks, has shown promising results compared to the existing approaches such as phrase-based statistical machine translation. Despite its recent success, neural machine translation has its limitation in handling a larger vocabulary, as training complexity as well as decoding complexity increase proportionally to the number of target words. In this paper, we propose a method based on importance sampling that allows us to use a very large target vocabulary without increasing training complexity. We show that decoding can be efficiently done even with the model having a very large target vocabulary by selecting only a small subset of the whole target vocabulary. The models trained by the proposed approach are empirically found to outperform the baseline models with a small vocabulary as well as the LSTM-based neural machine translation models. Furthermore, when we use the ensemble of a few models with very large target vocabularies, we achieve the state-of-the-art translation performance (measured by BLEU) on the English!German translation and almost as high performance as state-of-the-art English!French translation system.

Regional Nodal Irradiation in Early-Stage Breast Cancer

Abstract: BackgroundMost women with breast cancer who undergo breast-conserving surgery receive whole-breast irradiation. We examined whether the addition of regional nodal irradiation to whole-breast irradiation improved outcomes. MethodsWe randomly assigned women with node-positive or high-risk node-negative breast cancer who were treated with breast-conserving surgery and adjuvant systemic therapy to undergo either whole-breast irradiation plus regional nodal irradiation (including internal mammary, supraclavicular, and axillary lymph nodes) (nodal-irradiation group) or whole-breast irradiation alone (control group). The primary outcome was overall survival. Secondary outcomes were disease-free survival, isolated locoregional disease-free survival, and distant disease-free survival. ResultsBetween March 2000 and February 2007, a total of 1832 women were assigned to the nodal-irradiation group or the control group (916 women in each group). The median follow-up was 9.5 years. At the 10-year follow-up, there was n...

A polyphenol-rich cranberry extract protects from diet-induced obesity, insulin resistance and intestinal inflammation in association with increased Akkermansia spp. population in the gut microbiota of mice

Abstract: Objective The increasing prevalence of obesity and type 2 diabetes (T2D) demonstrates the failure of conventional treatments to curb these diseases. The gut microbiota has been put forward as a key player in the pathophysiology of diet-induced T2D. Importantly, cranberry ( Vaccinium macrocarpon Aiton) is associated with a number of beneficial health effects. We aimed to investigate the metabolic impact of a cranberry extract (CE) on high fat/high sucrose (HFHS)-fed mice and to determine whether its consequent antidiabetic effects are related to modulations in the gut microbiota. Design C57BL/6J mice were fed either a chow or a HFHS diet. HFHS-fed mice were gavaged daily either with vehicle (water) or CE (200 mg/kg) for 8 weeks. The composition of the gut microbiota was assessed by analysing 16S rRNA gene sequences with 454 pyrosequencing. Results CE treatment was found to reduce HFHS-induced weight gain and visceral obesity. CE treatment also decreased liver weight and triglyceride accumulation in association with blunted hepatic oxidative stress and inflammation. CE administration improved insulin sensitivity, as revealed by improved insulin tolerance, lower homeostasis model assessment of insulin resistance and decreased glucose-induced hyperinsulinaemia during an oral glucose tolerance test. CE treatment was found to lower intestinal triglyceride content and to alleviate intestinal inflammation and oxidative stress. Interestingly, CE treatment markedly increased the proportion of the mucin-degrading bacterium Akkermansia in our metagenomic samples. Conclusions CE exerts beneficial metabolic effects through improving HFHS diet-induced features of the metabolic syndrome, which is associated with a proportional increase in Akkermansia spp. population.

A Recurrent Latent Variable Model for Sequential Data

Abstract: In this paper, we explore the inclusion of latent random variables into the dynamic hidden state of a recurrent neural network (RNN) by combining elements of the variational autoencoder. We argue that through the use of high-level latent random variables, the variational RNN (VRNN)1 can model the kind of variability observed in highly structured sequential data such as natural speech. We empirically evaluate the proposed model against related sequential models on four speech datasets and one handwriting dataset. Our results show the important roles that latent random variables can play in the RNN dynamic hidden state.

The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems

Abstract: This paper introduces the Ubuntu Dialogue Corpus, a dataset containing almost 1 million multi-turn dialogues, with a total of over 7 million utterances and 100 million words. This provides a unique resource for research into building dialogue managers based on neural language models that can make use of large amounts of unlabeled data. The dataset has both the multi-turn property of conversations in the Dialog State Tracking Challenge datasets, and the unstructured nature of interactions from microblog services such as Twitter. We also describe two neural learning architectures suitable for analyzing this dataset, and provide benchmark performance on the task of selecting the best next response.