scispace - formally typeset

Author

Raymond J. Mooney

Bio: Raymond J. Mooney is an academic researcher from University of Texas at Austin. The author has contributed to research in topic(s): Natural language & Parsing. The author has an hindex of 86, co-authored 308 publication(s) receiving 32776 citation(s). Previous affiliations of Raymond J. Mooney include University of Illinois at Urbana–Champaign.
Papers
More filters

Journal ArticleDOI
Matthew Marge1, Carol Y. Espy-Wilson2, Nigel Ward3, Abeer Alwan4  +24 moreInstitutions (21)
TL;DR: This article identifies key scientific and engineering advances needed to enable effective spoken language interaction with robotics, and makes 25 recommendations, involving eight general themes: putting human needs first, better modeling the social and interactive aspects of language, improving robustness, creating new methods for rapid adaptation, and improving research infrastructure and resources.
Abstract: With robotics rapidly advancing, more effective human–robot interaction is increasingly needed to realize the full potential of robots for society. While spoken language must be part of the solution, our ability to provide spoken language interaction capabilities is still very limited. In this article, based on the report of an interdisciplinary workshop convened by the National Science Foundation, we identify key scientific and engineering advances needed to enable effective spoken language interaction with robotics. We make 25 recommendations, involving eight general themes: putting human needs first, better modeling the social and interactive aspects of language, improving robustness, creating new methods for rapid adaptation, better integrating speech and language with other communication modalities, giving speech and language components access to rich representations of the robot’s current knowledge and state, making all components operate in real time, and improving research infrastructure and resources. Research and development that prioritizes these topics will, we believe, provide a solid foundation for the creation of speech-capable robots that are easy and effective for humans to work with.

3 citations


Posted Content
Abstract: When a software bug is reported, developers engage in a discussion to collaboratively resolve it. While the solution is likely formulated within the discussion, it is often buried in a large amount of text, making it difficult to comprehend, which delays its implementation. To expedite bug resolution, we propose generating a concise natural language description of the solution by synthesizing relevant content within the discussion, which encompasses both natural language and source code. Furthermore, to support generating an informative description during an ongoing discussion, we propose a secondary task of determining when sufficient context about the solution emerges in real-time. We construct a dataset for these tasks with a novel technique for obtaining noisy supervision from repository changes linked to bug reports. We establish baselines for generating solution descriptions, and develop a classifier which makes a prediction following each new utterance on whether or not the necessary context for performing generation is available. Through automated and human evaluation, we find these tasks to form an ideal testbed for complex reasoning in long, bimodal dialogue context.

Posted Content
Abstract: There has been a growing interest in developing machine learning (ML) models for code learning tasks, e.g., comment generation and method naming. Despite substantial increase in the effectiveness of ML models, the evaluation methodologies, i.e., the way people split datasets into training, validation, and testing sets, were not well designed. Specifically, no prior work on the aforementioned topics considered the timestamps of code and comments during evaluation (e.g., examples in the testing set might be from 2010 and examples from the training set might be from 2020). This may lead to evaluations that are inconsistent with the intended use cases of the ML models. In this paper, we formalize a novel time-segmented evaluation methodology, as well as the two methodologies commonly used in the literature: mixed-project and cross-project. We argue that time-segmented methodology is the most realistic. We also describe various use cases of ML models and provide a guideline for using methodologies to evaluate each use case. To assess the impact of methodologies, we collect a dataset of code-comment pairs with timestamps to train and evaluate several recent code learning ML models for the comment generation and method naming tasks. Our results show that different methodologies can lead to conflicting and inconsistent results. We invite the community to adopt the time-segmented evaluation methodology.

1 citations


Proceedings ArticleDOI
01 Aug 2021
Abstract: Answering questions about why characters perform certain actions is central to understanding and reasoning about narratives. Despite recent progress in QA, it is not clear if existing models have the ability to answer "why" questions that may require commonsense knowledge external to the input narrative. In this work, we introduce TellMeWhy, a new crowd-sourced dataset that consists of more than 30k questions and free-form answers concerning why characters in short narratives perform the actions described. For a third of this dataset, the answers are not present within the narrative. Given the limitations of automated evaluation for this task, we also present a systematized human evaluation interface for this dataset. Our evaluation of state-of-the-art models show that they are far below human performance on answering such questions. They are especially worse on questions whose answers are external to the narrative, thus providing a challenge for future QA and narrative understanding research.

Proceedings ArticleDOI
Abstract: Answering questions about why characters perform certain actions is central to understanding and reasoning about narratives. Despite recent progress in QA, it is not clear if existing models have the ability to answer "why" questions that may require commonsense knowledge external to the input narrative. In this work, we introduce TellMeWhy, a new crowd-sourced dataset that consists of more than 30k questions and free-form answers concerning why characters in short narratives perform the actions described. For a third of this dataset, the answers are not present within the narrative. Given the limitations of automated evaluation for this task, we also present a systematized human evaluation interface for this dataset. Our evaluation of state-of-the-art models show that they are far below human performance on answering such questions. They are especially worse on questions whose answers are external to the narrative, thus providing a challenge for future QA and narrative understanding research.

Cited by
More filters

Journal ArticleDOI
Abstract: In the literature on classification problems, it is widely discussed how the presence of label noise can bring about severe degradation in performance. Several works have applied Prototype Selection techniques, Ensemble Methods, or both, in an attempt to alleviate this issue. Nevertheless, these methods are not always able to sufficiently counteract the effects of noise. In this work, we investigate the effects of noise on a particular class of Ensemble Methods, that of Dynamic Selection algorithms, and we are especially interested in the behavior of the Fire-DES++ algorithm, a state of the art algorithm which applies the Edited Nearest Neighbors (ENN) algorithm to deal with the effects of noise and imbalance. We propose a method which employs multiple Dynamic Selection sets, based on the Bagging-IH algorithm, which we dub Multiple-Set Dynamic Selection (MSDS), in an attempt to supplant the ENN algorithm on the filtering step. We find that almost all methods based on Dynamic Selection are severely affected by the presence of label noise, with the exception of the K-Nearest Oracles-Union algorithm. We also find that our proposed method can alleviate the issues caused by noise in some scenarios.

Journal ArticleDOI
Hedi Ben-younes1, Eloi Zablocki1, Patrick Pérez1, Matthieu Cord2  +1 moreInstitutions (2)
Abstract: In this era of active development of autonomous vehicles, it becomes crucial to provide driving systems with the capacity to explain their decisions. In this work, we focus on generating high-level driving explanations as the vehicle drives. We present BEEF, for BEhavior Explanation with Fusion, a deep architecture which explains the behavior of a trajectory prediction model. Supervised by annotations of human driving decisions justifications, BEEF learns to fuse features from multiple levels. Leveraging recent advances in the multi-modal fusion literature, BEEF is carefully designed to model the correlations between high-level decisions features and mid-level perceptual features. The flexibility and efficiency of our approach are validated with extensive experiments on the HDD and BDD-X datasets.

DOI
01 Mar 2022
Abstract: Incremental learning enables the learner to accommodate new knowledge without retraining the existing model. It is a challenging task that requires learning from new data and preserving the knowledge extracted from the previously accessed data. This challenge is known as the stability-plasticity dilemma. We propose A-iLearn, a generic model for incremental learning which overcomes the stability-plasticity dilemma by carefully integrating the ensemble of base classifiers trained on new data with the current ensemble without retraining the model from scratch using entire data. We demonstrate the efficacy of the proposed A-iLearn model on spoof fingerprint detection application. One of the significant challenges associated with spoof fingerprint detection is the performance drop on spoofs generated using new fabrication materials. A-iLearn is an adaptive incremental learning model that adapts to the features of the “live” and “spoof” fingerprint images and efficiently recognizes the new spoof fingerprints and the known spoof fingerprints when the new data is available. To the best of our knowledge, A-iLearn is the first attempt in incremental learning algorithms that adapts to the properties of data for generating a diverse ensemble of base classifiers. From the experiments conducted on standard high-dimensional datasets LivDet 2011, LivDet 2013 and LivDet 2015, we show that the performance gain on new fake materials is significantly high. On average, we achieve 49.57% improvement in accuracy between the consecutive learning phases.

Journal ArticleDOI
Shixiong Zhao1, Fanxin Li1, Xusheng Chen1, Xiuxian Guan1  +9 moreInstitutions (3)
Abstract: The increasing computational complexity of DNNs achieved unprecedented successes in various areas such as machine vision and natural language processing (NLP), e.g., the recent advanced Transformer has billions of parameters. However, as large-scale DNNs significantly exceed GPU’s physical memory limit, they cannot be trained by conventional methods such as data parallelism. Pipeline parallelism that partitions a large DNN into small subnets and trains them on different GPUs is a plausible solution. Unfortunately, the layer partitioning and memory management in existing pipeline parallel systems are fixed during training, making them easily impeded by out-of-memory errors and the GPU under-utilization. These drawbacks amplify when performing neural architecture search (NAS) such as the evolved Transformer, where different network architectures of Transformer needed to be trained repeatedly. vPipe is the first system that transparently provides dynamic layer partitioning and memory management for pipeline parallelism. vPipe has two unique contributions, including (1) an online algorithm for searching a near-optimal layer partitioning and memory management plan, and (2) a live layer migration protocol for re-balancing the layer distribution across a training pipeline. vPipe improved the training throughput of two notable baselines (Pipedream and GPipe) by 61.4-463.4 percent and 24.8-291.3 percent on various large DNNs and training settings.

2 citations


Journal ArticleDOI
Rabin Chakrabortty1, Subodh Chandra Pal1, M. Santosh2, M. Santosh3  +2 moreInstitutions (3)
Abstract: Land degradation significantly impacts habitats, agriculture and economy, particularly in regions with high population density. Gully erosion poses one of the major challenges for land degradation despite several conservation measures. Here we present a novel technique of gully erosion susceptibility mapping by employing EBO (Eco-geography based optimization) with its ensembles: Bagging, Dagging, and Decorate. The EBO and its ensembles model were evaluated by various statistical approaches such as SST (sensitivity), PPV (positive predictive values), NPV (negative predictive values), SPF (specificity), ACC (accuracy), RMSE (root mean square error) and Cohen's Kappa model. We measured the morphological characteristics and chemical weathering in addition to gully head cut that is responsible for surface soil erosion where chemical weathering indirectly increases gullying process. The AUC values of EBO (Eco-biogeography-based optimization), EBO-Bagging, EBO-Dagging and EBO-Decorate for training datasets are 0.969, 0.915, 0.954 and 0.920 respectively. The AUC values of EBO, EBO-Bagging, EBO-Dagging and EBO-Decorate for validation datasets are 0.934, 0.901, 0.912 and 0.842 respectively. We apply this technique in the Kangsabati catchment area where we found that the upper part is more vulnerable to gullying leading to land degradation. Truer novel technique would aid in identifying land degradation-prone areas, and in formulating better strategies for better land use.

Network Information
Related Authors (5)
Kate Saenko

287 papers, 39K citations

77% related
Sergio Guadarrama

68 papers, 35.6K citations

75% related
Marcus Rohrbach

132 papers, 26.1K citations

74% related
Peter Stone

1.2K papers, 79.7K citations

73% related
Trevor Darrell

678 papers, 181.1K citations

71% related
Performance
Metrics

Author's H-index: 86

No. of papers from the Author in previous years
YearPapers
20221
20217
202012
201917
201810
201713