Amazon.com

About: Amazon.com is a company organization based out in Seattle, Washington, United States. It is known for research contribution in the topics: Computer science & Service (business). The organization has 13363 authors who have published 17317 publications receiving 266589 citations.

Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?

Abstract: While pretrained models such as BERT have shown large gains across natural language understanding tasks, their performance can be improved by further training the model on a data-rich intermediate task, before fine-tuning it on a target task. However, it is still poorly understood when and why intermediate-task training is beneficial for a given target task. To investigate this, we perform a large-scale study on the pretrained RoBERTa model with 110 intermediate-target task combinations. We further evaluate all trained models with 25 probing tasks meant to reveal the specific skills that drive transfer. We observe that intermediate tasks requiring high-level inference and reasoning abilities tend to work best. We also observe that target task performance is strongly correlated with higher-level abilities such as coreference resolution. However, we fail to observe more granular correlations between probing and target task performance, highlighting the need for further work on broad-coverage probing benchmarks. We also observe evidence that the forgetting of knowledge learned during pretraining may limit our analysis, highlighting the need for further work on transfer learning methods in these settings.

Method and apparatus for multi-destination item selection using motes

Abstract: Method and apparatus for multi-destination pick using motes. In embodiments, each receptacle may be assigned to a destination and may have a mote that may include an indicator that may be activated by a control system to indicate to the agent that the receptacle is the destination receptacle for a picked item. The agent may then place the item in the indicated destination receptacle. A mote may include a communication interface for communicating with a control system and with other motes in an ad-hoc network. In one embodiment, the mote on the destination receptacle may be activated when the picked item is scanned by the agent. In some embodiments, each receptacle may also have a sensor that detects when an item is placed in the receptacle to deactivate the indicator and/or to verify that the item was placed in the correct receptacle.

Variational Reasoning for Question Answering with Knowledge Graph

Abstract: Knowledge graph (KG) is known to be helpful for the task of question answering (QA), since it provides well-structured relational information between entities, and allows one to further infer indirect facts. However, it is challenging to build QA systems which can learn to reason over knowledge graphs based on question-answer pairs alone. First, when people ask questions, their expressions are noisy (for example, typos in texts, or variations in pronunciations), which is non-trivial for the QA system to match those mentioned entities to the knowledge graph. Second, many questions require multi-hop logic reasoning over the knowledge graph to retrieve the answers. To address these challenges, we propose a novel and unified deep learning architecture, and an end-to-end variational learning algorithm which can handle noise in questions, and learn multi-hop reasoning simultaneously. Our method achieves state-of-the-art performance on a recent benchmark dataset in the literature. We also derive a series of new benchmark datasets, including questions for multi-hop reasoning, questions paraphrased by neural translation model, and questions in human voice. Our method yields very promising results on all these challenging datasets.

Toward an integrated monitoring framework to assess the effects of tropical forest degradation and recovery on carbon stocks and biodiversity

Abstract: Tropical forests harbor a significant portion of global biodiversity and are a critical component of the climate system. Reducing deforestation and forest degradation contributes to global climate-change mitigation efforts, yet emissions and removals from forest dynamics are still poorly quantified. We reviewed the main challenges to estimate changes in carbon stocks and biodiversity due to degradation and recovery of tropical forests, focusing on three main areas: (1) the combination of field surveys and remote sensing; (2) evaluation of biodiversity and carbon values under a unified strategy; and (3) research efforts needed to understand and quantify forest degradation and recovery. The improvement of models and estimates of changes of forest carbon can foster process-oriented monitoring of forest dynamics, including different variables and using spatially explicit algorithms that account for regional and local differences, such as variation in climate, soil, nutrient content, topography, biodiversity, disturbance history, recovery pathways, and socioeconomic factors. Generating the data for these models requires affordable large-scale remote-sensing tools associated with a robust network of field plots that can generate spatially explicit information on a range of variables through time. By combining ecosystem models, multiscale remote sensing, and networks of field plots, we will be able to evaluate forest degradation and recovery and their interactions with biodiversity and carbon cycling. Improving monitoring strategies will allow a better understanding of the role of forest dynamics in climate-change mitigation, adaptation, and carbon cycle feedbacks, thereby reducing uncertainties in models of the key processes in the carbon cycle, including their impacts on biodiversity, which are fundamental to support forest governance policies, such as Reducing Emissions from Deforestation and Forest Degradation.

Long-term thermal sensitivity of Earth’s tropical forests

Abstract: The sensitivity of tropical forest carbon to climate is a key uncertainty in predicting global climate change. Although short-term drying and warming are known to affect forests, it is unknown if such effects translate into long-term responses. Here, we analyze 590 permanent plots measured across the tropics to derive the equilibrium climate controls on forest carbon. Maximum temperature is the most important predictor of aboveground biomass (−9.1 megagrams of carbon per hectare per degree Celsius), primarily by reducing woody productivity, and has a greater impact per °C in the hottest forests (>32.2°C). Our results nevertheless reveal greater thermal resilience than observations of short-term variation imply. To realize the long-term climate adaptation potential of tropical forests requires both protecting them and stabilizing Earth’s climate.