Amazon.com

About: Amazon.com is a company organization based out in Seattle, Washington, United States. It is known for research contribution in the topics: Computer science & Service (business). The organization has 13363 authors who have published 17317 publications receiving 266589 citations.

A Distributional Perspective on Reinforcement Learning

Abstract: In this paper we argue for the fundamental importance of the value distribution: the distribution of the random return received by a reinforcement learning agent. This is in contrast to the common approach to reinforcement learning which models the expectation of this return, or value. Although there is an established body of literature studying the value distribution, thus far it has always been used for a specific purpose such as implementing risk-aware behaviour. We begin with theoretical results in both the policy evaluation and control settings, exposing a significant distributional instability in the latter. We then use the distributional perspective to design a new algorithm which applies Bellman's equation to the learning of approximate value distributions. We evaluate our algorithm using the suite of games from the Arcade Learning Environment. We obtain both state-of-the-art results and anecdotal evidence demonstrating the importance of the value distribution in approximate reinforcement learning. Finally, we combine theoretical and empirical evidence to highlight the ways in which the value distribution impacts learning in the approximate setting.

Distributed storage system with web services client interface

Abstract: A distributed, web-services based storage system A system may include a web services interface configured to receive, according to a web services protocol, a given client request for access to a given data object, the request including a key value corresponding to the object The system may also include storage nodes configured to store replicas of the objects, where each replica is accessible via a respective unique locator value, and a keymap instance configured to store a respective keymap entry for each object For the given object, the respective keymap entry includes the key value and each locator value corresponding to replicas of the object A coordinator may receive the given client request from the web services interface, responsively access the keymap instance to identify locator values corresponding to the key value and, for a particular locator value, retrieve a corresponding replica from a corresponding storage node

FEVER: a large-scale dataset for Fact Extraction and VERification

Abstract: In this paper we introduce a new publicly available dataset for verification against textual sources, FEVER: Fact Extraction and VERification. It consists of 185,445 claims generated by altering sentences extracted from Wikipedia and subsequently verified without knowledge of the sentence they were derived from. The claims are classified as Supported, Refuted or NotEnoughInfo by annotators achieving 0.6841 in Fleiss $\kappa$. For the first two classes, the annotators also recorded the sentence(s) forming the necessary evidence for their judgment. To characterize the challenge of the dataset presented, we develop a pipeline approach and compare it to suitably designed oracles. The best accuracy we achieve on labeling a claim accompanied by the correct evidence is 31.87%, while if we ignore the evidence we achieve 50.91%. Thus we believe that FEVER is a challenging testbed that will help stimulate progress on claim verification against textual sources.

System and methods for collaborative recommendations

Abstract: A recommendation service is disclosed which uses collaborative filtering techniques to recommend books to users of a Web site. The Web site includes a catalog of the various titles that can be purchased via the site. The recommendation service includes a database of titles that have previously been rated and that can therefore be recommended by the service using collaborative filtering methods. At least initially, the titles and title categories (genres) that are included within this database (and thus included within the service) are respective subsets of the titles and categories included within the catalog. As users browse the site to read about the various titles contained within the catalog, the users are presented with the option of rating specific titles, including titles that are not currently included within the service. The ratings information obtained from this process is used to automatically add new titles and categories to the service. The breadth of categories and titles covered by the service thus grows automatically over time, without the need for system administrators to manually collect and input ratings data. To establish profiles for new users of the service, the service presents new users with a startup list of titles, and asks the new users to rate a certain number of titles on the list. To increase the likelihood that new users will be familiar with these titles, the service automatically generates the startup list by identifying the titles that are currently the most popular, such as the titles that have been rated the most over the preceding week.