“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

1. Word Frequencies. 2. Non-parametric models. 3. Parametric models. 4. Mixture distributions. 5. The Randomness Assumption. 6. Examples of Applications. A. List of Symbols. B. Solutions of the exercises. C. Software. D. Data sets. Bibliography. Index.

Word Frequency Distributions

Reinforcement learning (RL) has proven its worth in a series of artificial domains, and is beginning to show some successes in real-world scenarios. However, much of the research advances in RL are often hard to leverage in real-world systems due to a series of assumptions that are rarely satisfied in practice. We present a set of nine unique challenges that must be addressed to productionize RL to real world problems. For each of these challenges, we specify the exact meaning of the challenge, present some approaches from the literature, and specify some metrics for evaluating that challenge. An approach that addresses all nine challenges would be applicable to a large number of real world problems. We also present an example domain that has been modified to present these challenges as a testbed for practical RL research.

/pdf/challenges-of-real-world-reinforcement-learning-5k1yi7pmm7.pdf

Challenges of Real-World Reinforcement Learning

This is a survey of the science and practice of web crawling. While at first glance web crawling may appear to be merely an application of breadth-first-search, the truth is that there are many challenges ranging from systems concerns such as managing very large data structures to theoretical questions such as how often to revisit evolving content sources. This survey outlines the fundamental challenges and describes the state-of-the-art models and solutions. It also highlights avenues for future work.

https://www.microsoft.com/en-us/research/wp-content/uploads/2010/01/1500000017.pdf

Web Crawling

People often ask others for restaurant recommendations as a way to discover new dining experiences. This makes restaurant recommendation an exciting scenario for recommender systems and has led to substantial research in this area. However, most such systems behave very differently from a human when asked for a recommendation. The goal of this paper is to begin to reduce this gap. In particular, humans can quickly establish preferences when asked to make a recommendation for someone they do not know. We address this cold-start recommendation problem in an online learning setting. We develop a preference elicitation framework to identify which questions to ask a new user to quickly learn their preferences. Taking advantage of latent structure in the recommendation space using a probabilistic latent factor model, our experiments with both synthetic and real world data compare different types of feedback and question selection strategies. We find that our framework can make very effective use of online user feedback, improving personalized recommendations over a static model by 25% after asking only 2 questions. Our results demonstrate dramatic benefits of starting from offline embeddings, and highlight the benefit of bandit-based explore-exploit strategies in this setting.

/pdf/towards-conversational-recommender-systems-t6x4xs04bw.pdf

Towards Conversational Recommender Systems

Automatically generated content is ubiquitous in the web: dynamic sites built using the three-tier paradigm are good examples (e.g., commercial sites, blogs and other sites edited using web authoring software), as well as less legitimate spamdexing attempts (e.g., link farms, faked directories).Those pages built using the same generating method (template or script) share a common “look and feel” that is not easily detected by common text classification methods, but is more related to stylometry.In this work we study and compare several HTML style similarity measures based on both textual and extra-textual features in HTML source code. We also propose a flexible algorithm to cluster a large collection of documents according to these measures. Since the proposed algorithm is based on locality sensitive hashing (LSH), we first review this technique.We then describe how to use the HTML style similarity clusters to pinpoint dubious pages and enhance the quality of spam classifiers. We present an evaluation of our algorithm on the WEBSPAM-UK2006 dataset.

Tracking Web spam with HTML style similarities

We study a stochastic online learning scheme with partial feedback where the utility of decisions is only observable through an estimation of the environment parameters. We propose a generic pure-exploration algorithm, able to cope with various utility functions from multi-armed bandits settings to dueling bandits. The primary application of this setting is to offer a natural generalization of dueling bandits for situations where the environment parameters reflect the idiosyncratic preferences of a mixed crowd.

/pdf/generic-exploration-and-k-armed-voting-bandits-3t7dci7202.pdf

Generic Exploration and K-armed Voting Bandits

The labelling of training examples is a costly task in a supervised classification. Active learning strategies answer this problem by selecting the most useful unlabelled examples to train a predictive model. The choice of examples to label can be seen as a dilemma between the exploration and the exploitation over the data space representation. In this paper, a novel active learning strategy manages this compromise by modelling the active learning problem as a contextual bandit problem. We propose a sequential algorithm named Active Thompson Sampling (ATS), which, in each round, assigns a sampling distribution on the pool, samples one point from this distribution, and queries the oracle for this sample point label. Experimental comparison to previously proposed active learning algorithms show superior performance on a real application dataset.

/pdf/contextual-bandit-for-active-learning-active-thompson-3txnhjo4j4.pdf

Contextual Bandit for Active Learning: Active Thompson Sampling

Automatically generated content is ubiquitous in the web: dynamic sites built using the three-tier paradigm are good examples (e.g. commercial sites, blogs and other sites pow- ered by a web authoring software), as well as less legitimous spamdexing attempts (e.g. link farms, faked directories...). Those pages built using the same generating method (tem- plate or script) share a common \look and feel" that is not easily detected by common text classiflcation methods, but is more related to stylometry. In this paper, we present a (hidden) style similarity mea- sure based on extra-textual features in html source code. We also describe a method to clusterize a large collection of documents according to this measure. The clustering algo- rithm being based on flngerprints, we also give some recalls about flngerprinting. By conveniently sorting the generated clusters, one can ef- flciently track back instances of a particular automatic con- tent generation method among web pages collected using a crawler. This is particularly useful to detect pages across difierent sites sharing the same design | this is often a good hint of either spamdexing attempt or mirrored content.

/pdf/tracking-web-spam-with-hidden-style-similarity-4a4atx9ieo.pdf

Tracking Web Spam with Hidden Style Similarity

To address the contextual bandit problem, we propose an online random forest algorithm. The analysis of the proposed algorithm is based on the sample complexity needed to find the optimal decision stump. Then, the decision stumps are recursively stacked in a random collection of decision trees, BANDIT FOREST. We show that the proposed algorithm is optimal up to logarithmic factors. The dependence of the sample complexity upon the number of contextual variables is logarithmic. The computational cost of the proposed algorithm with respect to the time horizon is linear. These analytical results allow the proposed algorithm to be efficient in real applications, where the number of events to process is huge, and where we expect that some contextual variables, chosen from a large set, have potentially non-linear dependencies with the rewards. In the experiments done to illustrate the theoretical analysis, BANDIT FOREST obtain promising results in comparison with state-of-the-art algorithms.

/pdf/random-forest-for-the-contextual-bandit-problem-4iv1kbqsvi.pdf

Tanguy Urvoy

Papers

Tracking Web spam with HTML style similarities

Generic Exploration and K-armed Voting Bandits

Contextual Bandit for Active Learning: Active Thompson Sampling

Tracking Web Spam with Hidden Style Similarity

Random Forest for the Contextual Bandit Problem