Data Mining - Concepts and Techniques.

User Generated Content (UGC) is re-shaping the way people watch video and TV, with millions of video producers and consumers. In particular, UGC sites are creating new viewing patterns and social interactions, empowering users to be more creative, and developing new business opportunities. To better understand the impact of UGC systems, we have analyzed YouTube, the world's largest UGC VoD system. Based on a large amount of data collected, we provide an in-depth study of YouTube and other similar UGC systems. In particular, we study the popularity life-cycle of videos, the intrinsic statistical properties of requests and their relationship with video age, and the level of content aliasing or of illegal content in the system. We also provide insights on the potential for more efficient UGC VoD systems (e.g. utilizing P2P techniques or making better use of caching). Finally, we discuss the opportunities to leverage the latent demand for niche videos that are not reached today due to information filtering effects or other system scarcity distortions. Overall, we believe that the results presented in this paper are crucial in understanding UGC systems and can provide valuable information to ISPs, site administrators, and content owners with major commercial and technical implications.

/pdf/i-tube-you-tube-everybody-tubes-analyzing-the-world-s-xhp615maf2.pdf

I tube, you tube, everybody tubes: analyzing the world's largest user generated content video system

This paper presents a traffic characterization study of the popular video sharing service, YouTube. Over a three month period we observed almost 25 million transactions between users on an edge network and YouTube, including more than 600,000 video downloads. We also monitored the globally popular videos over this period of time.In the paper we examine usage patterns, file properties, popularity and referencing characteristics, and transfer behaviors of YouTube, and compare them to traditional Web and media streaming workload characteristics. We conclude the paper with a discussion of the implications of the observed characteristics. For example, we find that as with the traditional Web, caching could improve the end user experience, reduce network bandwidth consumption, and reduce the load on YouTube's core server infrastructure. Unlike traditional Web caching, Web 2.0 provides additional meta-data that should be exploited to improve the effectiveness of strategies like caching.

/pdf/youtube-traffic-characterization-a-view-from-the-edge-53v2b6giu5.pdf

Youtube traffic characterization: a view from the edge

Video-on-demand over IP (VOD) is one of the best-known examples of "next-generation" Internet applications cited as a goal by networking and multimedia researchers. Without empirical data, researchers have generally relied on simulated models to drive their design and developmental efforts. In this paper, we present one of the first measurement studies of a large VOD system, using data covering 219 days and more than 150,000 users in a VOD system deployed by China Telecom. Our study focuses on user behavior, content access patterns, and their implications on the design of multimedia streaming systems. Our results also show that when used to model the user-arrival rate, the traditional Poisson model is conservative and overestimates the probability of large arrival groups. We introduce a modified Poisson distribution that more accurately models our observations. We also observe a surprising result, that video session lengths has a weak inverse correlation with the video's popularity. Finally, we gain better understanding of the sources of video popularity through analysis of a number of internal and external factors.

/pdf/understanding-user-behavior-in-large-scale-video-on-demand-2ercv8cu0q.pdf

Understanding user behavior in large-scale video-on-demand systems

As the distribution of the video over the Internet becomes main- stream and its consumption moves from the computer to the TV screen, user expectation for high quality is constantly increasing. In this context, it is crucial for content providers to understand if and how video quality affects user engagement and how to best invest their resources to optimize video quality. This paper is a first step towards addressing these questions. We use a unique dataset that spans different content types, including short video on demand (VoD), long VoD, and live content from popular video con- tent providers. Using client-side instrumentation, we measure quality metrics such as the join time, buffering ratio, average bitrate, rendering quality, and rate of buffering events.We quantify user engagement both at a per-video (or view) level and a per-user (or viewer) level. In particular, we find that the percentage of time spent in buffering (buffering ratio) has the largest impact on the user engagement across all types of content. However, the magnitude of this impact depends on the content type, with live content being the most impacted. For example, a 1% increase in buffering ratio can reduce user engagement by more than three minutes for a 90-minute live video event. We also see that the average bitrate plays a significantly more important role in the case of live content than VoD content.

/pdf/understanding-the-impact-of-video-quality-on-user-engagement-dbkb3mbmmp.pdf

Understanding the impact of video quality on user engagement

This paper provides an extensive analysis of pre-stored streaming media workloads, focusing on the client interactive behavior. We analyze four workloads that fall into three different domains, namely, education, entertainment video and entertainment audio. Our main goals are: (a) to identify qualitative similarities and differences in the typical client behavior for the three workload classes and (b) to provide data for generating realistic synthetic workloads.

/pdf/analyzing-client-interactivity-in-streaming-media-12n26qkqn4.pdf

Analyzing client interactivity in streaming media

Content pollution is a de facto problem in several currently popular Peer-to-Peer file sharing systems. Previously proposed strategies to fight content pollution include a peer reputation system named Scrubber as well as the alternative Credence object reputation system. This paper builds on previous work into three directions. First, it evaluates the cost-effectiveness of Scrubber and Credence in reducing pollution dissemination when polluters make use of collusion and Sybil attacks. Second, it modifies the Scrubber system to increase its effectiveness, in particular under collusion. Finally, it proposes a hybrid peer and object reputation system that combines the benefits of both strategies. We performed an extensive evaluation of all three systems, for various configurations and pollution mechanisms, as well as collusion and Sybil attacks. Our results show that the new hybrid system is much more effective and robust to malicious actions than any individual strategy, even in very uncooperative and unreliable peer communities.

Reputation Systems for Fighting Pollution in Peer-to-Peer File Sharing Systems

Recent studies reported a new form of malicious behavior in popular file-sharing Peer-to-Peer (P2P) systems, namely, content pollution, which reduces content availability, decreasing the confidence of users in such systems. This paper proposes Scrubber, a new descentralized peer reputation system that imposes severe and quick punishment to content polluters but also promotes peer rehabilitation. We evaluate the efficiency of Scrubber in reducing pollution dissemination via simulation, comparing it against the previously proposed Credence object reputation system as well as a system without reputation. Two pollution mechanisms, namely, decoy insertion and identifier corruption, are considered. Our results show that, for various scenarios, Scrubber is able to quickly reduce the fraction of daily downloads to polluted content to a small percentage. If compared to Credence, Scrubber has a much better convergence and competitive maximum efficiency, unless the fraction of peers that delete their polluted content in response to punishment (i.e., download request refusals) is very small (under 25%). In this case, Credence achieves a slightly higher efficiency in the long run.

Fighting pollution dissemination in peer-to-peer networks

Recent studies have reported a new form of malicious behavior in file-sharing Peer-to-Peer systems: content pollution. The dissemination of polluted content in a P2P system has the detrimental effect of reducing content availability, and ultimately, decreasing the confidence of users in such systems. Two potential strategies for polluting P2P content are decoy insertion, which consists of injecting corrupted copies of a file into the system, and hash corruption, which consists of injecting a corrupted file with the same hash code as a non-corrupted one. Polluted content disseminates through P2P networks because users typically do not delete the corrupted files that they download.This paper investigates the effectiveness of peer incentives to delete corrupted files in reducing the dissemination of polluted content, considering the two aforementioned pollution mechanisms. Our simulation results show that the effectiveness of incentives is highly dependent on the pollution mechanism. We show that for a pollution dissemintation techinique called hash corruption, only effective incentive mechanisms are able to avoid spreading of polluted content.

http://www.dcc.ufmg.br/~fabricio/download/treck-124.pdf

Impact of peer incentives on the dissemination of polluted content

The generation of realistic interactive synthetic streaming media workloads is of great importance for the effective evaluation of alternative media distribution techniques. This paper fills a gap left by previous studies and proposes a hierarchical model that captures key aspects of media user behavior and workloads, in particular, interactivity and heterogeneity. It also introduces GENIUS, a highly flexible and realistic streaming media workload generator that implements the proposed model and is parameterized with results from an extensive characterization of a rich set of real media workloads. Preliminary experiments show that our generator accurately captures workload aspects of key impact to system performance.

Cristiano Costa

Papers

Analyzing client interactivity in streaming media

Reputation Systems for Fighting Pollution in Peer-to-Peer File Sharing Systems

Fighting pollution dissemination in peer-to-peer networks

Impact of peer incentives on the dissemination of polluted content

GENIUS: a generator of interactive user media sessions