Data Mining Practical Machine Learning Tools and Techniques

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

Twitter, a microblogging service less than three years old, commands more than 41 million users as of July 2009 and is growing fast. Twitter users tweet about any topic within the 140-character limit and follow others to receive their tweets. The goal of this paper is to study the topological characteristics of Twitter and its power as a new medium of information sharing.We have crawled the entire Twitter site and obtained 41.7 million user profiles, 1.47 billion social relations, 4,262 trending topics, and 106 million tweets. In its follower-following topology analysis we have found a non-power-law follower distribution, a short effective diameter, and low reciprocity, which all mark a deviation from known characteristics of human social networks [28]. In order to identify influentials on Twitter, we have ranked users by the number of followers and by PageRank and found two rankings to be similar. Ranking by retweets differs from the previous two rankings, indicating a gap in influence inferred from the number of followers and that from the popularity of one's tweets. We have analyzed the tweets of top trending topics and reported on their temporal behavior and user participation. We have classified the trending topics based on the active period and the tweets and show that the majority (over 85%) of topics are headline news or persistent news in nature. A closer look at retweets reveals that any retweeted tweet is to reach an average of 1,000 users no matter what the number of followers is of the original tweet. Once retweeted, a tweet gets retweeted almost instantly on next hops, signifying fast diffusion of information after the 1st retweet.To the best of our knowledge this work is the first quantitative study on the entire Twittersphere and information diffusion on it.

https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=7104&context=sis_research

What is Twitter, a social network or a news media?

The term “Linked Data” refers to a set of best practices for publishing and connecting structured data on the Web. These best practices have been adopted by an increasing number of data providers over the last three years, leading to the creation of a global data space containing billions of assertions— the Web of Data. In this article, the authors present the concept and technical principles of Linked Data, and situate these within the broader context of related technological developments. They describe progress to date in publishing Linked Data on the Web, review applications that have been developed to exploit the Web of Data, and map out a research agenda for the Linked Data community as it moves forward.

/pdf/linked-data-the-story-so-far-2a9zvmaysa.pdf

Linked Data - the story so far

Obesity now affects one in five children in the United States. Discrimination against overweight children begins early in childhood and becomes progressively institutionalized. Because obese children tend to be taller than their nonoverweight peers, they are apt to be viewed as more mature. The inappropriate expectations that result may have an adverse effect on their socialization. Many of the cardiovascular consequences that characterize adult-onset obesity are preceded by abnormalities that begin in childhood. Hyperlipidemia, hypertension, and abnormal glucose tolerance occur with increased frequency in obese children and adolescents. The relationship of cardiovascular risk factors to visceral fat independent of total body fat remains unclear. Sleep apnea, pseudotumor cerebri, and Blount's disease represent major sources of morbidity for which rapid and sustained weight reduction is essential. Although several periods of increased risk appear in childhood, it is not clear whether obesity with onset early in childhood carries a greater risk of adult morbidity and mortality. Obesity is now the most prevalent nutritional disease of children and adolescents in the United States. Although obesity-associated morbidities occur more frequently in adults, significant consequences of obesity as well as the antecedents of adult disease occur in obese children and adolescents. In this review, I consider the adverse effects of obesity in children and adolescents and attempt to outline areas for future research. I refer to obesity as a body mass index greater than the 95th percentile for children of the same age and gender.

Health Consequences of Obesity in Youth: Childhood Predictors of Adult Disease

We describe mod_oai, an Apache 2.0 module that implements the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). OAIPMH is the de facto standard for metadata exchange in digital libraries and allows repositories to expose their contents in a structured, application-neutral format with semantics optimized for accurate incremental harvesting. Current implementations of OAI-PMH are either separate applications that access an existing repository, or are built-in to repository software packages. mod_oai is different in that it optimizes harvesting web content by building OAI-PMH capability into the Apache server. We discuss the implications of adding harvesting capability to an Apache server and describe our initial experimental results accessing a departmental web site using both web crawling and OAIPMH harvesting techniques.

mod_oai: An Apache Module for Metadata Harvesting

When viewing an archived page using the archive's user interface (UI), the user selects a datetime to view from a list. The archived web page, if available, is then displayed. From this display, the web archive UI attempts to simulate the web browsing experience by smoothly transitioning between archived pages. During this process, the target datetime changes with each link followed, potentially drifting away from the datetime originally selected. For sparsely archived resources, this almost transparent drift can be many years in just a few clicks. We conducted 200,000 acyclic walks of archived pages, following up to 50 links per walk, comparing the results of two target datetime policies. The Sliding Target policy allows the target datetime to change as it does in archive UIs such as the Internet Archive's Wayback Machine. The Sticky Target policy, represented by the Memento API, keeps the target datetime the same throughout the walk. We found that the Sliding Target policy drift increases with the number of walk steps, number of domains visited, and choice (number of links available). However, the Sticky Target policy controls temporal drift, holding it to $$<$$<30 days on average regardless of walk length or number of domains visited. The Sticky Target policy shows some increase as choice increases, but this may be caused by other factors. We conclude that based on walk length, the Sticky Target policy generally produces at least 30 days less drift than the Sliding Target policy.

Evaluating sliding and sticky target policies by measuring temporal drift in acyclic walks through a web archive

This is the second paper in D-Lib Magazine about the ResourceSync effort conducted by the National Information Standards Organization (NISO) and the Open Archives Initiative (OAI). The first part provided a perspective on the resource synchronization problem and introduced a template that organized possible components of a resource synchronization framework in a modular manner. This paper details a technical framework devised using that template.

https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1137&context=computerscience_fac_pubs

A Technical Framework for Resource Synchronization

In this work we proposed MementoMap, a flexible and adaptive framework to summarize holdings of a web archive efficiently. We described a simple, yet extensible, file format suitable for MementoMap. We used the complete index of the Arquivo.pt comprising 5B mementos (archived web pages/files) to understand the nature and shape of its holdings. We generated MementoMaps with varying amount of detail from its HTML pages that have an HTTP status code of 200 OK. Additionally, we designed a single-pass, memory-efficient, and parallelization-friendly algorithm to compact a large MementoMap into a small one and an in-file binary search method for efficient lookup. We analyzed more than three years of MemGator (a Memento aggregator) logs to understand the response behavior of 14 public web archives. We evaluated MementoMaps by measuring their Accuracy using 3.3M unique URIs from MemGator logs. We found that a MementoMap of less than 1.5% Relative Cost (as compared to the comprehensive listing of all the unique original URIs) can correctly identify the presence or absence of 60% of the lookup URIs in the corresponding archive while maintaining 100% Recall (i.e., zero false negatives).

Mementomap framework for flexible and adaptive web archive profiling

Web archives have become a significant repository of our recent history and cultural heritage. Archival integrity and accuracy is a precondition for future cultural research. Currently, there are no quantitative or content-based tools that allow archivists to judge the quality of the Web archive captures. In this paper, we address the problems of detecting off-topic pages in Web archive collections. We evaluate six different methods to detect when the page has gone off-topic through subsequent captures. Those predicted off-topic pages will be presented to the collection’s curator for possible elimination from the collection or cessation of crawling. We created a gold standard data set from three Archive-It collections to evaluate the proposed methods at different thresholds. We found that combining cosine similarity at threshold 0.10 and change in size using word count at threshold $-$0.85 performs the best with accuracy = 0.987, $F_{1}$ score = 0.906, and AUC = 0.968. We evaluated the performance of the proposed method on several Archive-It collections. The average precision of detecting the off-topic pages is 0.92.

/pdf/detecting-off-topic-pages-in-web-archives-4xj28svr0u.pdf

Michael L. Nelson

Papers

mod_oai: An Apache Module for Metadata Harvesting

Evaluating sliding and sticky target policies by measuring temporal drift in acyclic walks through a web archive

A Technical Framework for Resource Synchronization

Mementomap framework for flexible and adaptive web archive profiling

Detecting Off-Topic Pages in Web Archives