Showing papers in "Communications of The ACM in 2009"

Scratch: programming for all

Abstract: "Digital fluency" should mean designing, creating, and remixing, not just browsing, chatting, and interacting.

Roofline: an insightful visual performance model for multicore architectures

Abstract: The Roofline model offers insight on how to improve the performance of software and hardware.

Lest we remember: cold-boot attacks on encryption keys

Abstract: Contrary to widespread assumption, dynamic RAM (DRAM), the main memory in most modern computers, retains its contents for several seconds after power is lost, even at room temperature and even if removed from a motherboard. Although DRAM becomes less reliable when it is not refreshed, it is not immediately erased, and its contents persist sufficiently for malicious (or forensic) acquisition of usable full-system memory images. We show that this phenomenon limits the ability of an operating system to protect cryptographic key material from an attacker with physical access to a machine. It poses a particular threat to laptop users who rely on disk encryption: we demonstrate that it could be used to compromise several popular disk encryption products without the need for any special devices or materials. We experimentally characterize the extent and predictability of memory retention and report that remanence times can be increased dramatically with simple cooling techniques. We offer new algorithms for finding cryptographic keys in memory images and for correcting errors caused by bit decay. Though we discuss several strategies for mitigating these risks, we know of no simple remedy that would eliminate them.

Formal verification of a realistic compiler

Abstract: This paper reports on the development and formal verification (proof of semantic preservation) of CompCert, a compiler from Clight (a large subset of the C programming language) to PowerPC assembly code, using the Coq proof assistant both for programming the compiler and for proving its correctness. Such a verified compiler is useful in the context of critical software and its formal verification: the verification of the compiler guarantees that the safety properties proved on the source code hold for the executable compiled code as well.

Eventually consistent

Abstract: Building reliable distributed systems at a worldwide scale demands trade-offs between consistency and availability.

The pathologies of big data

Abstract: Scale up your datasets enough and your apps come undone. What are the typical problems and where do the bottlenecks surface?

Automatically profiling the author of an anonymous text

Abstract: ImagIne that you have been gIven an Important text of unknown authorship, and wish to know as much as possible about the unknown author (demographics, personality, cultural background, among others), just by analyzing the given text. This authorship profiling problem is of growing importance in the current global information environment– applications abound in forensics, security, and commercial settings. For example, authorship profiling can help police identify characteristics of the perpetrator of a crime when there are too few (or too many) specific suspects to consider. Similarly, large corporations may be interested in knowing what types of people like or dislike their products, based on analysis of blogs and online product reviews. The question we therefore ask is: How much can we discern about the author of a text simply by analyzing the text itself? It turns out that, with varying degrees of accuracy, we can say a great deal indeed. Unlike the problem of authorship attribution (determining the author of a text from a given candidate set) discussed recently in these pages by Li, Zheng, and Chen authorship profiling does not begin with a set of writing samples from known candidate authors. Instead, we exploit the sociolinguistic observation that different groups of people speaking or writing in a particular genre and in a particular language use that language differently. That is, they vary in how often they use certain words or syntactic constructions (in addition to variation in pronunciation or intonation, for example). The particular profile dimensions we consider here are author gender, age,8 native language7 and personality.10

One laptop per child: vision vs. reality

Abstract: The vision is being overwhelmed by the reality of business, politics, logistics, and competing interests worldwide.

The status of the P versus NP problem

Abstract: It's one of the fundamental mathematical problems of our time, and its importance grows with the rise of powerful computers.

Global software development: where are the benefits?

Abstract: IntroductionGlobal Software Development (GSD) is increasingly becoming the normal practice in the software industry, readily evidenced by U.S. estimates that the value of the offshore software development market has increased 25-fold over the past 10 years, to the extent that one-quarter of U.S. spending on application development, integration and management services is expected to go off-shore according to recent predictions. There are many potential benefits that can arise from GSD. The most frequently cited one is that of reduced development costs due to the salary savings possible. Also, GSD can lead to reduced development duration due to greater time zone effectiveness as companies practice the so-called 'follow-the-sun' software development model. GSD also affords new opportunities for cross-site modularization of development work, potential access to a larger and better-skilled developer pool, and the possibility of greater innovation, learning and transfer of best practices. Finally, GSD can facilitate closer proximity to markets and customers.However, GSD also introduces a number of challenges in relation to communication, coordination and control of the development process. These arise due to the distances involved in three dimensions -- geographical, temporal, and socio-cultural (See Figure 1). As a consequence, much research and practice has focused on trying to find ways to overcome the GSD challenges identified in Figure 1. In the literature to date, the potential benefits of GSD are usually just mentioned very briefly, if they are mentioned at all, and the realization of these benefits seems to be more or less taken for granted. The primary focus instead is on how the problems inherent in GSD might be addressed. Here, we reverse this trend and focus instead on the benefits and the extent to which they are actually being realized in practice in three global companies practicing GSD.

Law and technologyThe end of the generative internet

Abstract: Exploring the expectations and implications for version 2.0 of the Net's new gated communities.

Who captures value in a global innovation network?: the case of Apple's iPod

Abstract: In novatIon Is often touted as a k ey drIv er o f economic growth.7, 11 However, when firms operate within production and innovation networks that span national and firm boundaries, the question arises as to who actually benefits from innovation. Is it the home country of the innovating firm, the country where the innovative product is manufactured, or the countries that supply the key high value components? This question recalls a debate in the early 1990s between Robert Reich and Laura Tyson. Reich 9 argued that the nationality of firms was less important than what activities they carried out in the U.S. or abroad; that is, a foreign company with a large U.S. workforce was more valuable to the U.S. than an American company whose workers were mostly abroad. Tyson12 responded that this case was actually quite rare, and that most companies retained a large share of high value activities in their home country; therefore ownership still mattered. The Reich-Tyson debate took place in an era when few had heard of the Internet or outsourcing, when vertically integrated multinational corporations still dominated most high-technology industries, and China and India were just taking their first tentative steps into the global hardware and software industries. Yet, while the world looks much different today, the core question debated then is just as relevant. For instance, an innovative product may be designed in one country, manufactured in another, with software developed in a third, and components sourced from several other countries. In such a case, how are the benefits distributed? To begin to unravel that question, we have moved away from macroeconomics and down to a micro-level analysis of one well-known innovative product, the Apple iPod. The iPod is designed and marketed by an American company, assembled by Taiwanese manufacturers in China, and includes key parts from Japanese, Korean and U.S. suppliers. So who captures the value generated by this hugely successful innovation? How much would the answer differ if the iPod were sold by Sony or Samsung instead of Apple, or if it were assembled in the U.S.? This paper develops a framework for analysis based on financial measures of value capture, and uses that framework to study one iPod model to provide one perspective on these questions.

The impact of the digital divide on e-government use

Abstract: In ternet use has been growIng steadIly for the past decade. In a recent report, the Pew Internet & American Life Project survey found that 75% of American adults use the Internet.7 As more citizens turn to the Internet as a medium for communication and commerce, the importance of e-government is rising. According to a 2004 report published by the same organization 77% of Internet users, utilized an e-government service in 2003; a 50% growth rate from 2002. Of those who access government information online, 77% search for information on tourism or recreation, 70% perform research for work or school, 63% download forms, and 62% retrieve information on policies or issues. Agencies are responding to the increase in demand by offering citizens more opportunities to interact with the government online. According to Pulliam, agency spending on e-government initiatives is expected to grow 6.9% annually to $5.8 billion by 2009. Defense Department agencies are expected to spend $152 million in 2009 on e-government efforts, up from $115 million in 2004. Further, civilian agencies are predicted to spend $114 million in 2009, up from $86 million in 2004. Proponents of e-government argue that online services offer constituents numerous benefits, including the improved responsiveness, efficiency and transparency of the public sector. Unfortunately, as a result of the digital divide, the entire population may not realize these benefits.

Probabilistic databases: diamonds in the dirt

Abstract: A wide range of applications have recently emerged that need to manage large, imprecise data sets. The reasons for imprecision in data are as diverse as the applications themselves: in sensor and RFID data, imprecision is due to measurement errors [15, 34]; in information extraction, imprecision comes from the inherent ambiguity in natural-language text [20, 26]; and in business intelligence, imprecision is tolerated because of the high cost of data cleaning [5]. In some applications, such as privacy, it is a requirement that the data be less precise. For example, imprecision is purposely inserted to hide sensitive attributes of individuals so that the data may be published [30]. Imprecise data has no place in traditional, precise database applications like payroll and inventory, and so, current database management systems are not prepared to deal with it. In contrast, the newly emerging applications offer value precisely because they query, search, and aggregate large volumes of imprecise data to find the “diamonds in the dirt”. This wide-variety of new applications points to the need for generic tools to manage imprecise data. In this paper, we survey the state of the art of techniques that handle imprecise data, by modeling it as probabilistic data [2–4,7,12,15,23,27,36]. A probabilistic database management system, or ProbDMS, is a system that stores large volumes of probabilistic data and supports complex queries. A ProbDMS may also need to perform some additional tasks, such as updates or recovery, but these do not differ from those in conventional database management systems and will not be discussed here. The major challenge in a ProbDMS is that it needs both to scale to large data volumes, a core competence of database management systems, and to do probabilistic inference, which is a problem studied in AI. While many scalable data management systems exists, probabilistic inference is a hard problem [35], and current systems do not scale to the same extent as data management systems do. To address this challenge, researchers have focused on the specific

Spamalytics: an empirical analysis of spam marketing conversion

Abstract: Spam-based marketing is a curious beast. We all receive the advertisements---"Excellent hardness is easy!"---but few of us have encountered a person who admits to following through on this offer and making a purchase. And yet, the relentlessness by which such spam continually clogs Internet inboxes, despite years of energetic deployment of antispam technology, provides undeniable testament that spammers find their campaigns profitable. Someone is clearly buying. But how many, how often, and how much?Unraveling such questions is essential for understanding the economic support for spam and hence where any structural weaknesses may lie. Unfortunately, spammers do not file quarterly financial reports, and the underground nature of their activities makes third-party data gathering a challenge at best. Absent an empirical foundation, defenders are often left to speculate as to how successful spam campaigns are and to what degree they are profitable. For example, IBM's Joshua Corman was widely quoted as claiming that spam sent by the Storm worm alone was generating "millions and millions of dollars every day."1 While this claim could in fact be true, we are unaware of any public data or methodology capable of confirming or refuting it.The key problem is our limited visibility into the three basic parameters of the spam value proposition: the cost to send spam, offset by the "conversion rate" (probability that an email sent will ultimately yield a "sale"), and the marginal profit per sale. The first and last of these are self-contained and can at least be estimated based on the costs charged by third-party spam senders and through the pricing and gross margins offered by various Interne marketing "affiliate programs."a However, the conversion rate depends fundamentally on group actions---on what hundreds of millions of Internet users do when confronted with a new piece of spam---and is much harder to obtain. While a range of anecdotal numbers exist, we are unaware of any well-documented measurement of the spam conversion rate.bIn part, this problem is methodological. There are no apparent methods for indirectly measuring spam conversion. Thus, the only obvious way to extract this data is to build an e-commerce site, market it via spam, and then record the number of sales. Moreover, to capture the spammer's experience with full fidelity, such a study must also mimic their use of illicit botnets for distributing email and proxying user responses. In effect, the best way to measure spam is to be a spammer.In this paper, we have effectively conducted this study, though sidestepping the obvious legal and ethical problems associated with sending spam.c Critically, our study makes use of an existing spamming botnet. By infiltrating the botnet parasitically, we convinced it to modify a subset of the spam it already sends, thereby directing any interested recipients to Web sites under our control, rather than those belonging to the spammer. In turn, our Web sites presented "defanged" versions of the spammer's own sites, with functionality removed that would compromise the victim's system or receive sensitive personal information such as name, address or credit card information.Using this methodology, we have documented three spam campaigns comprising over 469 million emails. We identified how much of this spam is successfully delivered, how much is filtered by popular antispam solutions, and, most importantly, how many users "click-through" to the site being advertised (response rate) and how many of those progress to a "sale" or "infection" (conversion rate).The remainder of this paper is structured as follows. Section 2 describes the economic basis for spam and reviews prior research in this area. Section 4 describes our experimental methodology for botnet infiltration. Section 5 describes our spam filtering and conversion results, Section 6 analyzes the effects of blacklisting on spam delivery, and Section 7 analyzes the possible influences on spam responses. We synthesize our findings in Section 8 and conclude.

Steps toward self-aware networks

Abstract: Network software adapts to user needs and load variations and failures to provide reliable communications in largely unknown networks.

ViewpointWhy "open source" misses the point of free software

Abstract: Decoding the important differences in terminology, underlying philosophy, and value systems between two similar categories of software.

Why did your project fail

Abstract: IntroductionWe have been developing software since the 1960s but still have not learned enough to ensure that our software development projects are successful. Boehm2 suggested that realistic schedule and budgets together with a continuing steam of requirements changes are high risk factors. The Standish Group in 1994 noted that approximately 31% of corporate software development projects were cancelled before completion and 53% were challenged and cost 180% above their original estimate.13 Glass discussed 16 project disasters.5 He found that the failed projects he reviewed were mostly huge and that the failure factors were not just management factors but also included technical factors. Linberg in 1999 found that 20% of software projects failed, and that 46% experienced cost and schedule overruns or significantly reduced functionality.8 Later, Glass revisited failed projects and found that poor estimation was high on his list of failure factors.6In 2007 the Standish Group reported that 35% of software projects started in 2006 were successful compared with only 16% in the corresponding 1994 report; however, the 2007 CHAOS report still identifies 46% (53% in 1994) of software projects as challenged (having cost or time overruns or not fully meeting user's requirements) and 19% (31% in 1994) as outright failures.12 The validity of the Standish Group findings has been questioned as not consistent with cost overrun results of other surveys.7 Jorgensen and Molokken-Ostvold suggested that there are serious problems with the way the Standish Group conducted their research and that the findings were biased toward reports of failure because a random sample of top IT executives was asked to share failure stories when mailed confidential surveys.Recently Charette4 commented that "billions of dollars are wasted each year on failed software projects" and that "we have a dismal history of projects that have gone awry."4 Charette suggests that from 5%-15% of projects will be abandoned before or shortly after delivery as hopelessly inadequate.4 Other recent studies, suggest various failure rates for software development projects up to 85%.7 Stories of software failure capture public attention and in general there is a perception that software quality is not improving but getting worse.Developing software systems is an expensive, and often a difficult process. Although there are many guidelines for successful software development,9,11 few project post-mortems are conducted, and little understanding is gained from the results of past projects. The project manager (PM) and the development team must deal with many pressures from project stakeholders (for example, upper level management, marketing, accounting, customers, and users) during the software development process. These pressures impact both the cost and the quality of the software produced. There are generally more than one or two reasons for a software project to fail, and it usually is a combination of technical, project management and business decisions. Many software project failure factors have been described in the literature.1-13 However, most organizations do not see preventing failure as an urgent matter. It is not understood why this attitude persists.4Because most of the literature is based on a handful of failed project case studies, in our research we analyze 70 failed software projects to determine those practices that affected project outcome. We are interested in providing, from the perspective of software practitioners, quantitative evidence targeting those aspects of the development process that contribute to project failure. Essentially, we are interested in updating the results of prior studies and testing the validity of previously reported anecdotal evidence. To date, no one has taken a set of project data and teased it out to identify, for a whole group of such projects, the most common failure factors. We are interested in everyday projects that are not high profile enough to be reported in the literature. Our work builds on that previously reported by Boehm,2 Charette,4 Glass,5,6 Jorgensen and Molokken,7 Linberg,8 and the Standish Group,12,13 among others.

Reflecting human values in the digital age

Abstract: HCI experts must broaden the field's scope and adopt new methods to be useful in 21st-century sociotechnical environments.

Four billion little brothers?: privacy, mobile phones, and ubiquitous data collection

Abstract: Participatory sensing technologies could improve our lives and our communities, but at what cost to our privacy?

Model checking: algorithmic verification and debugging

Abstract: Turing Lecture from the winners of the 2007 ACM A.M. Turing Award.In 1981, Edmund M. Clarke and E. Allen Emerson, working in the USA, and Joseph Sifakis working independently in France, authored seminal papers that founded what has become the highly successful field of model checking. This verification technology provides an algorithmic means of determining whether an abstract model---representing, for example, a hardware or software design---satisfies a formal specification expressed as a temporal logic (TL) formula. Moreover, if the property does not hold, the method identifies a counterexample execution that shows the source of the problem.The progression of model checking to the point where it can be successfully used for complex systems has required the development of sophisticated means of coping with what is known as the state explosion problem. Great strides have been made on this problem over the past 28 years by what is now a very large international research community. As a result many major hardware and software companies are beginning to use model checking in practice. Examples of its use include the verification of VLSI circuits, communication protocols, software device drivers, real-time embedded systems, and security algorithms.The work of Clarke, Emerson, and Sifakis continues to be central to the success of this research area. Their work over the years has led to the creation of new logics for specification, new verification algorithms, and surprising theoretical results. Model checking tools, created by both academic and industrial teams, have resulted in an entirely novel approach to verification and test case generation. This approach, for example, often enables engineers in the electronics industry to design complex systems with considerable assurance regarding the correctness of their initial designs. Model checking promises to have an even greater impact on the hardware and software industries in the future. ---Moshe Y. Vardi, Editor-in-Chief

Computing needs time

Abstract: The passage of time is essential to ensuring the repeatability and predictability of software and networks in cyber-physical systems.

Boolean satisfiability from theoretical hardness to practical success

Abstract: Satisfiability solvers can now be effectively deployed in practical applications.

The profession of ITBeyond computational thinking

Abstract: If we are not careful, our fascination with "computational thinking" may lead us back into the trap we are trying to escape.

An overview of IT service management

Abstract: “To the majority of computer scientists, whether in academia or industry, the term “services” is associated with Web services and service-oriented architectures. However, there is a broader story to be told of the remarkable growth of the service sector, which has come to dominate economic activity in most advanced economies over the last 50 years. ... The opportunity to innovate in services, to realize business and societal value from knowledge about service, to research, develop, and deliver new information services and business services, has never been greater.”

Improving performance on the internet

Abstract: Given the Internet's bottlenecks, how can we build fast, scalable, content-delivery systems?

The metropolis model a new logic for development of crowdsourced systems

Abstract: It takes a city of developers to build a big system that is never done.

CTO Roundtable: Cloud Computing

Abstract: The age of cloud computing has begun. How can companies take advantage of the new opportunities it provides?

Engineering rural development

Abstract: Information systems enable rural development by increasing the accountability of nongovernmental organizations.

Mobile phones in the classroom: if you can't beat them, join them

Abstract: Classroom interaCtivity has a number of significant benefits: it promotes an active learning environment, provides valuable feedback for lecturers, increases student motivation, and enables a learning community. 6 On the other hand, interactive activities for large classes (over 100 students) have proven to be quite difficult and, often, inefficient. 3 During the past six years the rapid proliferation of mobile devices, particularly cellular phones, has presented an opportunity to develop new interactive classroom systems which have the potential to enhance students' learning experience. The present challenge for researchers is to go beyond anecdotal perceptions and obtain empirical evidence about the impact of these technologies in the classroom. This paper describes a novel application of short-message-services (SMS) for large-class interactivity, and assesses its impact on the learning experiences of 1200 students in a large undergraduate class. The traditional lecture theatre environment has provided universities with a cost effective and scaleable means of teaching students. However, this has come at the price of making interaction difficult and inefficient, leading to reduced student engagement, motivation and learning. 3 Classroom Feedback Systems (CFS) provide one possible technological mechanism that can efficiently enable interaction in large classes. Known by various names (e.g., " clickers ") and produced commercially by a range of vendors , CFS technologies have been used since the sixties to allow students to respond to questions and have the results processed and displayed for use by the lecturer and the class as a whole. The more sophisticated CFSs provide the ability to answer a range of question types, from simple yes/no through to detailed responses, free-form questions and role-playing. Current platforms range from small infra-red units, through radio units, to the use of Web systems accessed by wireless personal digital assistants (PDAs) or laptops. These systems are generally well regarded by students when they are used. Numerous case studies have described the use of CFS technologies in disciplines ranging from the physical sciences through mathematics, accountancy and literature. A variety of positive outcomes from the use of CFS technologies have been reported including improved understanding of important concepts, 6, 7 increased student engagement and participation, 3, 7 improved quality of discussion in the classroom 7 and improved teacher awareness of student difficulties. 7 A clearer perception of their students' current level of understanding allows instructors to adjust their teaching appropriately. However, CFSs are not a panacea. Using CFS technology without specific pedagogical …