There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

We explore the effect of dimensionality on the nearest neighbor problem. We show that under a broad set of conditions (much broader than independent and identically distributed dimensions), as dimensionality increases, the distance to the nearest data point approaches the distance to the farthest data point. To provide a practical perspective, we present empirical results on both real and synthetic data sets that demonstrate that this effect can occur for as few as 10-15 dimensions. These results should not be interpreted to mean that high-dimensional indexing is never meaningful; we illustrate this point by identifying some high-dimensional workloads for which this effect does not occur. However, our results do emphasize that the methodology used almost universally in the database literature to evaluate high-dimensional indexing techniques is flawed, and should be modified. In particular, most such techniques proposed in the literature are not evaluated versus simple linear scan, and are evaluated over workloads for which nearest neighbor is not meaningful. Often, even the reported experiments, when analyzed carefully, show that linear scan would outperform the techniques being proposed on the workloads studied in high (10-15) dimensionality!.

When is nearest neighbor meaningful

SUMMARY Since 1984, the Condor project has enabled ordinary users to do extraordinary computing. Today, the project continues to explore the social and technical problems of cooperative computing on scales ranging from the desktop to the world-wide computational Grid. In this paper, we provide the history and philosophy of the Condor project and describe how it has interacted with other projects and evolved along with the field of distributed computing. We outline the core components of the Condor system and describe how the technology of computing must correspond to social structures. Throughout, we reflect on the lessons of experience and chart the course travelled by research ideas as they grow into production systems. Copyright c � 2005 John Wiley & Sons, Ltd.

/pdf/distributed-computing-in-practice-the-condor-experience-1xldnwf3zo.pdf

Distributed computing in practice: the Condor experience

Food systems contribute 19%–29% of global anthropogenic greenhouse gas (GHG) emissions, releasing 9,800–16,900 megatonnes of carbon dioxide equivalent (MtCO2e) in 2008. Agricultural production, including indirect emissions associated with land-cover change, contributes 80%–86% of total food system emissions, with significant regional variation. The impacts of global climate change on food systems are expected to be widespread, complex, geographically and temporally variable, and profoundly influenced by socioeconomic conditions. Historical statistical studies and integrated assessment models provide evidence that climate change will affect agricultural yields and earnings, food prices, reliability of delivery, food quality, and, notably, food safety. Low-income producers and consumers of food will be more vulnerable to climate change owing to their comparatively limited ability to invest in adaptive institutions and technologies under increasing climatic risks. Some synergies among food security, adaptati...

https://sustainabledevelopment.un.org/content/documents/881annurev.pdf

Climate Change and Food Systems

Power Aware Routing in Mobile Ad Hoc Networks

A new paradigm: Data-aware scheduling in grid computing

Todays scientific applications have huge data requirements which continue to increase drastically every year. These data are generally accessed by many users from all across the the globe. This implies a major necessity to move huge amounts of data around wide area networks to complete the computation cycle, which brings with it the problem of efficient and reliable data placement. The current approach to solve this problem of data placement is either doing it manually, or employing simple scripts which do not have any automation or fault tolerance capabilities. Our goal is to make data placement activities first class citizens in the Grid just like the computational jobs. They will be queued, scheduled, monitored, managed, and even check-pointed. More importantly, it will be made sure that they complete successfully and without any human interaction. We also believe that data placement jobs should be treated differently from computational jobs, since they may have different semantics and different characteristics. For this purpose, we have developed Stork, a scheduler for data placement activities in the grid.

Stork: making data placement a first class citizen in the grid

The Condor project began in 1988 and has evolved into a feature-rich batch system that targets high-throughput computing; that is, Condor ([262], [414]) focuses on providing reliable access to computing over long periods of time instead of highly tuned, high-performance computing for short periods of time or a small number of applications.

/pdf/workflow-management-in-condor-4leocuyuiq.pdf

Workflow Management in Condor

Data placement is an essential part of today's distributed applications since moving the data close to the application has many benefits. The increasing data requirements of both scientific and commercial applications, and collaborative access to these data make it even more important. In the current approach, data placement is regarded as a side affect of computation. Our goal is to make data placement a first class citizen in distributed computing systems just like the computational jobs. They will be queued, scheduled, monitored, managed, and even checkpointed. Since data placement jobs have different characteristics than computational jobs, they cannot be treated in the exact same way as computational jobs. For this purpose, we are proposing a framework which can be considered as a ''data placement subsystem'' for distributed computing systems, similar to the I/O subsystem in operating systems. This framework includes a specialized scheduler for data placement, a high level planner aware of data placement jobs, a resource broker/policy enforcer and some optimization tools. Our system can perform reliable and efficient data placement, it can recover from all kinds of failures without any human intervention, and it can dynamically adapt to the environment at the execution time.

/pdf/a-framework-for-reliable-and-efficient-data-placement-in-2nph4ud9ft.pdf

A framework for reliable and efficient data placement in distributed computing systems

The trend in scientific, as well as commercial, applications from a diverse range of fields has been towards being more and more data-intensive over time. Data Intensive Distributed Computing: Challenges and Solutions for Large-scale Information Management focuses on the challenges of distributed systems imposed by data intensive applications and on the different state-of-the-art solutions proposed to overcome such challenges. Providing hints on how to manage low-level data handling issues when performing data intensive distributed computing, this publication is ideal for scientists, researchers, engineers, and application developers, alike. With the knowledge of the correct data management techniques for their applications, readers will be able to focus on their primary goal, assured that their data management needs are handled reliably and efficiently.

Tevfik Kosar

Papers

A new paradigm: Data-aware scheduling in grid computing

Stork: making data placement a first class citizen in the grid

Workflow Management in Condor

A framework for reliable and efficient data placement in distributed computing systems

Data Intensive Distributed Computing: Challenges and Solutions for Large-scale Information Management