Emerging Web services, such as email, photo sharing, and web site archives, must preserve large volumes of quickly accessible data indefinitely into the future. The costs of doing so often determine whether the service is economically viable. We make the case that these applications' demands on large scale storage systems over long time horizons require us to reevaluate traditional system designs. We examine threats to long-lived data from an end-to-end perspective, taking into account not just hardware and software faults but also faults due to humans and organizations. We present a simple model of long-term storage failures that helps us reason about various strategies for addressing some of these threats. Using this model we show that the most important strategies for increasing the reliability of long-term storage are detecting latent faults quickly, automating fault repair to make it cheaper and faster, and increasing the independence of data replicas.

/pdf/a-fresh-look-at-the-reliability-of-long-term-digital-storage-3yfe0reuqs.pdf

A fresh look at the reliability of long-term digital storage

Create, curate, re-use: the expanding life course of digital research data

Observations of solar and lunar eclipses recorded in a variety of sources between around 700 B.C. and A.D. 1600 have proved of considerable value in studying changes in the Earth’s rate of rotation. These variations are mainly produced by tides, but non-tidal effects, such as the continuing rise of land that was glaciated during the last ice-age, are also significant. Eclipse observations that are preserved from this 2300year interval originate largely from Babylon, China, Europe and the Arab lands. The most valuable data are of three main types: (i) timed measurements of solar eclipse contacts; (ii) lunar eclipse timings; and (iii) untimed reports of total or near-total solar eclipses. In all, several hundred useful records are extant. In analysing these observations, the primary objective is to investigate the cumulative effect of long-term changes in the Earth’s rate of spin, usually termed ∆T. This parameter is designated as the difference between Terrestrial Time (TT, a theoretically uniform time system based on the motion of the Moon and planets) and Universal Time (UT, time as defined by the Earth’s rotation, and determined from observation). Although changes in the length of the day over the historical period are of the order of milliseconds, the cumulative result is large. Between 700 B.C. and the present, almost one million days have elapsed, each marginally shorter than at present. As a result, in ancient times ∆T attained several hours. Such a significant time-interval can be easily measured even using observations that are crude by modern standards. The study of Earth’s rotation is important not only in geophysics. Improved determination of ∆T leads to more accurate calculation of the circumstances of early eclipses and is thus also of value in both the history of astronomy and chronology. Over the last 25 years, Leslie V. Morrison and I have jointly made extensive determinations of the parameter ∆T throughout the interval from 700 B.C. to the present-day. We have derived results over the period since A.D. 1600 mainly from occultations of stars by the Moon. However, at all previous dates we have concentrated exclusively on eclipse observations. In computing lunar positions, we have systematically adopted a value for the lunar tidal acceleration of –26 arcsec/cy. This is very close to the most recent results derived from lunar laser ranging. It should be emphasized that our earliest solutions for ∆T were based on rather limited data and only preliminary results were obtained. However, by around 1995, largely due to protracted searches for further eclipse observations, a fairly stable data set had been acquired. A series of spline functions was fitted to the numerous individual ∆T values derived from the observations. This led to our publication of an improved paper on the subject. Over the last decade, relatively few further observations of JHA, xxxix (2008)

How Reliable are Archaic Records of Large Solar Eclipses

The challenge of establishing the chronology of the early Zhou dynasty, especially the date of the Zhou Conquest of Shang, periodically arouses the intense interest of scholars. Down the centuries interest in the general problem has revived whenever advances in calendrical, astronomical, or textual knowledge seemed to offer the promise of a breakthrough. Extraordinary efforts have repeatedly been brought to bear, but until quite recently little progress was achieved in verifying benchmarks prior to 841 B.C., a date already regarded as secure in the time of Sima Qian toward the end of the second century B.C.

The Bamboo Annals revisited: problems of method in using the Chronicle as a source for the chronology of early Zhou, Part I

All astronomers use historical data. A large percentage of the papers presented at astronomy conferences and published in astronomical journals make use of observations made by other astronomers five or ten years before and which have been published in journal articles, on-line catalogues, or through the more ambiguous 'personal communications' and 'preprints' that pervade scientific literature. Depending upon the circumstances, astronomers may treat such observations critically, for example by evaluating the measurement techniques used by the initial observer, or as established 'facts' with known error estimates, such as the positions given in the standard star catalogues. We can trace similar activity back through history. For example, two of the major figures in Renaissance astronomy, Copernicus and Kepler, made extensive use of a run of more than thirty years of careful observations by the less well-known fifteenth-century astronomer Bernard Walther. I Similarly the medieval Egyptian astronomer Ibn Yunus tested contemporary astronomical tables for calculating eclipses through comparison with eclipses observed by himself and by other astronomers from the preceding two centuries.' But none of these activities falls into the category of applied historical astronomy, the subject of this review. 3 'Applied historical astronomy', as I understand the term, is concerned not just with using observations made by our astronomer predecessors, but also with using astronomical records that come from a different cultural setting to that in which we ourselves live. This manifests itself most frequently in having to deal with records written in other languages, often dead languages such as Akkadian or ancient Greek. But even where language is not an issue, applied historical astronomers have to cope with unfamiliar cultural settings, and address questions such as: Did the writer of a particular astronomical report intend it to be a faithful representation of an observation, or were other motivations, for example, literary or astrological, at play? Are particular sources containing astronomical records reliable? Can we always be sure that certain records report observations, or might they be the results of ancient calculation? Finding satisfactory answers to these important historical questions underpins all of applied historical astronomy.' Without them, we run the risk of constructing our theories on a foundation of sand. In this review I attempt to view applied historical astronomy from an historical perspective. In the first part I will discuss how astronomers in the past have used historical astronomical observations in their work, in effect looking to see which early astronomers were undertaking applied historical astronomy, what problems they faced, and how they overcame these problems. In the second, I will turn to recent research in applied historical astronomy and consider whether such work

Applied historical astronomy : an historical perspective.

Computer analysis of some ancient Chinese sunrise eclipse records to determine the earth's rotation rate

Of the forces changing the Earth’s rotation, tidal braking and postglacial rebound predominate at a timescale ≥102yr (HIDE and DICKEY, 1991; DICKEY, 1992). Analysis of ancient eclipse records has given values for the clock error ΔT and the time derivative of the Earth’s dynamic oblateness J 2 for the past 3,300 yr. Since ΔT = AT - UT = ct 2, where AT is Atomic (cesium clock) Time, UT is Universal (Earth rotation) Time, and t is the number of centuries before 1800, the oldest data have the most weight. Sunrise and sunset eclipses are especially valuable, as they can be retrospectively timed.

The Earth’s Palaeorotation, Postglacial Rebound and Lower Mantle Viscosity from Analysis of Ancient Chinese Eclipse Records

All 13 Shang dynasty oracle bone eclipse records have been uniquely matched to six solar and seven lunar eclipses in the 14th to 12th centuries B.C. The King Zhong Kang fifth year autumnal (October 16, 1876 B.C.) and King Yu third year “double sunset” (September 24, 1912 B.C.) eclipses confirm the accuracy of the revised Bamboo Annals Xia dynasty chronology (Nivison and Pang, Early China 15, 1990, 87-95). The eclipse dates are plotted against the number of generations before 841 B.C. (the earliest accurate historical date), the respective kings ruled. The curve of bestfit has both the strengths of verified royal genealogy — continuity — and eclipse dating — accuracy. It is 99% accurate, and can be confidently used as a foundation for building a detailed absolute chronology for the Xia, Shang and Zhou dynasties, an important project in China’s current Five-Year Plan (Song Jian, Sci. Tech. Daily, May 17, 1996; Wehrfritz, Newsweek, July 7, 1997).

https://link.springer.com/content/pdf/10.1007%2F978-94-015-9862-0_9.pdf

Astronomical Dating and Statistical Analysis of Ancient Eclipse Data

All 13 Shang dynasty oracle bone eclipse records have been uniquely matched to six solar and seven lunar eclipses in the 14th to 12th centuries B.C. The King Zhong Kang fifth year autumnal (October 16, 1876 B.C.) and King Yu third year "double sunset" (September 24, 1912 B.C.) eclipses confirm the accuracy of the revised Bamboo Annals Xia dynasty chronology (Nivison and Pang, Early China 15, 1990,87-95). The eclipse dates are plotted against the number of generations before 841 B.C. (the earliest accurate historical date), the respective kings ruled. The curve of best fit has both the strengths of verified royal genealogy continuity - and eclipse dating - accuracy. It is 99% accurate, and can be confidently used as a foundation for building a detailed absolute chronology for the Xia, Shang and Zhou dynasties, an important project in China's current Five-Year Plan (Song Jian, Sci. Tech. Daily, May 17, 1996; Wehrfritz, Newsweek, July 7, 1997).

https://link.springer.com/content/pdf/10.1007/978-94-015-9862-0_9.pdf

Hung-hsiang Chou

Papers

Computer analysis of some ancient Chinese sunrise eclipse records to determine the earth's rotation rate

The Earth’s Palaeorotation, Postglacial Rebound and Lower Mantle Viscosity from Analysis of Ancient Chinese Eclipse Records

Astronomical Dating and Statistical Analysis of Ancient Eclipse Data

1.9. Astronomical Dating and Statistical Analysis of Ancient Eclipse Data