The need for sound ecological science has escalated alongside the rise of the information age and “big data” across all sectors of society. Big data generally refer to massive volumes of data not readily handled by the usual data tools and practices and present unprecedented opportunities for advancing science and inform- ing resource management through data-intensive approaches. The era of big data need not be propelled only by “big science” – the term used to describe large-scale efforts that have had mixed success in the individual-driven culture of ecology. Collectively, ecologists already have big data to bolster the scientific effort – a large volume of distributed, high-value information – but many simply fail to contribute. We encourage ecologists to join the larger scientific community in global initiatives to address major scientific and societal problems by bringing their distributed data to the table and harnessing its collective power. The scientists who contribute such information will be at the forefront of socially relevant science – but will they be ecologists?

/pdf/big-data-and-the-future-of-ecology-1jqs4ev4s7.pdf

Big data and the future of ecology

The Open Archives Initiative (OAI) develops and promotes interoperabil ity solutions that aim to facilitate the efficient dissemination of content The roots of the OAI lie in the E-Print community Over the last year its focus has been extended to include all content providers This paper describes the recent history of the OAI - its origins in promoting E-Prints, the broadening of its focus, the details of its technical standard for metadata harvesting, the applications of this standard, and future plans

http://www.openarchives.org/documents/jcdl2001-oai.pdf

The open archives initiative: building a low-barrier interoperability framework

Ecology is evolving rapidly and increasingly changing into a more open, accountable, interdisciplinary, collaborative and data-intensive science. Discovering, integrating and analyzing massive amounts of heterogeneous data are central to ecology as researchers address complex questions at scales from the gene to the biosphere. Ecoinformatics offers tools and approaches for managing ecological data and transforming the data into information and knowledge. Here, we review the state-of-the-art and recent advances in ecoinformatics that can benefit ecologists and environmental scientists as they tackle increasingly challenging questions that require voluminous amounts of data across disciplines and scales of space and time. We also highlight the challenges and opportunities that remain.

/pdf/ecoinformatics-supporting-ecology-as-a-data-intensive-5433hch3p3.pdf

Ecoinformatics: supporting ecology as a data-intensive science.

Methods and apparatus for managing, finding and displaying objects such as digital images. Objects are tagged (“associated”) with descriptive textual and numeric data (“metadata”), and stored in a relational database from which they can be selected, sorted, and found. Tags can be defined by name, tag type, and associated attributes. Objects can be tagged by dropping a tag onto the object, or relating a database record for the tag to a database record for the object. Tagged objects can be searched for and displayed according to the degree to which their metadata matches the search criteria. Visual cues can indicate whether displayed objects match all, some but not all, or none of the search criteria. Database object distributions can be displayed as histograms or scatter plots, including timelines, calendars or maps. Object distributions can be used to search for objects or to limit search results for a previous search.

Digital media management apparatus and methods

Recent evolutions in computing science and web technology provide the environmental community with continuously expanding resources for data collection and analysis that pose unprecedented challenges to the design of analysis methods, workflows, and interaction with data sets. In the light of the recent UK Research Council funded Environmental Virtual Observatory pilot project, this paper gives an overview of currently available implementations related to web-based technologies for processing large and heterogeneous datasets and discuss their relevance within the context of environmental data processing, simulation and prediction. We found that, the processing of the simple datasets used in the pilot proved to be relatively straightforward using a combination of R, RPy2, PyWPS and PostgreSQL. However, the use of NoSQL databases and more versatile frameworks such as OGC standard based implementations may provide a wider and more flexible set of features that particularly facilitate working with larger volumes and more heterogeneous data sources. We review web service related technologies to manage, transfer and process Big Data.We examine international standards and related implementations.Many existing algorithms can be easily exposed as services and cloud-enabled.The adoption of standards facilitate the implementation of workflows.Use of web technologies to tackle environmental issues is acknowledged worldwide.

/pdf/web-technologies-for-environmental-big-data-5f0rc9hy90.pdf

Web technologies for environmental Big Data

This paper describes DataONE, a federated data network that is being built to improve access to, and preserve data about, life on Earth and the environment that sustains it. DataONE supports science by: (1) engaging the relevant science, library, data, and policy communities; (2) facilitating easy, secure, and persistent storage of data; and (3) disseminating integrated and user-friendly tools for data discovery, analysis, visualization, and decision-making. The paper provides an overview of the DataONE architecture and community engagement activities. The role of identifiers in DataONE and the policies and procedures involved in data submission, curation, and citation are discussed for one of the affiliated data centers. Finally, the paper highlights EZID, a service that enables digital object producers to easily obtain and manage long-term identifiers for their digital content.

DataONE: Data Observation Network for Earth - Preserving Data and Enabling Innovation in the Biological and Environmental Sciences

Within a digital library, collections may range from an ad hoc set of objects that serve a temporary purpose to established library collections intended to persist through time. The objects in these collections vary widely, from library and data center holdings to pointers to real-world objects, such as geographic places, and the various metadata schemas that describe them. The key to integrated use of such a variety of collections in a digital library is collection metadata that represents the inherent and contextual characteristics of a collection. The Alexandria Digital Library (ADL) Project has designed and implemented collection metadata for several purposes: in XML form, the collection metadata "registers" the collection with the user interface client; in HTML form, it is used for user documentation; eventually, it will be used to describe the collection to network search agents; and it is used for internal collection management, including mapping the object metadata attributes to the common search parameters of the system.

Collection metadata solutions for digital library applications

We describe our Simple Digital Library Interoperability Protocol (SDLIP), which allows clients to query information sources in a uniform syntax. The protocol was developed in a collaboration between Stanford, the Universities of California at Berkeley, and Santa Barbara, the San Diego Supercomputer Center, and the California Digital Library. In addition to introducing the protocol, we describe several of our design choices, and compare them with the choices made in other search middleware approaches. The protocol allows for both stateful and stateless operation, supports multiple query languages, and defines a simple XML-based return format. A default query language that is included in SDLIP follows the evolving IETF DASL 'basicsearch'. This is an XML-encoded language reminiscent of SQL, but adjusted for use in full-text environments. SDLIP can be used with CORBA or HTTP.

Search Middleware and the Simple Digital Library Interoperability Protocol (Long Version)

The Alexandria Digital Earth ProtoType (ADEPT) architecture is a framework for building distributed digital libraries of georeferenced information. An ADEPT system comprises one or more autonomous libraries, each of which provides a uniform interface to one or more collections, each of which manages metadata for one or more items. The primary standard on which the architecture is based is the ADEPT bucket framework, which defines uniform client-level metadata query services that are compatible with heterogeneous underlying collections. ADEPT functionality strikes a balance between the simplicity of Web document delivery and the richness of Z39.50. The current ADEPT implementation runs as servlet-based middleware and supports collections housed in arbitrary relational databases.

The ADEPT digital library architecture

We describe our Simple Digital Library Interoperability Protocol (SDLIP), which allows clients to query information sources in a uniform syntax The protocol was developed in a collaboration between Stanford, the Universities of California at Berkeley, and Santa Barbara, the San Diego Supercomputer Center, and the California Digital Library In addition to introducing the protocol, we describe several of our design choices, and compare them with the choices made in other search middleware approaches The protocol allows for both stateful and stateless operation, supports multiple query languages, and defines a simple XML-based return format A default query language that is included in SDLIP follows the evolving IETF DASL 'basicsearch' This is an XML-encoded language reminiscent of SQL, but adjusted for use in full-text environments SDLIP can be used with CORBA or HTTP

Greg Janée

Papers

DataONE: Data Observation Network for Earth - Preserving Data and Enabling Innovation in the Biological and Environmental Sciences

Collection metadata solutions for digital library applications

Search Middleware and the Simple Digital Library Interoperability Protocol (Long Version)

The ADEPT digital library architecture

Search Middleware and the Simple Digital Library Interoperability Protocol