important problems involve clustering large datasets. Although naive implementations of clustering are computa- tionally expensive, there are established ecient techniques for clustering when the dataset has either (1) a limited num- ber of clusters, (2) a low feature dimensionality, or (3) a small number of data points. However, there has been much less work on methods of eciently clustering datasets that are large in all three ways at once|for example, having millions of data points that exist in many thousands of di- mensions representing many thousands of clusters. We present a new technique for clustering these large, high- dimensional datasets. The key idea involves using a cheap, approximate distance measure to eciently divide the data into overlapping subsets we call canopies .T hen cluster- ing is performed by measuring exact distances only between points that occur in a common canopy. Using canopies, large clustering problems that were formerly impossible become practical. Under reasonable assumptions about the cheap distance metric, this reduction in computational cost comes without any loss in clustering accuracy. Canopies can be applied to many domains and used with a variety of cluster- ing approaches, including Greedy Agglomerative Clustering, K-means and Expectation-Maximization. We present ex- perimental results on grouping bibliographic citations from the reference sections of research papers. Here the canopy approach reduces computation time over a traditional clus- tering approach by more than an order of magnitude and decreases error in comparison to a previously used algorithm by 25%.

http://dc-pubs.dbs.uni-leipzig.de/files/McCallum2000Efficientclusteringofhighdimensionaldatasetswith.pdf

Efficient clustering of high-dimensional data sets with application to reference matching

Systems and methods for hosting variable schema data such as dynamic tables and columns in a fixed physical database schema. Standard objects, such as tables are provided for use by multiple tenants or organizations in a multi-tenant database system. Each organization may add or define custom fields for inclusion in a standard object. Custom fields for multiple tenants are stored in a single field within the object data structure, and this single field may contain different data types for each tenant. Indexing columns are also provided, wherein a tenant may designate a field for indexing. Data values for designated fields are copied to an index column, and each index column may include multiple data types. Each organization may also define custom objects including custom fields and indexing columns. Custom objects for multiple tenants are stored in a single custom object data structure. The primary key values for the single custom object table are globally unique, but also include an object-specific identifier which may be re-used among different entities.

Custom entities and fields in a multi-tenant database system

A system, method, and article of manufacture are provided for managing information in a development architecture framework. Common information that is used by a plurality of components of a system is allowed to be accessed in a single, shared repository. Unique information that is unique to the components of the system is stored in corresponding designated folders. Media content communicated in the system is managed based on metadata thereof.

Managing information in an integrated development architecture framework

In general, in one aspect, the invention relates to a method for transaction processing. The method includes specifying metadata and storing the metadata. An index is created in response to the stored metadata. The method also includes receiving a transaction, generating an index log of changes to the index in response to the received transaction, and modifying the first index in response to the generated index log.

Database system and methods

Disclosed is a method for creating a real-time search engine over the Internet which provides a search response containing data object descriptions and server descriptions of data objects that are currently available for transfer from a provider server directly to a recipient client in response to a recipient client search request The method comprises the provider server connecting to a Real-time search engine through the Internet, the provider server providing the Real-time search engine with data object descriptions of data objects residing on the provider server, and the Real-time search engine indexing data object descriptions associated with the data object of the provider server, wherein the data object descriptions provided by the provider server are purged from the Real-time search engine when the provider server is disconnected from the Real-time search engine

Real-time search engine

A method and system for accessing and sharing metadata amongst a number of data marts through the use of object referencing. A global repository is created. Shared folders containing metadata is contained in the global repository. Any number of data marts can be linked to the global repository. Stored within the folders of the linked data marts are references pointing to certain metadata stored in the shared folders of the global repository. Multiple data marts can reference the same metadata in the global repository. Thereby, the same metadata can be accessed, used, and shared by multiple users across different data marts, provided that they have been granted the proper authority. New metadata can be promoted from a linked data mart into the global repository so that it can be shared, and updates to shared metadata can be automatically propagated to all users referencing that metadata.

Method and apparatus for sharing metadata between multiple data marts through object references

A method of performing incremental aggregation of dynamically increasing database data sets. An embodiment of the present invention operates within a data mart or data warehouse to aggregate data stored within an operational database corresponding to newly received data to provide current information. Initially, a computer server creates an intermediate file which is initialized by the server with an aggregate data set. The aggregate data set consists of data values and count values that each correspond to specific group identifiers. The computer determines if any group identifiers within a new set of inputs data are identical to any group identifiers stored within the intermediate file. If an inputted group identifier matches a stored group identifier, the inputted data value is aggregated with the stored data value and the count value corresponding to the specific stored group identifier is incremented by one. If an inputted group identifier does not match any of the stored group identifiers, the inputted group identifier and corresponding data value are stored within the intermediate file and a count value of one is appended to that specific group identifier. Once all the group identifiers within the new set of input data have been determined, the computer stores all the changes that were made to the intermediate file into the aggregate data set.

A method for incremental aggregation of dynamically increasing database data sets

A method for updating a target table of a data mart in response to changes made by a transaction to data stored in a source table of an operational database. Data that was changed in the source table by the transaction is stored in a dynamic image table of a change capture database. Data that was not changed in the source table by the transaction, but which is nevertheless required to be mapped to the target table, is stored in a static image table of the change capture database. The change capture database also contains relevant information regarding the transaction. Once the dynamic and static image tables are properly staged, the changes are propagated from the change capture database to the target tables of the data marts. In other words, data is extracted from the change capture database and subsequently transformed and loaded, thereby minimizing the impact to the operational database. Thereupon, the tables of the change capture database are truncated to discard data which is now no longer needed.

Apparatus and method for capturing and propagating changes from an operational database to data marts

An advertising system delivers video ads into hypertext documents, where these ads provide enhanced user interactivity features selected and configured by advertisers or other ad sponsors The system also provides sophisticated tracking and real time reporting of viewer interactivity with the video ads and their enhanced features

Method and apparatus to deliver video advertisements with enhanced user interactivity

Users create advertisement components using a browser-based application. Once the components are generated, the application compiles the components on the backend to generate a seamless advertisement. Metadata are applied to the advertisements to specify a geographic location, demographics, and keywords. The advertisements are matched up with clients based on identifying information in the user profiles. The geographic location includes zip codes and surrounding areas. The advertisement includes coupons that are presented in local stores. The cost of displaying the advertisement is negotiated between the advertiser and the publisher according to the number of slots available, and the prices that each party is willing to accept.

Diaz Nesamoney

Papers

Method and apparatus for sharing metadata between multiple data marts through object references

A method for incremental aggregation of dynamically increasing database data sets

Apparatus and method for capturing and propagating changes from an operational database to data marts

Method and apparatus to deliver video advertisements with enhanced user interactivity

Method and system for serving localized advertisements