Poor data quality (DQ) can have substantial social and economic impacts. Although firms are improving data quality with practical approaches and tools, their improvement efforts tend to focus narrowly on accuracy. We believe that data consumers have a much broader data quality conceptualization than IS professionals realize. The purpose of this paper is to develop a framework that captures the aspects of data quality that are important to data consumers.A two-stage survey and a two-phase sorting study were conducted to develop a hierarchical framework for organizing data quality dimensions. This framework captures dimensions of data quality that are important to data consumers. Intrinsic DQ denotes that data have quality in their own right. Contextual DQ highlights the requirement that data quality must be considered within the context of the task at hand. Representational DQ and accessibility DQ emphasize the importance of the role of systems. These findings are consistent with our understanding that high-quality data should be intrinsically good, contextually appropriate for the task, clearly represented, and accessible to the data consumer.Our framework has been used effectively in industry and government. Using this framework, IS managers were able to better understand and meet their data consumers' data quality needs. The salient feature of this research study is that quality attributes of data are collected from data consumers instead of being defined theoretically or based on researchers' experience. Although exploratory, this research provides a basis for future studies that measure data quality along the dimensions of this framework.

https://courses.washington.edu/geog482/resource/14_Beyond_Accuracy.pdf

Beyond accuracy: what data quality means to data consumers

The author systematically examines one of the important issues of information — establishing the market price. He introduces the concept of «search» — where a buyer wanting to get a better price, is forced to question sellers. The article deals with various aspects of finding the necessary information.

Economics of Information

Database management systems will continue to manage large data volumes. Thus, efficient algorithms for accessing and manipulating large sets and sequences will be required to provide acceptable performance. The advent of object-oriented and extensible database systems will not solve this problem. On the contrary, modern data models exacerbate the problem: In order to manipulate large sets of complex objects as efficiently as today's database systems manipulate simple records, query-processing algorithms and software will become more complex, and a solid understanding of algorithm and architectural issues is essential for the designer of database management software. This survey provides a foundation for the design and implementation of query execution facilities in new database management systems. It describes a wide array of practical query evaluation techniques for both relational and postrelational database systems, including iterative execution of complex query evaluation plans, the duality of sort- and hash-based set-matching algorithms, types of parallel query execution and their implementation, and special operators for emerging database application domains.

/pdf/query-evaluation-techniques-for-large-databases-45td1anbls.pdf

Query evaluation techniques for large databases

Organizational databases are pervaded with data of poor quality. However, there has not been an analysis of the data quality literature that provides an overall understanding of the state-of-art research in this area. Using an analogy between product manufacturing and data manufacturing, this paper develops a framework for analyzing data quality research, and uses it as the basis for organizing the data quality literature. This framework consists of seven elements: management responsibilities, operation and assurance costs, research and development, production, distribution, personnel management, and legal function. The analysis reveals that most research efforts focus on operation and assurance costs, research and development, and production of data products. Unexplored research topics and unresolved issues are identified and directions for future research provided. >

/pdf/a-framework-for-analysis-of-data-quality-research-2el9vxf14a.pdf

A framework for analysis of data quality research

Evaluation of clustering algorithms for financial risk analysis using MCDM methods

The paper aims to shed some light on the question why clustering algorithms, despite being quantitative and hence supposedly objective in nature, yield different and varied results. To do that, we took 10 common clustering algorithms and tested them over four known datasets, used in the literature as baselines with agreed upon clusters. One additional method, Binary-Positive, developed by our team, was added to the analysis. The results affirm the unpredictable nature of the clustering process, point to different assumptions taken by different methods. One conclusion of the study is to carefully choose the appropriate clustering method for any given application.

Investigating diversity of clustering methods: An empirical comparison

Technology and knowledge: bridging a generating gap

A knowledge tool, which includes a binary dataset for representing relationship patterns between objects and methods of its use. The use of the binary representation is based on an algorithm of data clustering according to binary similarity indices, which are derived from the binary matrix. Applications which are based on the binary representation and its compression capability include data mining, text mining, search engines, pattern recognition, enhancing data exchange rate between computerized devices, database implementation on hardware, saving storage space and adaptive network addressing.

Israel Spiegler

Papers

Investigating diversity of clustering methods: An empirical comparison

Technology and knowledge: bridging a generating gap

Unified binary model and methodology for knowledge representation and for data and information mining

Developing risk breakdown structure for information technology organizations

Integrating system analysis and project management tools