Showing papers on "Data mart published in 2011"

Original Article: A proposed model for data warehouse ETL processes

Abstract: Extraction-transformation-loading (ETL) tools are pieces of software responsible for the extraction of data from several sources, its cleansing, customization, reformatting, integration, and insertion into a data warehouse. Building the ETL process is potentially one of the biggest tasks of building a warehouse; it is complex, time consuming, and consumes most of data warehouse project's implementation efforts, costs, and resources. Building a data warehouse requires focusing closely on understanding three main areas: the source area, the destination area, and the mapping area (ETL processes). The source area has standard models such as entity relationship diagram, and the destination area has standard models such as star schema, but the mapping area has not a standard model till now. In spite of the importance of ETL processes, little research has been done in this area due to its complexity. There is a clear lack of a standard model that can be used to represent the ETL scenarios. In this paper we will try to navigate through the efforts done to conceptualize the ETL processes. Research in the field of modeling ETL processes can be categorized into three main approaches: Modeling based on mapping expressions and guidelines, modeling based on conceptual constructs, and modeling based on UML environment. These projects try to represent the main mapping activities at the conceptual level. Due to the variation and differences between the proposed solutions for the conceptual design of ETL processes and due to their limitations, this paper also will propose a model for conceptual design of ETL processes. The proposed model is built upon the enhancement of the models in the previous models to support some missing mapping features.

Method and system for performing diagnostics or software maintenance for a vehicle

Abstract: A method and system for performing diagnostics or software maintenance on a vehicle comprises recording high-fidelity data at the vehicle consistent with configuration files to support engineering analysis or diagnostics on vehicle components, systems or performance (S220). Supplemental data is retrieved via a low bandwidth transmission at the vehicle to supplement the recorded high-fidelity data (S222). The recorded high- fidelity data and retrieved supplemental data is processed to generate a diagnostic status report message from transmission to the central electronic data processing system (S224). A data processor or central electronic data processing system organizes the diagnostic report message, the recorded high fidelity data and the retrieved supplemental data into a data mart or diagnostics database to support engineering analysis or diagnostics on vehicle components, systems or performance (S226).

An MDA Approach and QVT Transformations for the Integrated Development of Goal-Oriented Data Warehouses and Data Marts

Abstract: To customize a data warehouse, many organizations develop concrete data marts focused on a particular department or business process. However, the integrated development of these data marts is an open problem for many organizations due to the technical and organizational challenges involved during the design of these repositories as a complete solution. In this article, the authors present a design approach that employs user requirements to build both corporate data warehouses and data marts in an integrated manner. The approach links information requirements to specific data marts elicited by using goal-oriented requirement engineering, which are automatically translated into the implementation of corresponding data repositories by means of model-driven engineering techniques. The authors provide two UML profiles that integrate the design of both data warehouses and data marts and a set of QVT transformations with which to automate this process. The advantage of this approach is that user requirements are captured from the early development stages of a data-warehousing project to automatically translate them into the entire data-warehousing platform, considering the different data marts. Finally, the authors provide screenshots of the CASE tools that support the approach, and a case study to show its benefits.

Patent data analysis method and system

Abstract: The invention discloses a patent data analysis method and system, belonging to the computer application field. The patent data analysis method comprises: constructing a subject corresponding to an analysis target in a local database; constructing a data mart consistent with the subject; constructing a data view corresponding to the subject in a data warehouse; extracting the patent data in the local database; storing the extracted patent data in the data warehouse; determining the corresponding data mart according to the request of a user; and analyzing the determined data mart, and returning the analysis result to the user in a view manner. The method can process the patent data by ETL (extract-transform-load) treatment specific to the subject, and can analyze the patent data to improve the patent data analysis efficiency and quality, thereby providing convenience for users.

Auxiliary decision supporting system of hospital

Abstract: The invention relates to an auxiliary decision supporting system of a hospital. The system comprises a production system layer, an interface layer, a data layer and an application layer. The system is characterized in that: the production system layer integrates data of each service system of the hospital, the data are integrated by the interface layer and organized to a data warehouse, the data required corresponding to each system of the application layer are extracted from the data warehouse through the data layer to form a data mart, and finally the data mart provides data support for the corresponding system so as to ensure development and operation of the corresponding system of the application layer. The system can integrate the complex data of the hospital and generate an intuitive data table to assist an administrator in integrating the all-round information of the hospital to make a decision so as to improve the efficiency and the validity of decision management.

Data mart automation

Abstract: Systems and methods are described for automating the creation of data marts within an enterprise. Data is maintained in a plurality of data sources that include at least a relational database and a multidimensional database. The system includes a business intelligence server that provides a virtual logical semantic model to integrate all of the plurality of data sources. The user specifies a list of levels and measures on the virtual logical semantic model. The list of levels and measures can span data from multiple data sources. The user can also specify a location in the plurality of data sources that will store the aggregate matrix. Once the list of levels and measures are specified, the business intelligence server generates a multidimensional cube to store the data for the aggregate matrix and stores the multidimensional cube in the data source location.

Blink: Not Your Father’s Database!

Abstract: The Blink project’s ambitious goals are to answer all Business Intelligence (BI) queries in mere seconds, regardless of the database size, with an extremely low total cost of ownership. It takes a very innovative and counter-intuitive approach to processing BI queries, one that exploits several disruptive hardware and software technology trends. Specifically, it is a new, workload-optimized DBMS aimed primarily at BI query processing, and exploits scale-out of commodity multi-core processors and cheap DRAM to retain a (copy of a) data mart completely in main memory. Additionally, it exploits proprietary compression technology and cache-conscious algorithms that reduce memory bandwidth consumption and allow most SQL query processing to be performed on the compressed data. Ignoring the general wisdom of the last three decades that the only way to scalably search large databases is with indexes, Blink always performs simple, “brute force” scans of the entire data mart in parallel on all nodes, without using any indexes or materialized views, and without any query optimizer to choose among them. The Blink technology has thus far been incorporated into two products: (1) an accelerator appliance product for DB2 for z/OS (on the “mainframe”), called the IBM Smart Analytics Optimizer for DB2 for z/OS, V1.1, which was generally available in November 2010; and (2) the Informix Warehouse Accelerator (IWA), a software-only version that was generally available in March 2011. We are now working on the next generation of Blink, called BLink Ultra, or BLU, which will significantly expand the “sweet spot” of Blink technology to much larger, disk-based warehouses and allow BLU to “own” the data, rather than copies of it.

QBX: a CASE tool for data mart design

Abstract: QBX is a CASE tool for data mart design resulting from a close collaboration between academy and industry. It supports designers during conceptual design, logical design, and deployment of ROLAP data marts in the form of star/snowflake schemata, and it can also be used by business users to interactively explore project-related knowledge at different levels of abstraction. We will demonstrate QBX functionalities focusing on both forward and reverse engineering scenarios.

Comparison plan for data warehouse system architectures

Abstract: Many organizations look for a proper way to make better and faster decisions about their businesses. Data warehouse has unique features such as data mining and ad hoc querying on data collected and integrated from many of the computerized systems used in organization. Data warehouse can be built using a number of architectures. Each one of the architecture has its own advantages and disadvantages. This paper investigates five different data warehouse architectures: centralized data warehouse, independent data mart, dependent data mart, homogeneous distributed data warehouse and heterogeneous distributed data warehouse and subsequently an optimal plan will be described and then all data warehouse architectures mentioned above will be parametrically measured by this method. Finally as case study, a company's problem in their data accessing system is studied and the best architecture is chosen to accommodate the needs of the company's system.

Master Data Management in a Data Warehouse/Data Mart

Abstract: A data warehouse incorporates processing for creating, managing, and otherwise maintaining master data. The data warehouse may communicate with a master data manager to obtain services for handling master data. A surrogate master data identifier may be defined by the data warehouse to reference the master data, thereby decoupling any modifications of the master data identifier that may be made by the master data manager. The data warehouse may export the master data to an application system, and conversely import master data from an application system.

Thinking Structurally Helps Business Intelligence Design

Abstract: The design of Business Intelligence (BI) systems needs the integration of different enterprise figures: on the one hand, business managers give their information requirements in terms of Key Performance Indicators (KPI). On the other hand, Information Technology (IT) experts provide the technical skill to compute KPI from transactional data. The gap between managerial and technical views of information is one of the main problems in BI systems design. In this paper we tackle the problem from the perspective of mathematical structures of KPI, and discuss the advantages that a semantic representation able to explicitly manage such structures can give in different phases of the design activity. In particular we propose a novel model of ontology for KPI, and show how this model can be exploited to support KPI elicitation and to analyze dependencies among indicators in terms of common components, thus giving the manager a structured overall picture of her requirements, and the IT personnel a valuable support for source selection and data mart design.

Computer-aided data-mart design

Abstract: With decision support systems, decisionmakers analyse data in data marts extracted from production bases. The data-mart sche ma design is generally performed by expert designers (administrator or com puter specialist). With data-driven, requirement-driven or hybrid-driven ap proaches, this designer builds a data-mart defining facts (analysis subject s) and analysis axes. This process, based on data sources and decision-makers requirements, often turns out to be approximate and complex. We propose to de sign a data-mart schema by the decision-maker himself, following a hybrid-d riven approach. Using an assistance process that visualises successively int rmediate schemas built from data sources, the decision-maker gradually builds h is multidimensional schema. He determines measures to be analysed, dimensions h ierarchies within dimensions. A CASE tool based on this concept has be en d veloped.

GeoCREV: veterinary geographical information system and the development of a practical sub-national spatial data infrastructure.

Abstract: This paper illustrates and discusses the key issues of the geographical information system (GIS) developed by the Unit of Veterinary Epidemiology of the Veneto region (CREV), defined according to user needs, spatial data (availability, accessibility and applicability), development, technical aspects, inter-institutional relationships, constraints and policies. GeoCREV, the support system for decision-making, was designed to integrate geographic information and veterinary laboratory data with the main aim to develop a sub-national, spatial data infrastructure (SDI) for the veterinary services of the Veneto region in north-eastern Italy. Its implementation required (i) collection of data and information; (ii) building a geodatabase; and (iii) development of a WebGIS application. Tools for the management, collection, validation and dissemination of the results (public access and limited access) were developed. The modular concept facilitates the updating and development of the system according to user needs and data availability. The GIS management practices that were followed to develop the system are outlined, followed by a detailed discussion of the key elements of the GIS implementation process (data model, technical aspects, inter-institutional relationship, user dimension and institutional framework). Problems encountered in organising the non-spatial data and the future work directions are also described.

Improving the Data Warehouse Architecture Using Design Patterns

Abstract: Data warehousing is an important part of the enterprise information system. Business intelligence (BI) relies on data warehouses to improve business performance. Data quality plays a key role in BI. Source data is extracted, transformed, and loaded (ETL) into the data warehouses periodically. The ETL operations have the most crucial impact on the data quality of the data warehouse. ETL-related data warehouse architectures including structure-oriented layer architectures and enterpriseview data mart architecture were studied in the literature. Existing architectures have the layer and data mart components but do not make use of design patterns; thus, those approaches are inefficient and pose potential problems. This paper relays how to use design patterns to improve data warehouse architectures.

Metadata for Data Warehousing

Abstract: A data warehouse is kind of database whose architecture (and underlying supporting technology) has been optimized for highly efficient query, at the cost of sacrificing features that support robust interactive inserts, updates and delete actions. The difference between a data warehouse and a data mart (which is also optimized for the same purpose) is partly one of scope. While warehouses are supposed to encompass data across an entire organization, data marts are typically smaller scale (e.g., departmental in scope) though in an ideal situation they would receive data from a warehouse, effectively serving as front-ends to the latter.

MDA compliant approach for data mart schemas generation

Abstract: Decision Support System (DSS) designers face two data models complexities: OLTP (On Line Transaction Processing) database model and multidimensional model. These models relying on different concepts force the designer to have double skills. The objective of this paper is to assist the DSS designer producing rapidly Data Mart (DM) schemas starting from the Data Warehouse (DW) data model. More accurately, we aim to automate the generation of DM schemas from a relational DW schema according to the MDA (Model Driven Architecture) paradigm. To do so, we propose an approach based on a set of transformation rules that we define in ATL language and illustrate with an example.

Design and Implementation of an Enterprise Data Warehouse

Abstract: DESIGN AND IMPLEMTATION OF AN ENTERPRISE DATA WAREHOUSE Edward M. Leonard, B.S. Marquette University, 2011 The reporting and sharing of information has been synonymous with databases as long as there have been systems to host them. Now more than ever, users expect the sharing of information in an immediate, efficient, and secure manner. However, due to the sheer number of databases within the enterprise, getting the data in an effective fashion requires a coordinated effort between the existing systems. There is a very real need today to have a single location for the storage and sharing of data that users can easily utilize to make improved business decisions, rather than trying to traverse the multiple databases that exist today and can do so by using an enterprise data warehouse. The Thesis involves a description of data warehousing techniques, design, expectations, and challenges regarding data cleansing and transforming existing data, as well as other challenges associated with extracting from transactional databases. The Thesis also includes a technical piece discussing database requirements and technologies used to create and refresh the data warehouse. The Thesis discusses how data from databases and other data warehouses could integrate. In addition, there is discussion of specific data marts within the warehouse to satisfy a specific need. Finally, there are explanations for how users will consume the data in the enterprise data warehouse, such as through reporting and other business intelligence. This discussion also includes the topics of system architecture of how data from databases and other data warehouses from different departments could integrate. An Enterprise Data Warehouse prototype developed will show how a pair of different databases undergoes the Extract, Transform and Load (ETL) process and loaded into an actual set of star schemas then makes the reporting easier. Separately, an important piece of this thesis takes an actual example of data and compares the performance between them by running the same queries against separate databases, one transactional and one data warehouse. As the queries expand in difficulty, larger grows the gap between the actual recorded times of running that same query in the different environments.

Zero Latency applied on a commercial Data Mart: Real-time information in support of decision making

Abstract: The growing development of technology, which has been experienced for several decades, caused the area of Information Technology (IT) to have a significant advance in the corporate world. The expansion of IT to different areas of the company has enabled a greater transmission and storage of information. The arrival of personal computers has also been a greater contributor to an increase in storage of information. Due to that, databases and Information Systems where deployed. The managers began to use the databases for specific business areas, which are called Data Marts (DM). Despite the use of Data Marts, the extracted scenarios did not express the updated information in the way management would require, because there was still a time lag between the occurrence of an event and its absorption by the company, which in fact, meant a delay in moving forward with essential and strategic areas of the business. Minimize this lag became a challenge and a necessity for the competitive business world. At the time, the solution adopted, in order to minimize the time lag between the occurrences and the effective absorption of this information, was called Zero Latency Enterprise (ZLE). The aim of this paper is to examine how real time information can be obtained through the implementation of a ZLE in the DM and how, effectively, it would support the strategic decision-making process in organizations. In order to achieve that, a proposal has been put forward to create a model of DM in ZLE with data obtained from the commercial department of a civil engineering supply company. This analysis reveals significant results, supporting the original idea to develop support for decision making.

PriTri: An Innovative Algorithm for Clustering Categorical Data in Data Warehouse

Abstract: the process of data mining to extract knowledge from large data set needs great potential to extract the hidden nuggets. To cluster the numerical data there are enormous clustering technique. Data mining for categorical data(qualitative and quantitative) the most frequently used algorithms are k-means, k-mediods and fuzzy rule all these methods needs a threshold value to overcome this problem. This paper propose an algorithm to optimize the number of clusters and it also uses novel way to construct the data mart using the concept of multiprocessing Pri-tri algorithm. Keywordsmining, clustering, k-means, Multiprocessing.

Method for automatically creating a data mart by aggregated data extracted from a business intelligence server

Abstract: Systems and methods are described for automating the creation of data marts within an enterprise. Data is maintained in a plurality of data sources that include at least a relational database and a multidimensional database. The system includes a business intelligence server that provides a virtual logical semantic model to integrate all of the plurality of data sources. The user specifies a list of levels and measures on the virtual logical semantic model. The list of levels and measures can span data from multiple data sources. The user can also specify a location in the plurality of data sources that will store the aggregate matrix. Once the list of levels and measures are specified, the business intelligence server generates a multidimensional cube to store the data for the aggregate matrix and stores the multidimensional cube in the data source location.

What Is Business Intelligence

Abstract: This chapter presents a blueprint for understanding the exciting potential of SQL Server 2008 R2’s Business Intelligence (BI) technologies to meet your company’s crucial business needs. It describes tools, techniques, and high-level implementation concepts for BI.

Methods and tools of on-line analytical data processing for Siberian region government

Abstract: The authoring OLAP-system “Analytic” based on integration of DOLAP architecture and MOLAP functionality is represented. The original embodiments of the OLAP rules realization are described. Functional modules of the “Analytic” are described. The data access tools for constructing cube based on data from different local sources (e.g. Excel, Access, InterBase, MySQL, Oracle) and data warehouse are developed. The data mart manage tools are developed. The dimensional cubes are processed by original OLAP tools: statistical table, cross-table, diagram and map for geographic data. The original high-level programming language to create scenario of the fact calculation in to a cube is realized. The cubes sharing method to store a cube as the materialized view and semantic metadata in the data warehouse is suggested. The method of creating and committing step-by-step calculating algorithm using pre-aggregated cubes is considered. Tools for forming operative and statistical reports based on results of OLAP are described. The complex of proposed methods and tools enables Siberian region government to improve effectiveness of decision making support based on OLAP. The “Analytic” usage for control medical care, governing in the field of social support and planning the social economic development of territories is represented by using the examples.

Perancangan data warehousesistem informasi eksekutif(studi kasus data akademik prodi teknik elektro ft ugm)

Abstract: A university as an academic organizations requires a complete, rapid and accurate information to support the process and planning activities, evaluating, and making the right decision. Data is collected and abstracted from various operational data scattered in a variety of academic applications. This study examines the design of data warehouse that are the result of data integration and abstraction from various academic applications that accommodate the needs of history and archiving data to support executive information systems in the academic field. Identification of the executive reports use reference of the program accreditation forms, AMAI UGM questionnaires, FT UGM Dean�s annual reports, self-evaluation forms of Competitive Grant Program, features of the existing SIE UGM, and features of executive data on the UGM academic administration website. The subject of this research is student data based on chronological stages starting from candidates selection process, new students and students during lectures. The success of data warehouse design is determined by the right description of the business events, data completeness and validity, data mart design using metrics bus architecture, star schema design and ETL process to integrate, extract, cleanse, transform and load it into the data warehouse. In this thesis, researcher examined the data warehouse using static and adhoc reporting that can accommodate the needs of executives. In addition, the researchers suggest the existence of an integrated policy for the uniformity of data formats in a variety of academic applications. the need for completeness and validity of data will be determined by the discipline of users in updating the data correctly and regularly.

Developing Data Marts for Healthcare

Abstract: Research shows that information technology adoption in healthcare continues to lag behind many other industries, In response to this challenge tThis paper outlines a graduate course designed to give students a hands-on experience in building a data mart in a healthcare setting. Students will learn technical skills as well as team work, communication and problem solving skills. The structure of the course requires students to apply classroom concepts. For example, students use multiple methods to collect information for multidimensional modeling techniques as they identify various InfoObjects, such as characteristics, units, key figures and time characteristics. This course design presents an opportunity for students to gain valuable experience in building a healthcare-related system in SAP and thereby developing highly marketable skills.

Multi-source information integration service system based on Web technology

Abstract: The utility model discloses a multi-source information integration service system based on Web technology, which comprises a data base server, a data application server, a Web server, a switchboard and a user terminal display device, wherein the data base server is provided with a data warehouse, the data application server is provided with an analytical mode, an expert system and a data mart. The data application server is connected with the data base server, the Web server is connected with the data application server, the input end of the switchboard is connected with the Web server, the output end of the switchboard is connected with the terminal display device, and the data sent by the Web server is inputted into the switchboard and transmitted to the user terminal display device in a wired or wireless mode. The service system has the advantages of reducing time of search and inquiry of information for users and providing speedy services and convenience for users.

A business intelligence infrastructure supporting respiratory health analysis

Abstract: Business Intelligence Systems are being designed and implemented to support data analysis tasks that help organizations in the achievement of their goals. Th ese tasks are accomplished using several technologies aimed to st ore and analyze data. This paper presents the particular ca se of the design and implementation of a business intelligenc e system to support health care specialists in the analysis and characterization of symptoms related with the chronic obstructive pulmonary disease. For this specific ap plication domain, a data mart model is proposed, implemented and loaded, allowing the analysis of the available data using on-line analytical processing technology and a spatial data mining algorithm. The results obtained so far are promisin g, demonstrating the usefulness of the proposed busine ss intelligence approach to characterize the key facto rs in the comprehension of the disease under analysis. Keywords-business intelligence; data mining; on-line analytical processing; chronic obstructive pulmonary disease

Data Analytics and Data Warehouses

Abstract: This chapter focuses on how to extract information from package software systems. This information will be used for decision support system (DSS) purposes and typically be presented to the user in an on-line display format or as a report. The concept behind DSS is that it deals with data “after the fact,” meaning that the data are no longer in a production mode, but rather in a storage mode where it can be used for different forms of analytical processing. The major benefit of operating on processed data is that it cannot be changed and therefore can be accessed without concern for data integrity. Another salient issue is that because the data are not subject to change it can be copied multiple times, allowing for some interesting performance improvements and “flattening” of the data stored in the relational database. The referential integrity that was attained in a production database does not maximize performance for the query of data for reporting purposes. Almost all systems today need to determine how best to extract information for what is now being called “data analytics,” that is, the ability to understand and analyze the information contained in application software systems. Increasingly, CIOs are being held responsible for providing this information.