Approximate String Matching

Nowadays, due to the development of mobile devices, the kinds of data that are generated are becoming diverse, and the amount is becoming huge. The vast amount of data generated in this way is called big data. Big data must be processed in a different way than existing data processing methods. Representative methods of big data processing are RDBMS (Relational Database System) and NoSQL method. We compare NoSQL and RDBMS, which are representative database systems. In this paper, we use MySQL query and MongoDB query to compare RDBMS and NoSQL. We gradually compare the performance of CRUD operations in MySQL and MongoDB by increasing the amount of data. MongoDB sets index and compares it again.  Through result of these operations is to choose a database system that fits the situation.  This makes it possible to design and analyse big data more efficiently.

Performance Comparison of CRUD Operations in IoT based Big Data Computing

Similarity measurement is a significant process to determine the degree of similarity between two records. This paper presents a comparative analysis of important similarity measurements which are utilised for the detection of duplicated records in databases. The work evaluates their strengths based on the efficiency of prevailing algorithms, the time required to process and identify duplications as well as performance accuracy. The analysis conducted found that among the most common similarity measurements, those based on the Jaro-Winkler algorithm significantly outperformed the other algorithms. This paper presents an enhanced strategy based on the Jaro-Winkler algorithm to improve the detection of similarity among database records. The ability to provide solutions to this problem will greatly enhance the quality of data used in decision-making.

A review of similarity measurement for record duplication detection

In recent years, the utilization of digital instruments in industries is quickly expanding. This is because digital instruments are typically more exact than the analog instruments, and easier to be read as they are hooked up to a liquid-crystal display (LCD). However, manual data entry from LCD display is tedious and less accurate. This paper proposes a real-time LCD digit recognition system for the industrial purposes. The system is interfaced with an IP webcam to capture the video frames from the LCD display. The digital data is pre-processed into grayscale and being cropped into a selected region of interest (ROI). Adaptive thresholding and morphological operation are applied for the digit segmentation process. Data extraction and characterization are done by utilizing neural network classifier. Finally, all the information are logged out to Microsoft Excel spreadsheet. The 90% accuracy is accomplished for 50 test images of various LCD display.

/pdf/real-time-lcd-digit-recognition-system-1zhoj8i2n2.pdf

Real-Time LCD Digit Recognition System

Data-based and data-driven decisions are at the core of digital government transformation. However, the more the data is to be used to guide policy development, the higher are the requirements to the data accuracy and readiness. Larger reliance on data to inform policy decisions should not lead to increased reporting requirements and hence excessive administrative burden on businesses. Therefore, identifying and reducing duplication in statistical data should be performed at the early stages of the government digital transformation. Given the constantly increasing number of strategic documents and continuous amendments to the list of statistic indicators measured, there is a need for an instrument allowing for timely identification and elimination of possible duplication in statistical and other indicators.

Identifying Duplication in Statistical Indicators: Methodic Approach

The recognition of similar entities in databases has gained substantial attention in many application areas. Despite several techniques proposed to recognize and locate duplication of database records, there is a dearth of studies available which rate the effectiveness of the diverse techniques used for duplicate record detection. Calculating time complexity of the proposed methods reveals their performance rating. The time complexity calculation showed that the efficiency of these methods improved when blocking and windowing is applied. Some domain-specific methods train systems to optimize results and improve efficiency and scalability, but they are prone to errors. Most of the existing methods fail to either discuss, or lack thoroughness in consideration of scalability. The process of sorting and searching form an essential part of duplication detection, but they are time-consuming. Therefore this paper proposes the possibility of eliminating the sorting process by utilization of tree structure to improve the record duplication detection. This has added benefits of reducing time required, and offers a probable increase in scalability. For database system, scalability is an inherent feature for any proposed solution, due to the fact that the data size is huge. Improving the efficiency in identifying duplicate records in databases is an essential step for data cleaning and data integration methods. This paper reveals that the current proposed methods lack in providing solutions that are scalable, high accurate, and reduce the processing time during detecting duplication of records in database. The ability to provide solutions to this problem will improve the quality of data that are used for decision making process.

/pdf/record-duplication-detection-in-database-a-review-3vuls34ehb.pdf

Record Duplication Detection in Database: A Review

This work presents a self-motivating personal vision assistant that serves as a self-reflecting and motivating system for people such as those with low self-esteem, lonely or depressed individuals, single mothers, divorced or widowed, who constantly needs to be motivated to remain positive and focused. The system is similar to the rest of the personal digital assistants in the market but with the added functionality of reading the mood of the user and motivating them based on their present states of mind such as happy, sad, bored etc. The system uses facial recognition to determine the users' mood and a database of user's history or past events to adequately motivate the users. Further extensions include creating and adding imagery of beloved ones that have impacts on the lives of the users, such as missed loved ones like mothers, spouse, etc as added stimulation.

Saleh Rehiel Alenazi

Papers

Record Duplication Detection in Database: A Review

A review of similarity measurement for record duplication detection

Mirror that Talks: A Self-Motivating Personal Vision Assistant

An efficient algorithm for data cleansing