Advance in Detecting Key Concepts as an Expert Model: Using Student Mental Model Analyzer for Research and Teaching (SMART)

TLDR: This study investigated which filtering method extract key concepts most accurately from experts’ concept maps and showed the PageRank filtering method outperformed the other methods in all accuracy measures.

Abstract: While key concepts embedded within an expert’s textual explanation have been considered an aspect of expert model, the complexity of textual data makes determining key concepts demanding and time consuming. To address this issue, we developed Student Mental Model Analyzer for Teaching and Learning (SMART) technology that can analyze an expert’ textual explanation to elicit an expert concept map from which key concepts are automatically derived. SMART draws on four graph-based metrics (i.e., clustering coefficient, betweenness, PageRank, and closeness) to automatically filter key concepts from experts’ concept maps. This study investigated which filtering method extract key concepts most accurately. Using 18 expert textual data, we compared the accuracy levels of those four competing filtering methods by referring to four accuracy measures (i.e., precision, recall, F-measure, and N-similarity). The results showed the PageRank filtering method outperformed the other methods in all accuracy measures. For example, on average, PageRank derived 79% of key concepts as accurately as human experts. SMART’s automatic filtering methods can help human experts save time when building an expert model, and it can validate their decision making on a list of key concepts.