In the last 16 years, more than 200 research articles were published about research-paper recommender systems. We reviewed these articles and present some descriptive statistics in this paper, as well as a discussion about the major advancements and shortcomings and an overview of the most common recommendation concepts and approaches. We found that more than half of the recommendation approaches applied content-based filtering (55 %). Collaborative filtering was applied by only 18 % of the reviewed approaches, and graph-based recommendations by 16 %. Other recommendation concepts included stereotyping, item-centric recommendations, and hybrid recommendations. The content-based filtering approaches mainly utilized papers that the users had authored, tagged, browsed, or downloaded. TF-IDF was the most frequently applied weighting scheme. In addition to simple terms, n-grams, topics, and citations were utilized to model users' information needs. Our review revealed some shortcomings of the current research. First, it remains unclear which recommendation concepts and approaches are the most promising. For instance, researchers reported different results on the performance of content-based and collaborative filtering. Sometimes content-based filtering performed better than collaborative filtering and sometimes it performed worse. We identified three potential reasons for the ambiguity of the results. (A) Several evaluations had limitations. They were based on strongly pruned datasets, few participants in user studies, or did not use appropriate baselines. (B) Some authors provided little information about their algorithms, which makes it difficult to re-implement the approaches. Consequently, researchers use different implementations of the same recommendations approaches, which might lead to variations in the results. (C) We speculated that minor variations in datasets, algorithms, or user populations inevitably lead to strong variations in the performance of the approaches. Hence, finding the most promising approaches is a challenge. As a second limitation, we noted that many authors neglected to take into account factors other than accuracy, for example overall user satisfaction. In addition, most approaches (81 %) neglected the user-modeling process and did not infer information automatically but let users provide keywords, text snippets, or a single paper as input. Information on runtime was provided for 10 % of the approaches. Finally, few research papers had an impact on research-paper recommender systems in practice. We also identified a lack of authority and long-term research interest in the field: 73 % of the authors published no more than one paper on research-paper recommender systems, and there was little cooperation among different co-author groups. We concluded that several actions could improve the research landscape: developing a common evaluation framework, agreement on the information to include in research papers, a stronger focus on non-accuracy aspects and user modeling, a platform for researchers to exchange information, and an open-source framework that bundles the available recommendation approaches.

/pdf/research-paper-recommender-systems-a-literature-survey-3o8izjcp9m.pdf

Research-paper recommender systems: a literature survey

This chapter presents an analysis of recommender systems in Technology-Enhanced Learning along their 15 years existence (2000–2014). All recommender systems considered for the review aim to support educational stakeholders by personalising the learning process. In this meta-review 82 recommender systems from 35 different countries have been investigated and categorised according to a given classification framework. The reviewed systems have been classified into seven clusters according to their characteristics and analysed for their contribution to the evolution of the RecSysTEL research field. Current challenges have been identified to lead the work of the forthcoming years.

Panorama of Recommender Systems to Support Learning

Recommending items to group of users using Matrix Factorization based Collaborative Filtering

The increasing number of publications on recommender systems for Technology Enhanced Learning (TEL) evidence a growing interest in their development and deployment. In order to support learning, recommender systems for TEL need to consider specific requirements, which differ from the requirements for recommender systems in other domains like e-commerce. Consequently, these particular requirements motivate the incorporation of specific goals and methods in the evaluation process for TEL recommender systems. In this article, the diverse evaluation methods that have been applied to evaluate TEL recommender systems are investigated. A total of 235 articles are selected from major conferences, workshops, journals, and books where relevant work have been published between 2000 and 2014. These articles are quantitatively analysed and classified according to the following criteria: type of evaluation methodology, subject of evaluation, and effects measured by the evaluation. Results from the survey suggest that there is a growing awareness in the research community of the necessity for more elaborate evaluations. At the same time, there is still substantial potential for further improvements. This survey highlights trends and discusses strengths and shortcomings of the evaluation of TEL recommender systems thus far, thereby aiming to stimulate researchers to contemplate novel evaluation approaches.

/pdf/evaluating-recommender-systems-for-technology-enhanced-1nonqr81cj.pdf

Evaluating Recommender Systems for Technology Enhanced Learning: A Quantitative Survey

https://repositorio.inesctec.pt/bitstream/123456789/7575/1/P-00N-1N7.pdf

Metalearning and Recommender Systems: A literature review and empirical study on the algorithm selection problem for Collaborative Filtering

A framework for recommendation in learning object repositories: An example of application in civil engineering

Nowadays, there is a wide variety of e-learning repositories that provide digital resources for education in the form of learning objects. Some of these systems provide recommender systems in order to help users in the search for and selection of the learning objects most appropriate to their individual needs. The search for and recommendation of learning objects are usually viewed as a solitary and individual task. However, a collaborative search can be more effective than an individual search in some situations - for example, when developing a digital course between a group of instructors. The problem of recommending learning objects to a group of users or instructors is much more difficult than the traditional problem of recommending to only one individual. To resolve this problem, this paper proposes a collaborative methodology for searching, selecting, rating and recommending learning objects. Additionally, voting aggregation strategies and meta-learning techniques are used in order to automatically obtain the final ratings without having to reach a consensus between all the instructors. A functional model has been implemented within the DELPHOS hybrid recommender system. Finally, various experiments have been carried out using 50 different groups in order to validate the proposed learning object group recommendation approach. A collaborative methodology for searching, selecting, rating and recommending learning objects.Voting aggregation strategies and meta-learning techniques are used in order to automatically obtain or predict the final ratings.A functional prototype system has been implemented and tested with 50 different groups of users.

Evaluation and selection of group recommendation strategies for collaborative searching of learning objects

In recent years, Recommender System's models and techniques, have been applied in e-Learning and main efforts are been centered in learners, their guide and their success when using Learning Management Systems and other Web-based and virtual artifacts. Several techniques were adapted or even developed for this purposes. Our current project is concerned to the need to develop a Recommender System Architecture that may assist teachers in their e-Learning design practices. Instructional Design Methods, Learning Theories, Efficient Searching Methods and Tools and Meta-data Managing, are examples of the necessary knowledge and abilities. In this document we are centered in presenting the overall system architecture and describing their main parts. We are also reporting actual results. Due to it's wide conception, this project involves Knowledge Engineering, Software Engineering, Machine Learning, Semantic Web Searching, and Data Mining models and tools.

A Recommender System Architecture for Instructional Engineering

Ontologies represent a method of sharing and reusing knowledge on the semantic web. A fuzzy ontology is an extension of domain ontologies for solving the problems of uncertainty. This paper shows how a Fuzzy Ontology based approach can represent user profiles in e-learning environments. The ontological representation of the user profile enhances the performance in tasks such as filtering and information retrieval. An algorithm that allows automatically creating the construction of the ontology is also introduced. This approach has been integrated into a management tool for Learning Objects, in which each user profile is built from Learning Objects published by the user himself. The initial experiments confirm that the automatically obtained fuzzy ontology is a good representation of the user's preferences. The experiment results also indicate that the approach is useful and warrants further research.

A fuzzy ontology approach to represent user profiles in e-learning environments

In recent years, Learning Content Management 
Systems have become an increasingly used option in the eLearning for search and retrieval of specialized resources. 
Mainly because they provide environments where teachers 
catalog, publish and download resources in an easy way. The use 
of Learning Objects as storage units is a strong tendency in these 
systems. The Learning Objects promote reusability, and 
interoperability. This article presents a platform for Learning 
Object management; it uses assistance and recommendation 
approach for the Learning Objects development. AGORA offers 
an integrated set of tools for storage, search and reuse of 
instructional resources. Interoperability is guaranteed to be in 
accordance with specifications and standards defined in the eLearning.

http://rita.det.uvigo.es/201005/uploads/IEEE-RITA.2010.V5.N2.A3.pdf

Víctor H. Menéndez

Papers

A framework for recommendation in learning object repositories: An example of application in civil engineering

Evaluation and selection of group recommendation strategies for collaborative searching of learning objects

A Recommender System Architecture for Instructional Engineering

A fuzzy ontology approach to represent user profiles in e-learning environments

Sistemas de Gestión Integral de Objetos de Aprendizaje