Online assessment in undergraduate medical education: Challenges and solutions from a LMIC university
TL;DR: The planning, processes, and outcomes of online assessments using video communication platforms conducted at a private university in Pakistan are described to device an effective protocol for e-assessments conducting multiple trial runs, and incorporating feedback from all stakeholders is a necessity.
Abstract: Background and Objectives: The Covid-19 pandemic has caused large-scale disruption in almost all educational programs across the world. Planning and rapid implementation of assessment through an online format presents the next set of novel challenges that must be addressed by academic administrations across the globe.
Methods: This cross-sectional study was conducted between March to August 2020 at the Aga Khan University Medical College. Two hundred medical students of year 1 and 2 participated in the study. We describe the planning, processes, and outcomes of online assessments using video communication platforms conducted at a private university in Pakistan. Standardized protocols were written and piloted, extensive training of student, proctors and staff for preparation and conduct of online assessments were developed. Feedback was recorded after each session and suggestions were incorporated in subsequent high-stakes assessments.
Results: A total of three pilot assessments were conducted to identify issues and process refinement. Commercially available lockdown browser and ZOOM were used in the first pilot; 80% of the class was unable to launch lockdown browser and laptops required repeated reload/reboot. For the second pilot assessment, University’s VLE page & MS Teams was trailed. Issues with internet connectivity, VLE page slowdown, and suboptimal recording feature in MS Teams were identified. For the final pilot assessment, phased launching of VLE page with single test item per page was implemented with success. The students reported that attempting the online exam on VLE with ZOOM support was user friendly. Ninety percent of the class was supportive of the continuing with the online assessments.
Conclusion: In order to device an effective protocol for e-assessments conducting multiple trial runs, and incorporating feedback from all stakeholders is a necessity.
doi: https://doi.org/10.12669/pjms.37.4.3948
How to cite this:Fatima SS, Idrees R, Jabeen K, Sabzwari S, Khan S. Online assessment in undergraduate medical education: Challenges and solutions from a LMIC university. Pak J Med Sci. 2021;37(4):945-951. doi: https://doi.org/10.12669/pjms.37.4.3948
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Citations
More filters
TL;DR: In this paper , the authors conducted a systematic literature review to investigate the latest findings regarding plagiarism, cheating, collusion, and using jockeys among students in online learning and found that individual factors such as being lazy to learn, lack of ability, and poor awareness as well as situational factors including the influence of friends, the pressure of the courses, and ease of access to information were strongly associated with AD.
Abstract: Background Academic dishonesty (AD) and trustworthy assessment (TA) are fundamental issues in the context of an online assessment. However, little systematic work currently exists on how researchers have explored AD and TA issues in online assessment practice. Objectives Hence, this research aimed at investigating the latest findings regarding AD forms, factors affecting AD and TA, and solutions to reduce AD and increase TA to maintain the quality of online assessment. Methods We reviewed 52 articles in Scopus and Web of Science databases from January 2017 to April 2021 using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses model as a guideline to perform a systematic literature review that included three stages, namely planning, conducting, and reporting. Results and conclusions Our review found that there were different forms of AD among students in online learning namely plagiarism, cheating, collusion, and using jockeys. Individual factors such as being lazy to learn, lack of ability, and poor awareness as well as situational factors including the influence of friends, the pressure of the courses, and ease of access to information were strongly associated with AD. A technology-based approach such as using plagiarism-checking software, multi-artificial intelligence (AI) in a learning management system, computer adaptive tests, and online proctoring as well as pedagogical-based approaches, such as implementing a research ethics course programme, and a re-design assessment form such as oral-based and dynamic assessment to reduce cheating behaviour and also sociocultural and sociotechnical adjustment related to the online assessment are reported to reduce AD and increase TA. Implications Educators should adjust the design of online learning and assessment methods as soon as possible. The identified gaps point towards unexplored study on AI, machine learning, learning analytics tools, and related issues of AD and TA in K12 education could motivated future work in the field.
16 citations
TL;DR: Evidence that, in the midst of the Covid‐19 pandemic, higher education institutions were largely influenced by cost, usability and efficiency in choosing online proctoring solutions to ensure academic and institutional integrity is found.
Abstract: Background Objectives Methods Results and Conclusions The Covid‐19 pandemic disrupted higher education in many ways, such as the move to Emergency Remote Online Teaching and Learning (EROTL), often including a move to online assessments and examinations. With evidence of increased academic dishonesty in unproctored online assessment, institutions sought ways to ensure academic and institutional integrity and reputation. In doing this, many institutions selected and implemented online proctoring solutions.This article maps considerations of online proctoring solutions in the nexus between ensuring academic and institutional integrity and reputation, and addressing stakeholder concerns regarding invasive surveillance and the impacts on student privacy.The study involved a PRISMA‐informed systematic review of three digital libraries, namely Clarivate's Web of Science, Elsevier's Scopus, and Springer's SpringerLink, for peer‐reviewed journal articles and conference proceedings. After screening, a final corpus of 27 articles was analysed.The findings include evidence that, in the midst of the Covid‐19 pandemic, higher education institutions were largely influenced by cost, usability and efficiency in choosing online proctoring solutions to ensure academic and institutional integrity. Student privacy was either considered in terms of data protection and transparency, or not at all. This article aims to provide valuable insight into the criteria used to select online proctoring solutions to ensure academic and institutional integrity in online examination environments. Student privacy appears not to have the consideration it warrants. [ FROM AUTHOR] Copyright of Journal of Computer Assisted Learning is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full . (Copyright applies to all s.)
6 citations
TL;DR: The proposed invigilation model can be implemented in colleges, universities, and schools to detect and monitor student suspicious activities and can prevent and solve the problem of cheating because it is unethical.
Abstract: Examination cheating activities like whispering, head movements, hand movements, or hand contact are extensively involved, and the rectitude and worthiness of fair and unbiased examination are prohibited by such cheating activities. The aim of this research is to develop a model to supervise or control unethical activities in real-time examinations. Exam supervision is fallible due to limited human abilities and capacity to handle students in examination centers, and these errors can be reduced with the help of the Automatic Invigilation System. This work presents an automated system for exams invigilation using deep learning approaches i.e., Faster Regional Convolution Neural Network (RCNN). Faster RCNN is an object detection algorithm that is implemented to detect the suspicious activities of students during examinations based on their head movements, and for student identification, MTCNN (Multi-task Cascaded Convolutional Neural Networks) is used for face detection and recognition. The training accuracy of the proposed model is 99.5% and the testing accuracy is 98.5%. The model is fully efficient in detecting and monitoring more than 100 students in one frame during examinations. Different real-time scenarios are considered to evaluate the performance of the Automatic Invigilation System. The proposed invigilation model can be implemented in colleges, universities, and schools to detect and monitor student suspicious activities. Hopefully, through the implementation of the proposed invigilation system, we can prevent and solve the problem of cheating because it is unethical.
2 citations
TL;DR: E-assessment was well accepted by most of the students and should target students inconvenience for online assessment, and new soft wares can be developed for a more reliable and valid assessment method.
Abstract: OBJECTIVE: Technology enhanced assessment or e-assessment is an emerging approach across the globe and is adapted by almost all medical and dental institutes. The objective of this study is to report on effects of COVID-19 on assessment format of medical institutes. This review seeks to find which methods are being used for a changed assessment format and who is being affected by sudden change in assessment format and what facilitators and/or barriers have been reported related to the success and/or failure of changed assessment format.
MATERIAL AND METHODS: This scoping review followed the methodology outlined by Arksey and O’Malley framework (2005).Data sources including PubMed, Eric, Medline, Semantic scholar and Google scholar were searched for last 5 years of articles from 20016 to 2021 related to e-assessment in medical education.
RESULTS: In total, 9 articles met our inclusion criteria. Open book exam was considered to be a new normal keeping in mind its strengths and advantages in an online format. Digital literacy plays an important role in the acceptance of technology enhanced assessment. Faculty need training to opt for this sudden change in teaching and assessment format. Connectivity issues need to be taken care of, as the whole format depends upon it.
CONCLUSION: e-assessment was well accepted by most of the students. Future studies should target students inconvenience for online assessment. Also lack of e-learning experience of some staff need attention. Insufficiency of IT technicians should be overcome. High speed internet is the requirement of the day. New soft wares can be developed for a more reliable and valid assessment method
1 citations
TL;DR: In this article , a review was conducted across Google Scholar, PubMed/MEDLINE, Science Direct, Directory of Open Access Journals, Complementary Index, SCOPUS, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL) with the intention of identifying opportunities and challenges presented by e-assessment in the HSE practical examination for student nurse educators during the COVID-19 pandemic.
Abstract: (1) Background: There is a gap in the literature that explores challenges and opportunities relating to virtual or e-assessment health science education with particular relevance to the Health Sciences Education practical examination for student nurse educators. Therefore, this review aimed to address this gap and provide recommendations for enhancing identified opportunities and for overcoming identified challenges.; (2) Methods: The review was conducted across Google Scholar, PubMed/MEDLINE, Science Direct, Directory of Open Access Journals, Complementary Index, SCOPUS, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL) with the intention of identifying opportunities and challenges presented by e-assessment in the HSE practical examination for student nurse educators during the COVID-19 pandemic.; (3) Results: The following aspects are discussed: (1) opportunities, including benefits, for both student nurse educators and facilitators and opportunities for Nursing Education; and (2) challenges, including issues with accessibility and connectivity as well as the attitudes of both students and facilitators.; (4) Conclusions: Despite challenges which included connectivity issues that led to frustration and stress, the unpreparedness and attitudes of students and facilitators, there are some opportunities that have emerged from e-assessment that can be beneficial to both the students and the facilitators, as well as the institutions. These include a reduced administrative burden, improved teaching and learning, and immediate feedback from facilitators to students and from students to facilitators.
1 citations
References
More filters
TL;DR: Despite rigorous global containment and quarantine efforts, the incidence of COVID-19 continues to rise, with 90,870 laboratory-confirmed cases and over 3,000 deaths worldwide.
4,124 citations
TL;DR: The socio-economic effects of COVID-19 on individual aspects of the world economy are summarised to show the need for medical supplies has significantly increased and the food sector has seen a great demand due to panic-buying and stockpiling of food products.
4,060 citations
TL;DR: Findings suggest that the key barriers which affect the development and implementation of online learning in medical education include time constraints, poor technical skills, inadequate infrastructure, absence of institutional strategies and support and negative attitudes of all involved.
Abstract: The aim of this study is to review the literature on known barriers and solutions that face educators when developing and implementing online learning programs for medical students and postgraduate trainees. An integrative review was conducted over a three-month period by an inter-institutional research team. The search included ScienceDirect, Scopus, BioMedical, PubMed, Medline (EBSCO & Ovid), ERIC, LISA, EBSCO, Google Scholar, ProQuest A&I, ProQuest UK & Ireland, UL Institutional Repository (IR), UCDIR and the All Aboard Report. Search terms included online learning, medical educators, development, barriers, solutions and digital literacy. The search was carried out by two reviewers. Titles and abstracts were screened independently and reviewed with inclusion/exclusion criteria. A consensus was drawn on which articles were included. Data appraisal was performed using the Critical Appraisal Skills Programme (CASP) Qualitative Research Checklist and NHMRC Appraisal Evidence Matrix. Data extraction was completed using the Cochrane Data Extraction Form and a modified extraction tool. Of the 3101 abstracts identified from the search, ten full-text papers met the inclusion criteria. Data extraction was completed on seven papers of high methodological quality and on three lower quality papers. Findings suggest that the key barriers which affect the development and implementation of online learning in medical education include time constraints, poor technical skills, inadequate infrastructure, absence of institutional strategies and support and negative attitudes of all involved. Solutions to these include improved educator skills, incentives and reward for the time involved with development and delivery of online content, improved institutional strategies and support and positive attitude amongst all those involved in the development and delivery of online content. This review has identified barriers and solutions amongst medical educators to the implementation of online learning in medical education. Results can be used to inform institutional and educator practice in the development of further online learning.
513 citations
466 citations
TL;DR: Despite the seriousness of the health situation, this new drive to move to online learning provides the necessary opportunity to assess the future approach to curriculum design and delivery for the anatomical sciences.
Abstract: \"All hands on deck, we need to move everything online as quickly as possible\". This is the message that has been ringing out across universities worldwide over the last couple of months as the Covid-19 pandemic has taken hold. Anatomists have responded to this call enthusiastically and creatively, ensuring that their students have resources available to continue and support their learning as they are forced to study remotely and exercise social distancing and where necessary self-isolation. Despite the seriousness of the health situation, this new drive to move to online learning provides the necessary opportunity to assess our future approach to curriculum design and delivery for the anatomical sciences. This pause to reevaluate is necessary as online learning, offers up refreshing and pedagogically innovative approaches to facilitate learning. For many courses and programs however, it is not the long-term silver bullet solution to all learning provision and has consequences that must be considered.
231 citations