Background

There is growing evidence for the positive impact of mindfulness on wellbeing. Mindfulness-based mobile apps may have potential as an alternative delivery medium for training. While there are hundreds of such apps, there is little information on their quality.

Objective

This study aimed to conduct a systematic review of mindfulness-based iPhone mobile apps and to evaluate their quality using a recently-developed expert rating scale, the Mobile Application Rating Scale (MARS). It also aimed to describe features of selected high-quality mindfulness apps.

Methods

A search for “mindfulness” was conducted in iTunes and Google Apps Marketplace. Apps that provided mindfulness training and education were included. Those containing only reminders, timers or guided meditation tracks were excluded. An expert rater reviewed and rated app quality using the MARS engagement, functionality, visual aesthetics, information quality and subjective quality subscales. A second rater provided MARS ratings on 30% of the apps for inter-rater reliability purposes.

Results

The “mindfulness” search identified 700 apps. However, 94 were duplicates, 6 were not accessible and 40 were not in English. Of the remaining 560, 23 apps met inclusion criteria and were reviewed. The median MARS score was 3.2 (out of 5.0), which exceeded the minimum acceptable score (3.0). The Headspace app had the highest average score (4.0), followed by Smiling Mind (3.7), iMindfulness (3.5) and Mindfulness Daily (3.5). There was a high level of inter-rater reliability between the two MARS raters.

Conclusions

Though many apps claim to be mindfulness-related, most were guided meditation apps, timers, or reminders. Very few had high ratings on the MARS subscales of visual aesthetics, engagement, functionality or information quality. Little evidence is available on the efficacy of the apps in developing mindfulness.

Review and evaluation of mindfulness-based iPhone apps

Background: In recent years, there has been rapid growth in the availability and use of mobile health (mHealth) apps around the world. A consensus regarding an accepted standard to assess the quality of such apps has yet to be reached. A factor that exacerbates the challenge of mHealth app quality assessment is variations in the interpretation of quality and its subdimensions. Consequently, it has become increasingly difficult for health care professionals worldwide to distinguish apps of high quality from those of lower quality. This exposes both patients and health care professionals to unnecessary risks. Despite progress, limited understanding of the contributions of researchers in low- and middle-income countries (LMICs) exists on this topic. Furthermore, the applicability of quality assessment methodologies in LMIC settings remains relatively unexplored.
Objective: This rapid review aims to identify current methodologies in the literature to assess the quality of mHealth apps, understand what aspects of quality these methodologies address, determine what input has been made by authors from LMICs, and examine the applicability of such methodologies in LMICs.
Methods: This review was registered with PROSPERO (International Prospective Register of Systematic Reviews). A search of PubMed, EMBASE, Web of Science, and Scopus was performed for papers related to mHealth app quality assessment methodologies, which were published in English between 2005 and 2020. By taking a rapid review approach, a thematic and descriptive analysis of the papers was performed.
Results: Electronic database searches identified 841 papers. After the screening process, 52 papers remained for inclusion. Of the 52 papers, 5 (10%) proposed novel methodologies that could be used to evaluate mHealth apps of diverse medical areas of interest, 8 (15%) proposed methodologies that could be used to assess apps concerned with a specific medical focus, and 39 (75%) used methodologies developed by other published authors to evaluate the quality of various groups of mHealth apps. The authors in 6% (3/52) of papers were solely affiliated to institutes in LMICs. A further 15% (8/52) of papers had at least one coauthor affiliated to an institute in an LMIC.
Conclusions: Quality assessment of mHealth apps is complex in nature and at times subjective. Despite growing research on this topic, to date, an all-encompassing appropriate means for evaluating the quality of mHealth apps does not exist. There has been engagement with authors affiliated to institutes across LMICs; however, limited consideration of current generic methodologies for application in LMIC settings has been identified.
Trial Registration: PROSPERO CRD42020205149; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=205149

Identification and Evaluation of Methodologies to Assess the Quality of Mobile Health Apps in High-, Low-, and Middle-Income Countries: Rapid Review.

Background The number of mobile health (mHealth) apps continues to rise each year. Widespread use of the Mobile App Rating Scale (MARS) has allowed objective and multidimensional evaluation of the quality of these apps. However, no Japanese version of MARS has been made available to date. Objective The purposes of this study were (1) to develop a Japanese version of MARS and (2) to assess the translated version’s reliability and validity in evaluating mHealth apps. Methods To develop the Japanese version of MARS, cross-cultural adaptation was used using a universalist approach. A total of 50 mental health apps were evaluated by 2 independent raters. Internal consistency and interrater reliability were then calculated. Convergent and divergent validity were assessed using multitrait scaling analysis and concurrent validity. Results After cross-cultural adaptation, all 23 items from the original MARS were included in the Japanese version. Following translation, back-translation, and review by the author of the original MARS, a Japanese version of MARS was finalized. Internal consistency was acceptable by all subscales of objective and subjective quality (Cronbach α=.78-.89). Interrater reliability was deemed acceptable, with the intraclass correlation coefficient (ICC) ranging from 0.61 to 0.79 for all subscales, except for “functionality,” which had an ICC of 0.40. Convergent/divergent validity and concurrent validity were also considered acceptable. The rate of missing responses was high in several items in the “information” subscale. Conclusions A Japanese version of MARS was developed and shown to be reliable and valid to a degree that was comparable to the original MARS. This Japanese version of MARS can be used as a standard to evaluate the quality and credibility of mHealth apps.

https://jmir.org/api/download?alt_name=mhealth_v10i4e33725_app1.pdf&filename=e931d7dee49108cff85fb5efaa7aa770.pdf

Japanese Version of the Mobile App Rating Scale (MARS): Development and Validation

The global burden of disease attributes 20% of deaths to poor nutrition. Although hundreds of nutrition-related mobile apps have been created, and these have been downloaded by millions of users, the effectiveness of these technologies on the adoption of healthy eating has had mixed.The aim of this study was to review which nutrition-related mobile apps are currently available on the French market and assess their quality.We screened apps on the Google Play Store and the French Apple App Store, from March 10 to 17, 2021, to identify those related to nutritional health. A shortlist of 15 apps was identified, and each was assessed using the French version of the Mobile App Rating Scale: 8 dietitians and nutritionists assessed 7 apps, and the remaining apps were randomly allocated to ensure 4 assessments per app. Intraclass correlation was used to evaluate interrater agreement. Means and standard deviations of scores for each section and each item were calculated.The top scores for overall quality were obtained by Yazio - Régime et Calories (mean 3.84, SD 0.32), FeelEat (mean 3.71, SD 0.47), and Bonne App (mean 3.65, SD 0.09). Engagement scores ranged from a mean of 1.95 (SD 0.5) for iEatBetter: Journal alimentaire to a mean of 3.85 (SD 0.44) for FeelEat. Functionality scores ranged from a mean of 2.25 (SD 0.54) for Naor to a mean of 4.25 (SD 0.46) for Yazio. Aesthetics scores ranged from a mean of 2.17 (SD 0.34) for Naor to a mean of 3.88 (SD 0.47) for Yazio. Information scores ranged from a mean of 2.38 (SD 0.60) for iEatBetter to a mean of 3.73 (SD 0.29) for Yazio. Subjective quality scores ranged from a mean of 1.13 (SD 0.25) for iEatBetter to a mean of 2.28 (SD 0.88) for Compteur de calories FatSecret. Specificity scores ranged from a mean of 1.38 (SD 0.64) for iEatBetter to a mean of 3.50 (SD 0.91) for FeelEat. The app-specific score was always lower than the subjective quality score, which was always lower than the quality score, which was lower than the rating from the iOS or Android app stores.Although prevention and information messages in apps regarding nutritional habits are not scientifically verified before marketing, we found that app quality was good. Subjective quality and specificity were associated with lower ratings. Further investigations are needed to assess whether information from these apps is consistent with recommendations and to determine the long-term impacts of these apps on users. 

https://jmir.org/api/download?alt_name=mhealth_v10i3e35879_app1.pdf&filename=6344530f42c46a44a543a62b396babb0.pdf

Nutrition-Related Mobile Apps in the French App Stores: Assessment of Functionality and Quality

The rapid growth of mHealth applications for Type 2 Diabetes Mellitus (T2DM) patients’ self-management has motivated the evaluation of these applications from both the usability and user point of view. The objective of this study was to identify mHealth applications that focus on T2DM from the Android store and rate them from the usability perspective using the MARS tool. Additionally, a classification of these mHealth applications was conducted using the ID3 algorithm to identify the most preferred application. The usability of the applications was assessed by two experts using MARS. A total of 11 mHealth applications were identified from the initial search, which fulfilled our inclusion criteria. The usability of the applications was rated using the MARS scale, from 1 (inadequate) to 5 (excellent). The Functionality (3.23) and Aesthetics (3.22) attributes had the highest score, whereas Information (3.1) had the lowest score. Among the 11 applications, “mySugr” had the highest average MARS score for both Application Quality (4.1/5) as well as Application Subjective Quality (4.5/5). Moreover, from the classification conducted using the ID3 algorithm, it was observed that 6 out of 11 mHealth applications were preferred for the self-management of T2DM.

/pdf/usability-evaluation-and-classification-of-mhealth-u0x2zc8i.pdf

Usability Evaluation and Classification of mHealth Applications for Type 2 Diabetes Mellitus Using MARS and ID3 Algorithm

Digital health interventions may improve different behaviours. However, the rapid proliferation of technological solutions often does not allow for a correct assessment of the quality of the tools. This study aims to review and assess the quality of the available mobile applications (apps) related to interventions for low back pain. Two reviewers search the official stores of Android (Play Store) and iOS (App Store) for localisation in Spain and the United Kingdom, in September 2019, searching for apps related to interventions for low back pain. Seventeen apps finally are included. The quality of the apps is measured using the Mobile App Rating Scale (MARS). The scores of each section and the final score of the apps are retrieved and the mean and standard deviation obtained. The average quality ranges between 2.83 and 4.57 (mean 3.82) on a scale from 1 (inadequate) to 5 (excellent). The best scores are found in functionality (4.7), followed by aesthetic content (mean 4.1). Information (2.93) and engagement (3.58) are the worst rated items. Apps generally have good overall quality, especially in terms of functionality and aesthetics. Engagement and information should be improved in most of the apps. Moreover, scientific evidence is necessary to support the use of applied health tools.

https://www.mdpi.com/1660-4601/17/24/9209/pdf

Assessment of the Quality of Mobile Applications (Apps) for Management of Low Back Pain Using the Mobile App Rating Scale (MARS).

Background: Sedentary behavior (SB) negatively impact health and is highly prevalent in the population. Digital behavior change interventions (DBCIs) have been developed to modify behaviors such as SB by technologies. However, it is unknown which behavior change techniques (BCTs) are most frequently employed in SB as well as the effect associated with DBCIs in this field. The aim of this systematic review was: (a) to evaluate the BCT most frequently employed in digital health including all technologies available and interventions aimed at increasing physical activity (PA), reducing sedentary time, and improving adherence to exercise in the clinical population, and (b) to review the effect associated with DBCIs in this field. Methods: The database used was Medline, as well as Scopus, Scielo, and Google Scholar. For the search strategy, we considered versions of behavior/behavioral, mHealth/eHealth/telemedicine/serious game/gamification. The terms related to PA and SB were included, the criteria for inclusion were randomized clinical trials (RCTs), adults, intervention based on digital media, and outcome variable lifestyle modification; a last 5 years filter was included. Michie's Taxonomy was used to identify BCTs. The study was registered under the number PROSPERO CRD42019138681. Results: Eighteen RCTs were included in the present systematic review, 5 of them healthy adults, and 13 of them with some illness. Studies included 2298 sedentary individuals who were followed up for 5 weeks-3 years. The most used BCTs were goal setting, problem solving, review outcomes/goals, feedback on behavior and outcomes of behavior, self-monitoring of behavior, social support, information about health consequences, and behavior practice/rehearsal. The effect associated with DBCIs showed improvements, among several related to PA and physiologic self-reported and anthropometric outcomes. Conclusion: The BCTs most used in digital health to change outcomes related to SB were goals and planning, feedback and monitoring, social support, natural consequences, repetition, and substitution. Besides these findings, DBCIs are influenced by several factors like the type of intervention, patients' preferences and values, or the number of BCTs employed. More research is needed to determine with precision which DBCIs or BCTs are the most effective to reduce SB in the clinical population.

/pdf/behavior-change-techniques-and-the-effects-associated-with-2dtgr1ls2o.pdf

Behavior Change Techniques and the Effects Associated With Digital Behavior Change Interventions in Sedentary Behavior in the Clinical Population: A Systematic Review.

Background: There are several mobile health (mHealth) apps in mobile app stores. These apps enter the business-to-customer market with limited controls. Both, apps that users use autonomously and those designed to be recommended by practitioners require an end-user validation to minimize the risk of using apps that are ineffective or harmful. Prior studies have reviewed the most relevant aspects in a tool designed for assessing mHealth app quality, and different options have been developed for this purpose. However, the psychometric properties of the mHealth quality measurement tools, that is, the validity and reliability of the tools for their purpose, also need to be studied. The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) initiative has developed tools for selecting the most suitable measurement instrument for health outcomes, and one of the main fields of study was their psychometric properties.
Objective: This study aims to address and psychometrically analyze, following the COSMIN guideline, the quality of the tools that are used to measure the quality of mHealth apps.
Methods: From February 1, 2019, to December 31, 2019, 2 reviewers searched PubMed and Embase databases, identifying mHealth app quality measurement tools and all the validation studies associated with each of them. For inclusion, the studies had to be meant to validate a tool designed to assess mHealth apps. Studies that used these tools for the assessment of mHealth apps but did not include any psychometric validation were excluded. The measurement tools were analyzed according to the 10 psychometric properties described in the COSMIN guideline. The dimensions and items analyzed in each tool were also analyzed.
Results: The initial search showed 3372 articles. Only 10 finally met the inclusion criteria and were chosen for analysis in this review, analyzing 8 measurement tools. Of these tools, 4 validated ≥5 psychometric properties defined in the COSMIN guideline. Although some of the tools only measure the usability dimension, other tools provide information such as engagement, esthetics, or functionality. Furthermore, 2 measurement tools, Mobile App Rating Scale and mHealth Apps Usability Questionnaire, have a user version, as well as a professional version.
Conclusions: The Health Information Technology Usability Evaluation Scale and the Measurement Scales for Perceived Usefulness and Perceived Ease of Use were the most validated tools, but they were very focused on usability. The Mobile App Rating Scale showed a moderate number of validated psychometric properties, measures a significant number of quality dimensions, and has been validated in a large number of mHealth apps, and its use is widespread. It is suggested that the continuation of the validation of this tool in other psychometric properties could provide an appropriate option for evaluating the quality of mHealth apps.

Tools for Evaluating the Content, Efficacy, and Usability of Mobile Health Apps According to the Consensus-Based Standards for the Selection of Health Measurement Instruments: Systematic Review.

Irene De-Torres

Papers

Assessment of the Quality of Mobile Applications (Apps) for Management of Low Back Pain Using the Mobile App Rating Scale (MARS).

Behavior Change Techniques and the Effects Associated With Digital Behavior Change Interventions in Sedentary Behavior in the Clinical Population: A Systematic Review.

Tools for Evaluating the Content, Efficacy, and Usability of Mobile Health Apps According to the Consensus-Based Standards for the Selection of Health Measurement Instruments: Systematic Review.