Read Time: 5 mins

Reconsidering Meaningful Outcomes in Diabetes mHealth Research

Copy Link
Published Online: Oct 24th 2019 US Endocrinology. 2019;15(2):74–6 DOI:
Authors: Katherine L Modzelewski, Kathryn L Fantasia, Devin W Steenkamp
Quick Links:
Article Information

Interest in improved access to diabetes care through mHealth interventions is widespread and is increasingly recognized as playing an important role in care. However, limited data is available on the use of mHealth in diabetes care, with traditional research models focusing on clinical outcomes. While traditional clinical outcomes are important when evaluating mHealth interventions, we propose that patient-reported outcomes be considered when evaluating mobile technology to ensure that technology is both clinically valuable and patient centric.


Diabetes technology, mHealth, clinical research endpoints, patient-reported outcomes


Technology has played a significant role in diabetes care for more than 50 years, from the advent of the original insulin pump to more recent artificial pancreas technologies. In healthcare more broadly, the use of technology through digital health has gained traction over the last 20 years, with interest expanding to mobile wireless technology, or mHealth.1 mHealth is defined as the use of mobile and wireless technologies to support health, and includes apps, text messaging systems, and web portals.2

With over 5 billion unique mobile subscribers worldwide, mHealth holds significant potential to expand access to healthcare to those for whom it may traditionally have been limited, due to either lack of access to care or low socioeconomic status. Diabetes, which is largely a patient-managed condition with growing prevalence in underserved communities, is an ideal candidate for mHealth integration.3,4 The role of technology in insulin delivery and glucose monitoring technologies is well documented, but the same is not necessarily true for mHealth.5 With an ever-expanding number of apps available, it is challenging for patients to identify which apps to use and for providers to be able to give evidence-based recommendations. A study of German- and English-speaking patients with diabetes, who were recruited from online diabetes communities, found that MySugr, Dexcom and MyFitnessPal were the most frequently used apps in 2019. In patients using these and other apps, there was a higher level of diabetes self-care, which included following a diet, physical activity, and blood-glucose monitoring.6 Despite this reported benefit, research into mHealth has lagged, with limited high-quality recommendations available to guide both consumers (patients) and care providers.

Safety, efficacy, and effectiveness of interventions have historically been the primary outcomes of interest in diabetes research. These clinical outcomes have remained the cornerstone of research, particularly with respect to glycemic control and the oft-chosen outcome of improvement in glycated hemoglobin (HbA1c). However, we propose that historically valuable clinical outcomes may not be the best measures of mHealth interventions, and alternate endpoints should be considered that evaluate technology from the patient viewpoint as a complement to clinical outcomes.

Clinical outcomes

Fingerstick blood glucose and HbA1c have long been the standard for clinical diabetes outcomes measures. This is largely as a result of data demonstrating an association with reduction in microvascular diabetes complications with reductions in these measures.7,8 While self-monitoring of blood glucose is well-established, the degree to which the information obtained is valuable is limited by the frequency and the timing of the specific blood sugar checks and may not accurately reflect overall trends, nor important nadir or peak glucose values. This is of particular concern as many studies focus on glucose data to assess efficacy, while this may not be appropriate for mHealth interventions such as meal tracking apps, where the app–user interface and usability is arguably more important and glucose data are less relevant as a primary outcome.

Even the gold standard HbA1c may not necessarily provide sufficient information to optimally assess intervention efficacy in mHealth studies. Wide variations in daily behaviors, interactivity with mHealth platforms, and blood sugar values can, on average, appear well-controlled based on an accepted HbA1c definition of <7%. Additionally, HbA1c may be inaccurate in individuals with a variety of medical conditions, who are often excluded from studies for this reason.9 The long duration of time required to see an effect on HbA1c can also be challenging when assessing mHealth interventions, where feedback is nearly instantaneous and iterative product development is the norm. Three months may be enough time for a new version of an app to become available, making the results obsolete.

Technological advances in diabetes, specifically the advent of real-time and intermittently-scanned continuous glucose monitoring (CGM) have allowed for improved characterization of glycemic variability through multiple measures and a new set of outcomes, including time in range and time spent in hypoglycemia. These technologies provide near continuous glucose values, allowing for a richer data set for analysis9,10 and most CGM devices now communicate directly with mobile devices and apps. Time in range can be adjusted to different targets, so it can be better individualized for groups in which the glycemic targets may not be as strict, such as elderly individuals or those with multiple medical comorbidities. Additionally, the time spent in hypoglycemia is an important safety consideration and identifying previously unidentified hypoglycemia can allow for specific interventions to reduce hypoglycemia. Glycemic variability is an increasingly important factor that is clearly associated with hypoglycemia and quality of life that has become available through more widespread incorporation of CGM into clinical care. Various methods of defining glycemic variability are available including, standard deviation, coefficient of variation, and interquartile glucose ranges. Although the benefits of these diabetes technologies are well-described, their integration with mHealth is still in the early stages and more studies and innovation are needed to determine how best to assess for potential benefits of the mHealth component itself.5

Patient-reported outcomes

Historically, patient-reported outcomes (PROs) have been notably excluded from clinical trials in diabetes care.11 PROs are defined as reports of health status, quality of life, or functional status that are directly reported by a patient.12 These outcomes are centrally important to those living with diabetes and provide important complementary data to guide clinical decision making and regulatory approvals. Given the increasing focus on patient-centered care and research, PROs are increasingly being included in clinical trials and have recently been integrated into evaluation and labeling of medical devices by the US Food and Drug Administration.11,13,14 However, there has yet to be agreement regarding which PROs to measure and how best to measure them. Additionally, mHealth is not considered to be a traditional medical device, so regulation surrounding mHealth does not necessarily follow the same standards as traditional diabetes technology.

At a national level, funding sources and societal groups, such as the Patient-Centered Outcomes Research Institute, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), and the American Diabetes Association (ADA) have called for increased attention to PROs through funding opportunities directed at improving PRO measures and determining how best to incorporate them into clinical care. Furthermore, the NIDDK and ADA convened a workshop of experts in 2017 to develop consensus and move towards increased use of standardized, validated, generalizable PRO measures both in research and clinical practice.15

With little data currently available surrounding diabetes mHealth, the incorporation of PROs into this research remains a limited, but essential, area for development. PROs are of particular importance for mHealth, as these mobile tools have evolved out of a movement to transition towards highly accessible, patient-centered care delivery. However, high rates of technology and mHealth application abandonment exist, and this must be overcome to make mHealth interventions useful.16 A recent retrospective study of real-world observations in patients using the MySugr app by Debong et al. included both clinical outcomes and patient satisfaction as end points, an important shift towards recognizing the need for both in study design.17

When evaluating mHealth, user experience in an important PRO, in addition to traditional measures such as patient burden and resultant quality of life measures. User experience is a broad outcome that encompasses the usability, accessibility, usefulness, desirability, navigability, and credibility of a technology.18 Usability has been defined as the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”19 Despite an expanding market with over 325,000 mHealth applications, usability is, unfortunately, frequently overlooked and scant data exists regarding usability in regards to diabetes mHealth.20 Fu et al. reported that an increase in app usability was associated with higher competence and autonomy in diabetes management, along with improved connectivity with healthcare providers, suggesting that focusing on these aspects of care may improve success with mHealth interventions.21 Usability is critical to the success and sustainability of any direct patient-facing health technology and ultimately, if a technology is found to have poor usability for the patient, the provider, or the larger health system, other outcomes are irrelevant.


As the mHealth market continues to rapidly expand, it is important to step back and consider the tools and metrics we employ to critically evaluate these technologies. mHealth is openly accessible to patients and not subject to strict regulatory oversight, and additionally is rapidly evolving. This becomes increasingly complicated as mHealth continues to directly integrate with traditional diabetes technologies. It is crucial that our clinical outcomes keep pace with technological advancements, but should not necessarily take precedent over PROs, and instead be used in conjunction with each other to fully assess an intervention. As mHealth is increasingly intended to facilitate access to care, in addition to traditional PROs such as burden, distress, and quality of life, the importance of user experience in technology and mHealth should not be overlooked as it plays a crucial role in their sustainability and effectiveness.

Article Information:

Katherine L Modzelewski, Kathryn L Fantasia, and Devin W Steenkamp have nothing to disclose in relation to this article.

Review Process

Double-blind peer review.


The named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship of this manuscript, take responsibility for the integrity of the work as a whole, and have given final approval for the version to be published.


Katherine L Modzelewski, Boston Medical Center, 720 Harrison Avenue, Suite 8100, Boston, MA 02118, USA. E:


No funding was received in the publication of this article.


This article involves a review of the literature and did not involve any studies with human or animal subjects performed by any of the authors.


August 29, 2019


  1. World Health Organisation. eHealth at WHO, 2019. Available at: (accessed May 13, 2019).
  2. World Health Organization. mHealth: New horizons for health through mobile technologies: second global survey on eHealth. Vol. 3, Global Observatory for eHealth series. 2011. Available at: (accessed October 12, 2019).
  3. Pew Research Center, Mobile Fact Sheet, 2018. Available at: (accessed May 13, 2019).
  4. GSMA Intelligence, The Mobile Economy 2018, 2018. Available at: (accessed October 12, 2019).
  5. Chamberlain JJ, Doyle-delgado K, Peterson L, Skolnik N. Diabetes technology: review of the 2019 American diabetes association standards of medical care in diabetes. Ann Intern Med. 2019; doi: 10.7326/M19-1638 [Epub ahead of print].
  6. Kebede MM, Pischke CR. Popular diabetes apps and the impact of diabetes app use on self-care behaviour: a survey among the digital community of persons with diabetes on social media. Front Endocrinol (Lausanne). 2019;10:220.
  7. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998;352:837–53.
  8. Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;330:141.
  9. Wright LA-C, Hirsch IB. Metrics beyond hemoglobin a1c in diabetes management: time in range, hypoglycemia, and other parameters. Diabetes Technol Ther. 2017;19(Suppl 2):S16–26.
  10. Rodbard D. The challenges of measuring glycemic variability. J Diabetes Sci Technol. 2012;6:21–2.
  11. Rodriguez-Gutierrez R, McCoy RG. Measuring what matters in diabetes. JAMA. 2019;321:1865–6.
  12. Weldring T, Smith SMS. Patient-reported outcomes (PROs) and Patient-reported outcome measures (PROMs). Heal Serv Insights. 2013;6:61–8.
  13. Spencer-Bonilla G, Quiñones AR, Montori VM. Assessing the burden of treatment. J Gen Intern Med. 2017;32:1141–5.
  14. US Food and Drug Administration. Patient-reported outcomes (PROs) in medical device decision making. Available at: (accessed May 28, 2019).
  15. Marrero DG, Hilliard ME, Maahs DM, et al. Using patient reported outcomes in diabetes research and practice: recommendations from a national workshop. Diabetes Res Clin Pract. 2019;153:23–9.
  16. Lee K, Kwon H, Lee B, et al. Effect of self-monitoring on long-term patient engagement with mobile health applications. PLoS One. 2018;13:1–12.
  17. Debong F, Mayer H, Kober J. Real-world assessments of mySugr mobile health app. Diabetes Technol Ther. 2019;21:S235–40.
  18. U.S. Department of Health and Human Services, User experience basics, 2019. Available at: (accessed May 13, 2019).
  19. National Institute of Standards and Technology, Health IT usability, 2017. Available at: (accessed May 13, 2019).
  20. Fu H, Mcmahon SK, Gross CR, et al. Usability and clinical efficacy of diabetes mobile applications for adults with type 2 diabetes: a systematic review. Diabetes Res Clin Pract. 2017;131:70–81.
  21. Fu HNC, Adam TJ, Konstan JA, et al. Influence of patient characteristics and psychological needs on diabetes mobile app usability in adults with type 1 or type 2 diabetes : crossover randomized trial corresponding author. JMIR Diabetes. 2019;4:1–12.

Further Resources

Share this Article
Related Content In Diabetes
  • Copied to clipboard!
    accredited arrow-down-editablearrow-downarrow_leftarrow-right-bluearrow-right-dark-bluearrow-right-greenarrow-right-greyarrow-right-orangearrow-right-whitearrow-right-bluearrow-up-orangeavatarcalendarchevron-down consultant-pathologist-nurseconsultant-pathologistcrosscrossdownloademailexclaimationfeedbackfiltergraph-arrowinterviewslinkmdt_iconmenumore_dots nurse-consultantpadlock patient-advocate-pathologistpatient-consultantpatientperson pharmacist-nurseplay_buttonplay-colour-tmcplay-colourAsset 1podcastprinter scenerysearch share single-doctor social_facebooksocial_googleplussocial_instagramsocial_linkedin_altsocial_linkedin_altsocial_pinterestlogo-twitter-glyph-32social_youtubeshape-star (1)tick-bluetick-orangetick-red tick-whiteticktimetranscriptup-arrowwebinar Sponsored Department Location NEW TMM Corporate Services Icons-07NEW TMM Corporate Services Icons-08NEW TMM Corporate Services Icons-09NEW TMM Corporate Services Icons-10NEW TMM Corporate Services Icons-11NEW TMM Corporate Services Icons-12Salary £ TMM-Corp-Site-Icons-01TMM-Corp-Site-Icons-02TMM-Corp-Site-Icons-03TMM-Corp-Site-Icons-04TMM-Corp-Site-Icons-05TMM-Corp-Site-Icons-06TMM-Corp-Site-Icons-07TMM-Corp-Site-Icons-08TMM-Corp-Site-Icons-09TMM-Corp-Site-Icons-10TMM-Corp-Site-Icons-11TMM-Corp-Site-Icons-12TMM-Corp-Site-Icons-13TMM-Corp-Site-Icons-14TMM-Corp-Site-Icons-15TMM-Corp-Site-Icons-16TMM-Corp-Site-Icons-17TMM-Corp-Site-Icons-18TMM-Corp-Site-Icons-19TMM-Corp-Site-Icons-20TMM-Corp-Site-Icons-21TMM-Corp-Site-Icons-22TMM-Corp-Site-Icons-23TMM-Corp-Site-Icons-24TMM-Corp-Site-Icons-25TMM-Corp-Site-Icons-26TMM-Corp-Site-Icons-27TMM-Corp-Site-Icons-28TMM-Corp-Site-Icons-29TMM-Corp-Site-Icons-30TMM-Corp-Site-Icons-31TMM-Corp-Site-Icons-32TMM-Corp-Site-Icons-33TMM-Corp-Site-Icons-34TMM-Corp-Site-Icons-35TMM-Corp-Site-Icons-36TMM-Corp-Site-Icons-37TMM-Corp-Site-Icons-38TMM-Corp-Site-Icons-39TMM-Corp-Site-Icons-40TMM-Corp-Site-Icons-41TMM-Corp-Site-Icons-42TMM-Corp-Site-Icons-43TMM-Corp-Site-Icons-44TMM-Corp-Site-Icons-45TMM-Corp-Site-Icons-46TMM-Corp-Site-Icons-47TMM-Corp-Site-Icons-48TMM-Corp-Site-Icons-49TMM-Corp-Site-Icons-50TMM-Corp-Site-Icons-51TMM-Corp-Site-Icons-52TMM-Corp-Site-Icons-53TMM-Corp-Site-Icons-54TMM-Corp-Site-Icons-55TMM-Corp-Site-Icons-56TMM-Corp-Site-Icons-57TMM-Corp-Site-Icons-58TMM-Corp-Site-Icons-59TMM-Corp-Site-Icons-60TMM-Corp-Site-Icons-61TMM-Corp-Site-Icons-62TMM-Corp-Site-Icons-63TMM-Corp-Site-Icons-64TMM-Corp-Site-Icons-65TMM-Corp-Site-Icons-66TMM-Corp-Site-Icons-67TMM-Corp-Site-Icons-68TMM-Corp-Site-Icons-69TMM-Corp-Site-Icons-70TMM-Corp-Site-Icons-71TMM-Corp-Site-Icons-72