The purpose of this study was to develop a measure of type 1 diabetes mellitus (T1DM) knowledge that is aimed at youth and is based on contemporary management standards. The Mercy What I Know About Diabetes (M-WIKAD) is a 19-item questionnaire that is a valid measure of diabetes knowledge in youth with T1DM. Factor analysis was conducted which indicated that five factors were associated with latent variables. This measure appears valid and promising as a new clinical and research tool. Children with T1DM and their families must acquire skills in order to become proficient in daily diabetes management. Patients with T1DM are also encouraged to maintain healthy lifestyles and to engage in daily physical activity. Finally, they need to have adequate coping skills in order to appropriately manage their diabetes, and ultimately, reduce the risk of acute and long-term complications.
Knowledge assessment tools are highly useful in clinical practice as they help diabetes care teams to customise education and clinical care based on the needs of patients and their families. There are numerous measurement instruments designed for adults with diabetes, including the widely-used Michigan Diabetes Knowledge Test (MDKT) and its recent revision,1,2 instruments designed to assess literacy and diabetes numeracy,3 and diabetes knowledge in rehabilitation patients.4 Several measures have been designed to target people with diabetes in specific populations, such as the Spanish-speaking population.5,6 There are several instruments designed specifically for youth with T1DM, including the Nutrition Knowledge Survey and the Nutricarb Quiz,7,8 and assessments of self-efficacy,9 parental involvement10 and problem-solving.11 The PedCarbQuiz is an important measure that specifically evaluates knowledge about carbohydrate counting and insulin dosing.12 Despite the availability of multiple tools for assessing knowledge related to diabetes self-management, the lack of simple, low-burden and digital survey instruments to measure diabetes knowledge among children and parents of children with T1DM has prevented their widespread deployment in clinical practices. Table 1 summarises validated, published knowledge measures for youth with T1DM.
In the present work, we addressed this gap in knowledge by developing the M-WIKAD, a short digital test designed to evaluate knowledge related to diabetes self-management among youth with T1DM. Specifically, we used a 15-person multidisciplinary team to create an updated diabetes knowledge assessment tool specifically for adolescents based on the seven self-care behaviours for success in diabetes management.13 This was a pilot study in which the M-WIKAD was distributed to a group of adolescents with T1DM. Subsequently, we leveraged multidisciplinary feedback and exploratory factor analysis (EFA) to select the highest-performing questions for inclusion in the final knowledge test. Finally, we performed initial validation studies on the M-WIKAD by relating test results to glycated haemoglobin (A1C) and to scores on the MDKT.
Methods
Participants
Adolescents with T1DM were recruited from the Children’s Mercy Kansas City network of diabetes clinics. Our centre provides care to more than 2,000 patients with T1DM. The inclusion criteria for this study were as follows: age 12–17 years; a diagnosis of T1DM according to American Diabetes Association (ADA) guidelines; and duration of T1DM for longer than 1 year. Eligible adolescents were approached in the clinic by a research coordinator. Seventy-seven youths completed the measure at baseline as part of their participation in a randomised control trial of motivational interviewing in adolescents with T1DM, for which the inclusion criteria was A1C >8.5% (69 mmol/mol).14,15 An additional 42 youth were recruited solely to complete the measure as part of the present study and had no A1C criteria. As such, the final sample represents 119 youths and data from each youth’s first experience with the measure.
Measures
Mercy – What I Know About Diabetes
We used a multi-stage process to design an initial ‘test’ version of the questionnaire. The M-WIKAD test questions were constructed to assess an individual’s knowledge of the goals and practice of diabetes self-care. First, a writing team composed of two physicians, two dietitians, two nurse educators, two psychologists and one nurse practitioner composed 88 draft questions specific to current diabetes treatment and self-care goals as outlined by the American Association of Diabetes Educators.16 Proposed questions were reviewed by a second multidisciplinary team (three physicians, one psychologist and two diabetes educators), which evaluated each question for relevance, accuracy and readability. Forty-nine questions were selected for initial testing in the present study based on their acceptability among reviewers (i.e., at least five of six reviewers endorsed the item). To make the questions understandable at a sixth-grade reading level, items were edited for grammar and readability by the lead author, but the essential content remained unchanged after editing. The questionnaire is in the English language and designed for individuals with medical care that is based on the current ADA Standards of Medical Care in Diabetes.16 This measure is designed to be used in clinical and/or research settings.
Michigan Diabetes Knowledge Test
The MDKT is a 23-item questionnaire which measures knowledge of diabetes treatment goals and treatment recommendations.1
Glycated haemoglobin
Youths’ A1C levels were collected during the baseline assessment visit using either a Tosoh G8 HPLC (Tosoh Bioscience Inc., San Francisco, CA) or the Afinion AS100 boronate affinity analyser (Alere Inc., Waltham, MA). Both instruments have documented traceability to the Diabetes Control and Complications Trial reference methods.17
Procedures
Parents provided written consent and youth provided assent. No study procedures were conducted until both consent from parent and assent from youth was obtained. The participants completed the knowledge measure electronically on an iPad using Research Electronic Data Capture (REDCap) tools hosted at CMH.18 REDCap is a secure, customisable web-based application designed to support data capture for research studies.
Statistical analysis
EFA19 was used to identify groups of highly correlated items that might represent latent factors. Because the M-WIKAD includes binary and ordinal measures, we also employed categorical principal component analysis (CATPCA)(SPSS 23) and compared these findings to the results from our EFA.20,21 For these analyses using our small sample size, we chose not to use traditional criteria for minimal acceptable loadings and Cronbach alphas, but instead we used more inclusive criteria to explore factor identifications. Thus, a priori, we decided to accept loadings of 0.350 or higher for factors identified by CATPCA and Cronbach alphas of >0.35. Factors were identified via SPSS 23, using CATPCA with a Varimax rotation method for final loadings.22 The Kaiser–Meyer–Olkin (KMO) and Bartlett’s tests both denoted that there were sufficient correlations among the 119 observations to make factor analysis feasible. The general interpretation of the KMO suggests that EFA may be adequate if values are >0.60. In contrast, for the Bartlett test, acceptable models should have p-values <0.05.23
Results
One hundred and nineteen youths completed the 49-item M-WIKAD questionnaire. A subset of the sample (n=77) also completed the MDKT as a baseline assessment during their participation in a clinical trial of motivational interviewing. The study team collected youth’s anthropometric data, sex, vital signs, duration of diabetes, insulin delivery (injections or pump), use of continuous glucose monitoring, concomitant medical diagnoses, laboratory results A1C from 6 months prior to enrolment, pubertal stage, history of smoking and occurrence of diabetic emergencies (diabetic ketoacidosis or severe hypoglycaemia) from the electronic health record. Youths were reimbursed with a $10 gift card for their participation.
Population demographics
Table 2 provides demographic information about the participants. The average age was 15.4 years, there were nearly an equal number of males and females, and the majority were non-Hispanic white. We noted that the average A1C was elevated at 9.9%±1.80 (84.7±19.7 mmol/mol), which is above target.16 Percentage scores on the test ranged from 50–95%, with a mean and SD of 83% and 7%, respectively, for all 49 questions. The 19 items found via CATPCA-EFA ranged from 33–98% with mean and SD of 89% and 10% respectively.
Factor structure
CATPCA-EFA analysis of the 49-item questionnaire yielded a five-factor 19-item model that accounted for 57% of the original variance. These factors identified latent variables that describe T1DM patient management characteristics that have been labelled ‘advanced problem solving’, ‘hypoglycaemia prevention and management’, ‘medication administration’, ‘daily management’ and ‘healthy active living’. Of the 19 questions that were identified, three were loaded on multiple factors, owing potentially to the lower limit for an acceptable loading. However, the factors and the factor loading for an EFA assuming a continuous scale for items in contrast to those transformed by CATPCA, yielded identical factors with nearly identical loadings, thus lending credence to the internal validity of the measure.
Table 3 summarises the five-factor CATPCA-EFA model. Only three out of the five factors had an acceptable Cronbach’s Alphas of ≥0.600. Nonetheless, the questions and loadings of all of the factors were justified from a clinical/theoretical perspective. The communalities of each question are shown in the last column. Items with communalities greater than or equal to 0.250 were included in factors, which yielded an item with as low a communality as 0.267. This is equivalent to a multiple correlation coefficient of about 0.50. Interpreting this correlation coefficient denotes that a 1 standard deviation (SD) increase in the factor loadings on an item yields about a 0.50 SD increase in the item’s score.
Test validation
Using the 49-(19) question version, the percentage correct on the knowledge test was inversely correlated with glycaemic control as measured by A1C (rPearson=-0.286, p<0.002), (rPearson=-0.275, p<0.002). The study evaluated the influences of several demographic variables on diabetes knowledge as a measure of test reliability. Race (white versus non-white) was not correlated to diabetes knowledge M-WIKAD (49 questions) [19 questions] in this study using Spearman’s rho (rrho=0.166, p=0.071), [rrho=0.134, p=0.148]. There was also no correlation between diabetes knowledge and sex (rrho=0.083, p=0.368), [rrho=0.000, p=0.996]. Age was a predictor of scores on the diabetes knowledge test for the 49 questions, (rrho=0.328, p<=0.000), but not as significant for the 19 questions [rrho=0.130, p=0.160]. The duration of diabetes was found to be correlated with the 49 questions (rrho=0.194, p=0.035) but not significantly with the 19 questions [rrho=0.117, p=0.206].
Cronbach’s alpha for standardised scores was 0.63 for all 49 questions and 0.70 for the subset of 19 questions found to load on five factors, where the alpha of 0.70 is considered acceptable. As shown in Figure 1, 19 different questions were identified the best-performing items based on the EFA: the fit for our final model was adequate (KMO = 0.649; Bartlett’s Test, χ2=549, df=171, p<0.0001).
External validation
Seventy-seven patients also took the MDKT, a previously validated instrument that was developed in 1998 and that assesses patient knowledge related to diabetes self-care.1 The correlation between scores on the M-WIKAD (49) and [19] questions and the MDKT using the Spearman’s Rank Correlation Coefficient (rrho=0.554, p<0.001), [rrho=0.261, p<0.025] suggested good concurrent validity In this sample, the MDKT showed adequate internal consistency (a ≥0.7).
Discussion
The M-WIKAD is a modern diabetes knowledge questionnaire specifically targeting youth. In this study, we showed that the M-WIKAD is both reliable based on correlations between youths’ knowledge scores and their age, and valid, based on correlations with the frequently cited MDKT.1 The EFA results suggested a five-factor structure that accounted for 57% of item variance. The following factors were identified: advanced problem-solving, hypoglycaemia prevention and management, taking insulin/medication administration, daily management and healthy active living.
The M-WIKAD scores are inversely correlated with youths’ glycaemic control as measured by A1C. This is similar to results from Beck et al. in 2015, though they explored an interaction between diabetes knowledge and A1C by diabetes duration.24 Age has also been found to predict diabetes knowledge in youths with T1DM.25 In particular, some studies have demonstrated a negative association between age at diagnosis and youths’ diabetes knowledge,25,26 while another study suggests that young children27 (aged 6–8 years) have significantly lower diabetes knowledge than older children. More recently, Koontz et al. found no correlation between youths knowledge scores and either their age at diagnosis, age at the time of the study or diabetes duration.12 In this analyses, age was directly associated with youths’ diabetes knowledge when looking at all piloted 49 questions (but not on the subset of 19 questions), which is consistent with findings from Bennett Johnson et al.27 There was an association between knowledge and diabetes duration when looking at all 49 questions, but not the subset of 19.
The M-WIKAD is clinically important and innovative in several ways. First, because it reflects current clinical diabetes practice in youth and because it is designed to be given electronically. The MDKT is one of the most widely used diabetes knowledge questionnaires and it has been revised based on changing diabetes therapy. The MDKT primarily targets adults with diabetes and is not specific for T1DM. The M-WIKAD is short, which makes it feasible to use in research and clinical practice. Since the time we started developing the M-WIKAD, Koontz et al. published a new diabetes knowledge tool for youth on flexible insulin regimens.12 While comprehensive, the Koontz measure includes 78 items, which would likely take too long to administer in a busy clinic setting. Our goal with the M-WIKAD was to create a brief knowledge measure that broadly assessed for information consistent with modern diabetes treatment.16 Again, the seven principles of the American Association of Diabetes Educators13 were used to help in guiding the initial content of the M-WIKAD. A brief instrument that is based on the educational goals from the American Association of Diabetes Educators will be more helpful in clinical situations as it will allow the care team to identify gaps in knowledge and customise education based on individual needs. For these reasons, the M-WIKAD has a place in clinical research, especially as an assessment tool for clinical intervention projects.
While most diabetes centres have formalised diabetes education programmes, few use standard, validated tools to measure knowledge —the desired effect of education programmes—and its impact on engagement and glycaemic control clinically. The identification of specific gaps in patient/family knowledge related to self-management creates an opportunity in the clinic for targeted diabetes education to improve patient understanding of the complex T1DM management regimen. In addition, it may offer the opportunity to better match interventions to patients who are most likely to benefit from those interventions. The M-WIKAD is a reliable and valid knowledge test for youth that may enable programmes to better assess the effectiveness of their current educational offerings as well as develop new targeted educational programmes.
Strengths of this study include its large sample size and the stepwise and systematic process that we followed in developing and refining the M-WIKAD. However, there are also some limitations. First, in our final 19-item questionnaire, the M-WIKAD may not assess for all aspects of diabetes care, which is both complex and multi-factorial. Second, some of the M-WIKAD items had a high percentage of correct answers by participants, which may indicate that they poorly discriminate between those with more or less knowledge. Specifically, with the M-WIKAD the percentage correct ranged from 51–95% – in contrast, the percentage correct ranged from 44–78% for the MDKT in our sample. Other knowledge questionnaires have demonstrated a wider range of correct responses on individual items, such as the Short Diabetes Knowledge instrument, which ranged from 6.8–89.2%28 or the PedCarbQuiz, which ranged from 42–98%.12 For the M-WIKAD, the dietary items seemed to be the most difficult for youths, which is contradictory to the results reported by Bennett Johnson et al., which found that the diet question to be the easiest for test-takers.27 In future versions of the M-WIKAD, more difficult questions will be trialled, which may decrease the measure’s overall percentage correct for items and ultimately help to better discriminate between youths with more or less knowledge. Third, this study is limited because only a subset of youths (77/119) completed the MDKT, which we used as our measure of concurrent validity. We also did not obtain some pertinent demographic information which may relate to youths’ diabetes knowledge such as parental education and parental diabetes knowledge.
In future versions of the M-WIKAD, these additional variables will be collected in order to fully explore their association with youths’ M-WIKAD scores. In addition, the 77 youths that were part of a different study in which an elevated A1C of ≥8.5% (69 mmol/mol) was part of the inclusion criteria,14,15 and this subset could have had less diabetes knowledge compared to those with glycaemic control closer to the target. Finally, the generalisability of the M-WIKAD may be at risk because the measure was only developed and piloted at one institution. The hospital system in which it was piloted includes a large clinical network of 13 sites, and the diabetes clinics employs 21 providers and 15 diabetes educators – therefore, in that sense it is somewhat mimicking the characteristics of a multi-centre study. Future endeavours will include a true multi-centre psychometric trial of the M-WIKAD to more adequately address any concerns related to its generalisability.
Implications for diabetes providers
The aim of this project was to develop an electronic diabetes knowledge assessment tool for youth that would be easy to deploy for both clinical and research use. As advocated by the ADA,16 education should be individualised and effort should thus be made to implement diabetes knowledge assessments in routine clinical care and diabetes education programmes. The M-WIKAD may be quite useful for this purpose and may also have numerous potential applications in research assessing novel treatment interventions designed to improve knowledge, adherence and glycaemic control among youth with T1DM. The future objectives will be to refine the M-WIKAD by trialling new items in another local sample and then retesting and updating its factor structure. Subsequently, a large multisite psychometrics trial of the M-WIKAD will be conducted and, if found reliable and valid, will be made available for use by clinicians and researchers.
