1.1. Early Intervention and Health Literacy
1.2. The Health Literacy Questionnaire
1.3. Qualitative Validity Testing and Cognitive Interviewing
2. Materials and Methods
2.1. Study Design
- Comparison of EI providers’ chosen HLQ score on each item and their narrative responses (RQ1): coded as match, no match, or unclear.
- Comparison of EI providers’ narrative data to the HLQ item intent descriptions provided by the developers of the HLQ (RQ2): coded as concordant, discordant, or unclear.
- Qualitative thematic analysis of narrative data across participants to identify reasons for coding of no match or discordance between the HLQ item intents, the HLQ item scores chosen, and interview narrative data (RQ3).
Health Literacy Questionnaire (HLQ)
2.6. Data Collection
2.7. Data Analysis
- Theme 1. USA culture/health system
- Sub-theme 1.1.: Word “entitled”.
- Sub-theme 1.2.: Difference in USA vs. Australian insurance.
- Theme 2. The healthcare provider perspective
- Sub-theme 2.1.: Answering based on provider perspective.
- Sub-theme 2.2.: Only thinking of doctors or healthcare providers for support (not family/friends).
- Theme 3. Respondents with no health problems to manage
Conflicts of Interest
- U.S. Department of Education’s Individuals with Disabilities Education Act (IDEA) Website. Available online: https://sites.ed.gov/idea/ (accessed on 29 January 2020).
- Yin, H.S.; Johnson, M.; Mendelsohn, A.L.; Abrams, M.A.; Sanders, L.M.; Dreyer, B.P. The health literacy of parents in the United States: A nationally representative study. Pediatrics 2009, 124 (Suppl. 3), S289–S298. [Google Scholar] [CrossRef]
- Kutner, M.; Greenberg, E.; Jin, Y.; Paulsen, C. The Health Literacy of America’s Adults: Results from the 2003 National Assessment of Adult Literacy (NCES 2006-483); National Center for Educational Statistics, US Department of Education: Washington, DC, USA, 2006.
- DeWalt, D.A.; Hink, A. Health literacy and child health outcomes: A systematic review of the literature. Pediatrics 2009, 124, S265–S274. [Google Scholar] [CrossRef] [PubMed]
- Shone, L.P.; Conn, K.M.; Sanders, L.; Halterman, J.S. The role of parent health literacy among urban children with persistent asthma. Patient Educ. Couns. 2009, 75, 368–375. [Google Scholar] [CrossRef] [PubMed]
- Yin, H.S.; Dreyer, B.P.; Vivar, K.L.; MacFarland, S.; Van Schaick, L.; Mendelsohn, A.L. Perceived barriers to care and attitudes towards shared decision-making among low socioeconomic status parents: Role of health literacy. Acad. Pediatr. 2012, 12, 117–124. [Google Scholar] [CrossRef] [PubMed]
- Logan, R.A.; Wong, W.F.; Villaire, M.; Daus, G.; Parnell, T.A.; Willis, E.; Paasche-Orlow, M.K. Health Literacy: A Necessary Element for Achieving Health Equity. NAM Perspect. 2015, 5. [Google Scholar] [CrossRef]
- Nutbeam, D. Health promotion glossary. Health Promot. Int 1998, 13, 349–364. [Google Scholar] [CrossRef]
- Starmer, A.J.; Spector, N.D.; West, D.C.; Srivastava, R.; Sectish, T.C.; Landrigan, C.P.; Landrigan, C.P.; Spector, N.D.; Starmer, A.J.; Sectish, T.C.; et al. Integrating Research, Quality Improvement, and Medical Education for Better Handoffs and Safer Care: Disseminating, Adapting, and Implementing the I-PASS Program. Jt. Comm. J. Qual. Patient Saf. 2017, 43, 319–329. [Google Scholar] [CrossRef]
- Berkman, N.D.; Sheridan, S.L.; Donahue, K.E.; Halpern, D.J.; Crotty, K. Low health literacy and health outcomes: An updated systematic review. Ann. Intern. Med. 2011, 155, 97–107. [Google Scholar] [CrossRef]
- Patel, S.J.; Landrigan, C.P. Communication at Transitions of Care. Pediatr. Clin. N. Am. 2019, 66, 751–773. [Google Scholar] [CrossRef]
- Coleman, C.A.; Hudson, S.; Maine, L.L. Health literacy practices and educational competencies for health professionals: A consensus study. J. Health Commun. 2013, 18, 82–102. [Google Scholar] [CrossRef]
- Coleman, C.; Hudson, S.; Pederson, B. Prioritized Health Literacy and Clear Communication Practices for Health Care Professionals. HLRP Heal. Lit. Res. Pract. 2017, 1, e91–e99. [Google Scholar] [CrossRef] [PubMed]
- Liang, L.; Brach, C. Health Literacy Universal Precautions Are Still a Distant Dream: Analysis of U.S. Data on Health Literate Practices. HLRP Heal. Lit. Res. Pract. 2017, 1, e216–e230. [Google Scholar] [CrossRef] [PubMed]
- Brown, D.R.; Ludwig, R.; Buck, G.A.; Durham, D.; Shumard, T.; Graham, S.S. Health literacy: Universal precautions needed. J. Allied Health 2004, 33, 150–155. [Google Scholar] [PubMed]
- Kelly, P.A.; Haidet, P. Physician overestimation of patient literacy: A potential source of health care disparities. Patient Educ. Couns. 2007, 66, 119–122. [Google Scholar] [CrossRef]
- Raver, S.A.; Childress, D.C. Early education and intervention for children from birth to three. In Family-Centered Early Intervention; Raver, S.A., Childress, D.C., Eds.; Paul H. Brooke: Baltimore, MD, USA, 2015; pp. 2–30. [Google Scholar]
- Division for Early Childhood. DEC Recommended Practices in Early Intervention/Early Childhood Special Education. 2014. Available online: http://www.dec-sped.org/recommendedpractices (accessed on 29 January 2020).
- Leslie, C.; Smith, D.; Nicholas, P. Health literacy understanding of early intervention providers: A scoping review. J. Nurse Pract. 2019, 15, 640–648. [Google Scholar] [CrossRef]
- Osborne, R.H.; Batterham, R.W.; Elsworth, G.R.; Hawkins, M.; Buchbinder, R. The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ). BMC Public Health 2013, 13, 658. [Google Scholar] [CrossRef]
- American Educational Research Association; American Psychological Association; National Council on Measurement in Education. Standards for Educational and Psychological Testing; American Educational Research Association: Washington, DC, USA, 2014. [Google Scholar]
- Hawkins, M.; Gill, S.D.; Batterham, R.; Elsworth, G.R.; Osborne, R.H. The Health Literacy Questionnaire (HLQ) at the patient-clinician interface: A qualitative study of what patients and clinicians mean by their HLQ scores. BMC Health Serv. Res. 2017, 17, 309. [Google Scholar] [CrossRef]
- Hawkins, M.; Elsworth, G.R.; Osborne, R.H. Application of validity theory and methodology to patient-reported outcome measures (PROMs): Building an argument for validity. Qual. Life Res. 2018, 27, 1695–1710. [Google Scholar] [CrossRef]
- Hawkins, M.; Elsworth, G.R.; Osborne, R.H. Questionnaire validation practice: A protocol for a systematic descriptive literature review of health literacy assessments. BMJ Open 2019, 9, e030753. [Google Scholar] [CrossRef]
- Kane, M.T. Validation. In Educational Measurement, 4th ed.; Brennan, R.B., Ed.; American Council on Education/Praeger: Westport, CT, USA, 2006; pp. 17–64. [Google Scholar]
- Kane, M.T. An argument-based approach to validity. Psychol Bull 1992, 112, 527–535. [Google Scholar] [CrossRef]
- Zumbo, B.D.; Chan, E.K. Validity and Validation in Social, Behavioral, and Health Sciences; Springer International Publishing: Basel, Switzerland, 2014. [Google Scholar]
- Sawatzky, R.; Chan, E.K.H.; Zumbo, B.D.; Ahmed, S.; Bartlett, S.J.; Bingham, C.O.; Gardner, W.; Jutai, J.; Kuspinar, A.; Sajobi, T.; et al. Montreal Accord on Patient-Reported Outcomes (PROs) use series–Paper 7: Modern perspectives of measurement validation emphasize justification of inferences based on patient reported outcome scores. J. Clin. Epidemiol. 2017, 89, 154–159. [Google Scholar] [CrossRef] [PubMed]
- Zumbo, B.D.; Hubley, A.M. Understanding and Investigating Response Processes in Validation Research (Vol. 69). Social Indicators Research Series; Springer International Publishing: Cham, Switzerland, 2017. [Google Scholar]
- Padilla, J.L.; Benítez, I. Validity evidence based on response processes. Psicothema 2014, 26, 136–144. [Google Scholar] [PubMed]
- Castillo-Díaz, M.; Padilla, J.L. How cognitive interviewing can provide validity evidence of the response processes to scale items. Soc. Indic. Res. 2013, 114, 963–975. [Google Scholar] [CrossRef]
- Boeije, H.; Willis, G. The cognitive interviewing reporting framework (CIRF): Towards the harmonization of cognitive testing reports. Methodology 2013, 9, 87–95. [Google Scholar] [CrossRef]
- Willis, G.B. Cognitive Interviewing: A Tool for Improving Questionnaire Design; Sage Publications: Thousand Oaks, CA, USA, 2005. [Google Scholar]
- Buchbinder, R.; Batterham, R.; Elsworth, G.; Dionne, C.E.; Irvin, E.; Osborne, R.H. A validity-driven approach to the understanding of the personal and societal burden of low back pain: Development of a conceptual and measurement model. Arthritis Res. Ther. 2011, 13, R152. [Google Scholar] [CrossRef]
- Beauchamp, A.; Buchbinder, R.; Dodson, S.; Batterham, R.W.; Elsworth, G.R.; McPhee, C.; Sparkes, L.; Hawkins, M.; Osborne, R.H. Distribution of health literacy strengths and weaknesses across socio-demographic groups: A cross-sectional survey using the Health Literacy Questionnaire (HLQ). BMC Public Health 2015, 15, 678. [Google Scholar] [CrossRef]
- Elsworth, G.R.; Beauchamp, A.; Osborne, R.H. Measuring health literacy in community agencies: A Bayesian study of the factor structure and measurement invariance of the health literacy questionnaire (HLQ). BMC Health Serv. Res. 2016, 16, 1–14. [Google Scholar] [CrossRef]
- Kolarcik, P.; Cepova, E.; Madarasova Geckova, A.; Elsworth, G.R.; Batterham, R.W.; Osborne, R.H. Structural properties and psychometric improvements of the Health Literacy Questionnaire in a Slovak population. Int. J. Public Health 2017, 62, 591–604. [Google Scholar] [CrossRef]
- Maindal, H.T.; Kayser, L.; Norgaard, O.; Bo, A.; Elsworth, G.R.; Osborne, R.H. Cultural adaptation and validation of the Health Literacy Questionnaire (HLQ): Robust nine-dimension Danish language confirmatory factor model. Springerplus 2016, 5, 1232. [Google Scholar] [CrossRef]
- Nolte, S.; Osborne, R.H.; Dwinger, S.; Elsworth, G.R.; Conrad, M.L.; Rose, M.; Härter, M.; Dirmaier, J.; Zill, J.M. German translation, cultural adaptation, and validation of the Health Literacy Questionnaire (HLQ). PLoS ONE 2017, 12, e0172340. [Google Scholar] [CrossRef]
- Bo, A.; Friis, K.; Osborne, R.H.; Maindal, H.T. National indicators of health literacy: Ability to understand health information and to engage actively with healthcare providers - A population based survey among Danish adults. BMC Public Health 2014, 14, 1095. [Google Scholar] [CrossRef] [PubMed]
- Debussche, X.; Lenclume, V.; Balcou-Debussche, M.; Alakian, D.; Sokolowsky, C.; Ballet, D.; Elsworth, G.R.; Osborne, R.H.; Huiart, L. Characterisation of health literacy strengths and weaknesses among people at metabolic and cardiovascular risk: Validity testing of the Health Literacy Questionnaire. SAGE Open Med. 2018, 6, 2050312118801250. [Google Scholar] [CrossRef] [PubMed]
- Morris, R.L.; Soh, S.E.; Hill, K.D.; Buchbinder, R.; Lowthian, J.A.; Redfern, J.; Etherton-Beer, C.D.; Hill, A.M.; Osborne, R.H.; Arendts, G.; et al. Measurement properties of the Health Literacy Questionnaire (HLQ) among older adults who present to the emergency department after a fall: A Rasch analysis. BMC Health Serv. Res. 2017, 17, 605. [Google Scholar] [CrossRef] [PubMed]
- Beatty, P.C.; Willis, G.B. Research synthesis: The practice of cognitive interviewing. Public Opin. Q. 2007, 71, 287–311. [Google Scholar] [CrossRef]
- Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [Google Scholar] [CrossRef]
- Australian Government Department of Health. Medicare. Available online: https://www.health.gov.au/health-topics/medicare (accessed on 29 January 2020).
- Merriam-Webster Dictionary. Available online: https://www.merriam-webster.com/dictionary/entitled (accessed on 21 January 2020).
- Jessup, R.L.; Osborne, R.H.; Beauchamp, A.; Bourne, A.; Buchbinder, R. Differences in health literacy profiles of patients admitted to a public and a private hospital in Melbourne, Australia. BMC Health Serv. Res. 2018, 18, 134. [Google Scholar] [CrossRef]
- Mullan, J.; Burns, P.; Weston, K.; McLennan, P.; Rich, W.; Crowther, S.; Mansfield, K.; Dixon, R.; Moselen, E.; Osborne, R. Health Literacy amongst Health Professional University Students: A Study Using the Health Literacy Questionnaire. Educ. Sci. 2017, 7, 54. [Google Scholar] [CrossRef]
- Elsborg, L.; Krossdal, F.; Kayser, L. Health literacy among Danish university students enrolled in health-related study programmes. Scand. J. Public Health 2017, 45, 831–838. [Google Scholar] [CrossRef]
- Rheault, H.; Coyer, F.; Jones, L.; Bonner, A. Health literacy in Indigenous people with chronic disease living in remote Australia. BMC Health Serv. Res. 2019, 19, 1–10. [Google Scholar]
- Aaby, A.; Beauchamp, A.; O’Hara, J.; Maindal, H.T. Large diversity in Danish health literacy profiles: Perspectives for care of long-term illness and multimorbidity. Eur. J. Public Health 2019, 30, 75–80. [Google Scholar] [CrossRef]
- Batterham, R.W.; Buchbinder, R.; Beauchamp, A.; Dodson, S.; Elsworth, G.R.; Osborne, R.H. The OPtimising HEalth LIterAcy (Ophelia) process: Study protocol for using health literacy profiling and community engagement to create and implement health reform. BMC Public Health 2014, 14, 694. [Google Scholar] [CrossRef] [PubMed]
- Beauchamp, A.; Batterham, R.W.; Dodson, S.; Astbury, B.; Elsworth, G.R.; McPhee, C.; Jacobson, J.; Buchbinder, R.; Osborne, R.H. Systematic development and implementation of interventions to OPtimise Health Literacy and Access (Ophelia). BMC Public Health 2017, 17, 1–18. [Google Scholar] [CrossRef] [PubMed]
- Goeman, D.; Conway, S.; Norman, R.; Morley, J.; Weerasuriya, R.; Osborne, R.H.; Beauchamp, A. Optimising Health Literacy and Access of Service Provision to Community Dwelling Older People with Diabetes Receiving Home Nursing Support. J. Diabetes Res. 2016, 2016, 2483263. [Google Scholar] [CrossRef] [PubMed]
- Bakker, M.M.; Putrik, P.; Aaby, A.S.E.; Debussche, X.; Morrissey, J.; Borge, C.R.; Do, D.N.; Kolarčik, P.; Batterham, R.; Osborne, R.H.; et al. Acting together—WHO National Health Literacy Demonstration Projects (NHLDPs) address health literacy needs in the European Region. Public Heal. Panor. 2019, 5, 233–243. [Google Scholar]
|The Nine Scales of the Health Literacy Questionnaire|
|Participant ID||EI Job Title (Years of EI Experience)||Education Level||Age in Years||Race||Chronic Health Condition(s)?||Self-Reported Health|
|P#03||TL (3)||Master’s||27||Hispanic||Yes||Very good|
|P#06||DS (4)||Bachelor’s||29||White||No||Very good|
|P#07||Manager (11)||Bachelor’s||42||White||No||Very good|
|Matching: Narrative Data and Item Score||Concordance: Narrative Data and HLQ Item Intent|
|HLQ Scale||M N (%)||NM N (%)||U N (%)||C N (%)||D N (%)||U N (%)|
|Scale 1||16 (84%)||2 (11%)||1 (5%)||19 (95%)||-||1 (5%)|
|Scale 2||19 (95%)||1 (5%)||-||18 (90%)||1 (5%)||1 (5%)|
|Scale 3||24 (96%)||1 (4%)||-||24 (96%)||1 (4%)||-|
|Scale 4||24 (96%)||1 (4%)||-||23 (92%)||1 (4%)||1 (4%)|
|Scale 5||24 (96%)||1 (4%)||-||22 (88%)||3 (12%)||-|
|Scale 6||23 (92%)||1 (4%)||1 (4%)||24 (96%)||-||1 (4%)|
|Scale 7||28 (93%)||1 (3%)||1 (3%)||27 (90%)||1 (3%)||2 (7%)|
|Scale 8||23 (92%)||2 (8%)||-||23 (92%)||2 (8%)||-|
|Scale 9||25 (100%)||-||-||24 (96%)||1 (4%)||-|
|206 (94%)||10 (5%)||3 (1%)||204 (93%)||10 (5%)||6 (3%)|
|Theme||EI Provider Quotes|
|Theme 1. USA culture/health system|
|Sub-theme 1.1. Word “entitled”||P#09 (Usually difficult) “I feel like that’s a usually difficult one, because I feel like it’s never really clear to anyone what you’re entitled to. So it’s kind of knowing. So unless you are like working in a hospital and know what people are entitled to because you have to [for your job as a healthcare provider] because you have to empower them and like that’s part of your job is to do that. But it’s hard to tell what you’re actually truly entitled to.”|
|Sub-theme 1.2. Difference in USA and Australian insurance systems||P#01 “Sometimes difficult only because of insurance. I was just trying to figure out what’s covered by insurance, and insurance is just confusing to me.”|
P#07 “I mean, it’s sometimes, depending on what insurance I’ve had at what point in time, sometimes it’s meant a referral from the primary care provider or you know them connecting me to a different department in order to have the care, to see they’re the right kind of provider that you need and that has always gone smoothly.”
P#09 I: “How would you start to figure out if you’re entitled to that? P: I mean I feel like I would first go to my doctor and be like ‘is this something that you would help me with or do I have to go somewhere else?’ And if that’s not working, then the insurance company would kind of be the next place to go, but I feel like that’s hard as well.”
|Theme 2. The Healthcare Provider Perspective|
|Sub-theme 2.1. Answering Based on Provider Perspective||P#04 “I don’t know if I’m answering this right, but like I feel like because, being a clinician, I know so many other clinicians [at work] that when I stumble on new information or I’m trying to learn something in terms of maybe like speech or like other stuff that I wouldn’t be an expert in, I typically ask other health care providers their own experience, own like understanding, and own information base”.|
P#07 “I mean I think of this in relationship to EI [early intervention job]. That’s such a different question than it is in relationship to my own health”.
P#09 “I mean I definitely am much better at children and helping families (at work) find the right health care that they need for their children. But like as an adult, I feel like that’s not an area that I’m well versed in in this part of the state.”
P#10 “I agree. Probably agree more in my professional life with that than my personal life. If I go to appointments with a family in early intervention, then I’ll often help them ask those questions that maybe they don’t know to ask. But oftentimes, like for my own self, that can be challenging.”
|Sub-theme 2.2. Only thinking of doctors/healthcare providers for support||P#04 “I know what questions to ask and I feel like, as a health care provider, I understand a lot of like, I don’t know, like what I need to gather, what information to gather, to find the supports that I need.”|
|Theme 3. Respondents with no health problems to manage||P#04 “If I need help? For me, I would say probably not health, because I don’t have any health issues, but if I need help, I would probably say right now like financially.”|
P#07 “I think the only reason I’m saying ‘agree’ instead of ‘strongly agree’ is more just you know I can see I don’t really have any really pressing health issues, and maybe if there were, then that might be something you’d want more information on. But as far as what I have now, I feel like I can effectively manage.”
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).