Learning to make informed health choices: Protocol for a pilot study in schools in Barcelona

Informed Health Choices project

The main objective of the Informed Health Choices (IHC) project is to teach people to assess treatment claims and make informed health choices.

The IHC project has a focus on enabling people to learn these skills at a young age and began their first work in developing learning resources for primary school children (10 to 12-year-olds) from low-income countries (Uganda)⁸. There are several reasons the IHC project started with primary school children: 1) children can learn about fair comparisons (controlled research) and critical appraisal (in some countries, teaching these basic capabilities is already part of the curriculum); 2) primary school interventions can reach a large population group, before many of them leave school; 3) compared to adults, children have more time to learn and show less resistance to change with regard to their beliefs, attitudes or behaviours; 4) teaching children to think critically improves their academic performance; and 5) learning how to think critically about claims about treatment effects can help them, once they become adults, to make decisions about their health and to contribute, as citizens or as health decision-makers, to develop and implement health policies⁹. In addition, the IHC project focused on the child population of low-income countries because making informed health choices can contribute to a more efficient use of resources in contexts with higher social and economic inequality⁸.

The IHC Working Group has developed several resources to help people understand the differences between trustworthy and untrustworthy health claims, and how to use reliable information to make informed health choices⁸. The main resources are: 1) key concepts, 2) learning resources, and 3) a tool to evaluate the ability to assess treatment claims.

IHC key concepts. Using the principles of a spiral curriculum, the IHC Working Group has compiled a list of concepts that individuals need to understand and apply when assessing claims about treatment effects and making health choices^3,10.

The list of concepts is reviewed and updated periodically. The list currently includes 44 concepts divided into three capability groups: 1) identify when the treatment claim has an untrustworthy basis, 2) recognise when evidence from comparisons of treatments is trustworthy and when it is not, and 3) make well-informed choices about treatments. Table 1 shows the list of key concepts¹¹.

IHC learning resources. Using a human-centred design approach^12–14, the IHC Working Group has produced various learning resources (IHC resources) to teach children and their families to understand and apply some of the key concepts⁸.

Table 1. List of key concepts from the Informed Health Choices project¹¹.

1. Beware of treatment claims like these: We hear claims about the effects of treatments all the time. Many of these are not trustworthy. When you hear someone use one of these reasons to support a claim about the effects of a treatment, you should beware and ask where the evidence is.
1.1. Beware of claims that are too good to be true	1.    “100% safe!”*
	2.    “100% effective!”
	3.    “100% certain!”
1.2. Beware of claims based on faulty logic	4.    “Treatment needed!”
	5.    “It works like this!”
	6.    “Associated with!”
	7.    “Real world data!”
	8.    “No comparison needed!”*
	9.    “A study shows!”*
	10.    “Old is better!”*
	11.    “New is better!”*
	12.    “More is better!”
	13.    “Early is better!”
	14.    “Personalised medicine!”
1.3. Beware of claims based on trust alone	15.    “As advertised!”*
	16.    “It worked for me!”*
	17.    “Recommended by experts!”*
	18.    “Peer reviewed!”
2. Check the evidence from treatment comparisons A treatment has to be compared to something else to know what would happen without the treatment. For treatment comparisons to be FAIR, the only important difference between comparison groups should be the treatments they receive. Unfair treatment comparisons and unsystematic summaries of treatment comparisons can be misleading. The way that treatment effects are described can also be misleading.
2.1. Don’t be misled by unfair comparisons!	19.    Dissimilar comparison groups*
	20.    Indirect comparisons
	21.    Dissimilar attention and care
	22.    Dissimilar expectations or behaviours*
	23.    Dissimilar assessment of outcomes
	24.    Unreliable assessment of outcomes
	25.    Lots of people not followed-up
	26.    Outcomes counted in the wrong group
2.2. Don’t be misled by unreliable summaries of treatment comparisons!	27.    Unsystematic summaries
	28.    Selective reporting
	29.    Unfounded assumptions
2.3. Don’t be misled by how treatment effects are described!	30.    Just words
	31.    Relative effects
	32.    Average effects
	33.    Few people or events*
	34.    Subgroup analyses
	35.    Statistically significant
	36.    No confidence interval
	37.    No evidence
3. Make well-informed treatment choices Deciding what to do requires judgements about the relevance of the evidence, how important the good and bad outcomes are to you, and how sure you can be about the treatment effects.
3.1. What is the problem and what are the options?	38.    What is your health problem and what are your options?
3.2. Is the evidence relevant?	39.    What outcomes matter to you?
	40.    Are the people (or animals) very different from you?
	41.    Are the treatments different from those available to you?
	42.    Are the circumstances different from yours?
3.3. Do the advantages outweigh the disadvantages?	43.    Do the advantages outweigh the disadvantages for you?*
	44.    How sure are you about the treatment effects?

* The 12 concepts included in the learning resources of the IHC project for primary school children. The IHC Key Concepts' explanations are available from the “That’s a Claim” website. This table has been reproduced with permission from Oxman et al. (Box 3)¹¹.

The following resources were produced for primary school children (10 to 12-year-olds): a book (that includes and explains 12 key concepts), an exercise book, a teachers’ guide, some activity cards, a poster and a song (Figure 1)^8,15. The book tells a story, narrated as a comic, about a brother and a sister, John and Julie, who know two teachers and health researchers, professor Compare and professor Fair. The professors teach John and Julie: 1) what questions they should ask when someone says something about a treatment; 2) what questions health researchers ask to find out more about treatment effects; and 3) what questions they should ask when deciding to use a treatment or not¹⁵.

Figure 1. Learning resources from the Informed Health Choices project for primary school children translated into Spanish.

The effect of the resources was assessed in a cluster randomised trial conducted in Uganda⁹. In the trial, 120 schools were assigned randomly to receive the intervention with the resources (60 schools, 76 teachers and 6,383 children) or not receive it (60 schools, 67 teachers and 4,430 children)⁹. The study showed that the children who used the resources improved their ability to assess treatment claims in comparison with the group without resources (69% of the children who use the learning resources got a passing score vs. 27% of children in the control group)⁹. A follow-up study one year later showed that children retained this knowledge and, in fact, the proportion of children with a passing score increased from 69% to 80%¹⁶.

Additionally, the IHC project team developed and evaluated a podcast with several episodes for parents (that introduce and explain nine key concepts)^8,14,17,18.

Tool to evaluate the ability to assess treatment claims. The IHC Working Group has created a database with questions to assess people’s understanding and ability to apply the key concepts; the CLAIM Evaluation Tools¹⁹. Each question is based on a scenario that involves a claim about a treatment. There are two types of questions: 1) individual multiple-choice questions and 2) several true or false statements¹⁹.

This tool is a flexible resource, since people may design a questionnaire according to the key concepts that they wish to evaluate, selecting the questions that are most relevant for their objectives¹⁹. For example, teachers can design questionnaires to assess children, and researchers can design questionnaires to assess interventions or to describe a population’s ability to make informed health choices¹⁹. All the questions have been designed to be answered by children over 10 years of age as well as by adults¹⁹. The CLAIM Evaluation Tools can be found on the Testing Treatments international website.

In the previously cited trials that assessed the effect of the IHC resources, the researchers used questions from this database^9,18.

Contextualization of the Informed Health Choices project

The IHC resources have proven to be effective in the Ugandan trial, but it is still unknown whether they may be useful in other contexts²⁰. Different working groups from more than 25 countries are adapting, or planning adaptation, of the IHC resources to their context²¹.

The IHC Working Group has proposed the following contextualization activities to explore how these resources can be used in a context different from the one that they were originally designed for: 1) context analysis, 2) translation of the resources, 3) pilot study, 3) content adaptation, 5) resource production, and 6) validation of the tool to assess treatment claims²⁰.

Currently, we do not have any specific learning resources to teach primary school children to think critically about their health in the context of Barcelona (Spain). The working group from the Iberoamerican Cochrane Centre (CCIb) - Sant Pau Biomedical Research Institute (IIB Sant Pau) has translated the IHC resources into Spanish and included them on the IHC website. The next step is to explore how to use and, if needed, how to adapt the IHC resources in this context.

Primary objective

    Explore the students’ and teachers’ experience when using the learning resources of the IHC project in the context of Barcelona (Spain).

Secondary objectives

    Explore potential changes to the IHC resources to adapt them to this context.

    Explore the feasibility of implementing the IHC resources in this context.

    Evaluate the ability of the students to assess treatment claims and make informed health choices after using the IHC resources in this context.

Participants

Establishment of the IHC-Barcelona Working Group. We will establish a “coordination group” to lead and coordinate the pilot study and to ensure it is completed according to the established work plan. We will establish a multidisciplinary “advisory group” (researchers, teachers, paediatricians, student representatives, family representatives, education and health stakeholders, and translators) to review and advise on the development of the different steps of the pilot study.

We will aim for profile representativeness of the IHC-Barcelona Working Group members. We will identify researchers from CIBER of Epidemiology and Public Health (CIBERESP) and expert colleagues; teachers, student representatives, and family representatives from selected schools; paediatricians from Asociación Española de Pediatría de Atención Primaria (AEPap); education and health stakeholders from Catalan Education and Health Departments; and translators who participated in the IHC resources translation into Spanish. Potential members will be contact and invite to participate by email. We will request and register the conflicts of interest of all the members of the IHC-Barcelona Working Group.

Selection of the schools. We will select a convenience sample of three schools in Barcelona. We will use the following eligibility criteria: 1) schools included in the school directory from the Department of Education from the Government of Catalonia (2018–2019); 2) schools that have participated in a health promotion programme (2016–2017)²³; and 3) schools that take part in the initiative Escola Nova 21. We will also take into consideration whether the schools include students that are representative of the neighbourhood, if they are in different neighbourhoods of the city, and their type of funding (two public schools and one private or charter school).

Selection of the students and teachers. We will select 4^th and 5^th-year primary school students (9 to 11-year-olds) in all the lines from the selected schools (in this context, the number of lines means the number of student groups per academic level). We expect to include a convenience sample of approximately 150 students (25 students per class * two lines per school * three schools). We will request written informed consent from the families (Extended data 1 and 2)^24,25.

We will select one teacher from every 4^th or 5^th year class in the selected schools. We expect to include six teachers (one teacher per class * two lines per school * three schools). The profile of the participatory teachers, as well as the subject where the lessons will be included (for example, in Science, Ethics or even Spanish) will depend on the education plan and the availability of the resources in each school. We will request informed consent from the teachers (Extended data 1 and 3)^24,26.

Intervention in the schools

The intervention in the schools will include: 1) a workshop with the teachers, and 2) lessons to the students. Each of the activities is described below:

1.       Workshop with the teachers

2.       Lessons to the students

Data collection

The data collection will include: 1) initial assessment of the resources by the teachers, 2) non-participatory observations during the lessons, 3) semi-structured interviews with the students after a lesson, 4) assessment of the lessons by the teachers, 5) treatment claim assessment by the students (CLAIM questionnaire test), and 6) final assessment of the resources by the teachers. Each of the activities is described below:

1.       Initial assessment of the resources by the teachers

2.       Non-participatory observations during the lessons

3.       Semi-structured interviews with the students after a lesson

4.       Assessment of the lessons by the teachers

5.       Treatment claim assessment by the students (CLAIM questionnaire test)

6.       Final assessment of the resources by the teachers

Data analysis

Quantitative analysis. We will perform a descriptive analysis of the qualitative variables (absolute and relative frequencies) and the quantitative variables (mean and standard deviation or median and range).

With regard to the treatment claim assessment by the students, we will show the mean score and the standard deviation, the proportion of the students with a passing score (basic knowledge of the concepts and how to apply them, 13 points or more over 24), and the proportion of the students with a high score (clear knowledge of the concepts and how to apply them, 20 points or more over 24)³³.

Qualitative analysis. We will register in an excel sheet the feedback from: 1) the initial assessments by the teachers, 2) the non-participatory observations, 3) the semi-structured interviews with the students, 4) the assessment of the lessons by the teachers, and 5) the final assessments by the teachers.

We will perform a thematic analysis based on the categories previously used in the IHC project (seriousness, user experience, facilitators and barriers, and potential changes) (Table 4)^22,34. One researcher will identify, codify, and summarise the feedback using these categories and search for emerging categories; another researcher will check the codification. They will discuss and review the definitions and limits of each category. Finally, using the summarised data, they will explore the nature of the phenomena (understandability, desirability, suitability, usefulness, facilitators and barriers) and the possible explanations of the results.

Formulation of the recommendations

The IHC-Barcelona Working Group will discuss most relevant results and ideas to improve users’ experience when using the IHC resources in the context of Barcelona (Spain). They will consensus the potential changes of IHC resources (Table 4). Finally, they will suggested and agreed recommendations - both practice and research purposes - on how to use, how to adapt (if needed), and how to implement the IHC resources to this context.

Dissemination of the results

The dissemination activities of the pilot study results will include: 1) publication in a peer-reviewed journal, 2) publication in several Internet resources (for example, related web pages, electronic bulletins and social media), and 3) introduction to the different users of interest (researchers, teachers, paediatricians, student representatives, family representatives, education and health stakeholders, and translators) in conferences, workshops and meetings. The implementation activities will include: 1) offering support to the schools that have participated in the pilot study and that are interested in including the IHC resources in the following school years, 2) giving support to other schools that are interested in including the IHC resources in their education plan.

User participation

Representatives from all the different areas of interest (researchers, teachers, paediatricians, student representatives, family representatives, education and health stakeholders, and translators) will be invited to be members of the IHC-Barcelona Working Group.

Ethical considerations

The study protocol has obtained an approval exemption (does not include patients, biological specimens or clinical data) from the Ethics Committee of the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain). The participants will be informed about the study, their written informed consent will be requested, and the data will be anonymised.

Study status

Figure 2 shows the work plan. Currently, we have started the intervention in schools with the teachers’ workshop.

Figure 2. Pilot study work plan.

Our study in the context of current knowledge

Introducing the IHC resources in schools can be considered as a health promotion and education intervention³⁵. According to the World Health Organization (WHO), the concept of health promotion comprises “the process of enabling people to increase control over, and to improve their health”³⁶. Additionally, health education comprises “consciously constructed opportunities for learning involving some form of communication designed to improve health literacy, including improving knowledge, and developing life skills which are conducive to individual and community health”³⁶. Health education and promotion interventions in schools have proven to be beneficial for the health of the population^37,38.

There are several definitions of critical thinking, as well as several strategies to teach how to think critically^39,40. In 1990, a Delphi panel of experts defined this ability as a “purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based”⁴¹. Therefore, promoting critical thinking at schools can be not only useful in the health area¹⁸, but also in other curricular areas (e.g., Mathematics, Science, Literacy)⁴².

The IHC project offers several learning resources that were created accurately and explicitly, and have been assessed in a cluster randomised trial^16,18. Thus far, there are few studies that assess the effect of the learning resources when acquiring competences⁴³. Moreover, the available studies show that the evaluated textbooks provide little learning support^44–46. We must start demanding the same standards for evaluating educational interventions that are used for evaluating health interventions⁴⁷.

Study strengths and limitations

Our proposal has several strengths. Firstly, before the study, we have translated the IHC resources into Spanish. Different users of interest (translator, researchers, students, teachers, and medical doctors) participated in the translation process and fit the text of the IHC resources to this context. Secondly, we have expanded the profile of the users of interest (researchers, teachers, paediatricians, student representatives, family representatives, education and health stakeholders, and translators) to establish the study's multidisciplinary working group. Thirdly and lastly, we will pilot an intervention in the schools that has already been shown to be effective in a cluster randomised trial in Uganda, where over 100 schools participated (100 teachers and 10,000 children).

Our proposal also has some limitations. The main limitation is using a convenience sample (small and geographically limited sample). However, an exploratory approach with few schools has some advantages: 1) assessing the feasibility of introducing the IHC resources into this education system, 2) familiarising ourselves with the intervention, and 3) adapting the methods to apply them to broader future studies in this context. It is also worth noting that we will not be assessing the impact of the IHC resources in this study; due to this, we will not include a control group and we will not have a questionnaire validated to assess treatment claims for students.

Implications for practice and research

The next contextualization activities will be: 1) content adaptation - if needed, 2) context analysis, and 3) validation of the CLAIM questionnaire test into Spanish for use in this context. Finally, a cluster randomised trial with the adapted IHC resources could be conducted to evaluate their effect in the students’ ability to assess treatment claims in this context.

Underlying data

No data are associated with this article.

Extended data

Figshare: IHC@BCN_01Information.pdf. https://doi.org/10.6084/m9.figshare.10272173.v1²⁴

Figshare: IHC@BCN_02InformedConsentFamilies.pdf https://doi.org/10.6084/m9.figshare.10272215.v1²⁵

Figshare: IHC@BCN_03InformedConsentTeachers.pdf https://doi.org/10.6084/m9.figshare.10272242.v1²⁶

Figshare: IHC@BCN_04Workshop.pdf https://doi.org/10.6084/m9.figshare.10272275.v1²⁷

Figshare: IHC@BCN_05InitialAssessment https://doi.org/10.6084/m9.figshare.10272278.v1²⁸

Figshare: IHC@BCN_06NonParticipatoryObservations https://doi.org/10.6084/m9.figshare.10272281.v1²⁹

Figshare: IHC@BCN_07SemiStructuredInterviews https://doi.org/10.6084/m9.figshare.10272287.v1³⁰

Figshare: IHC@BCN_08AssessmentLessons https://doi.org/10.6084/m9.figshare.10272305.v1³¹

Figshare: IHC@BCN_09FinalAssessment https://doi.org/10.6084/m9.figshare.10272308.v1³²

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).