Design and Development of an Instrument on Knowledge of Food Safety, Practices, and Risk Perception Addressed to Children and Adolescents from Low-Income Families

Abstract

In the fight against foodborne diseases, expanding access to information for different groups is needed. In this aspect, it is crucial to evaluate the target audience’s particularities. This study constructed and validated an instrument containing three questionnaires to identify the level of knowledge, practices, and risk perception of food safety by low-income students between 11 and 14 years old. The following steps were used: systematic search of the databases; conducting and analyzing focus groups; questionnaires development; and questionnaires analysis. After two judges’ rounds, the final version was reached with 11 knowledge items, 11 practice items, and five risk perception items. The content validation index values were higher than 0.80. The adopted methodology considered the students’ understanding and perceptions, as well the appropriate language to be used. Besides, it allowed the development of questionnaires that directly and straightforwardly covers the rules set by the World Health Organization for foodborne disease control called Five Keys to Safer Food (keep clean; separate raw and cooked; cook thoroughly; keep food at safe temperatures; and use safe water and raw materials). Its use can result in a diagnosis for elaborating educational proposals and other actions against foodborne illness in the most vulnerable population.

1. Introduction

Even though the prevalence of outbreaks of foodborne disease (FBD) is underreported, it results in socioeconomic losses worldwide. In 2010, there were 600 million cases, with 420,000 deaths, caused mainly by agents of diarrheal disease. FBD represents a challenge for food security, human health, economic prosperity, agriculture, market access, tourism, and sustainable development.

Another important aspect is an imbalance in the disease’s distribution with a higher incidence in vulnerable populations, either economically, due to comorbidities, or by the age group (children and the elderly). In addition to being more susceptible to disease complications, such as dehydration, these populations may be more exposed to disease-causing agents, mainly biological ones, viruses, parasites, and bacteria). Fungi and their associated toxins have also been characterized as a significant safety problem worldwide [1]. This fact may result from a lack of knowledge and an environment conducive to such agents’ proliferation, such as inadequate conditions for handling and preserving food in the domestic environment [2].

A study carried out by Hadler et al. [3] identified this situation of greater vulnerability. Children and the elderly in poverty in the United States showed a higher incidence of Salmonellosis. The study highlighted the fact that 91.1% (n = 48.111) of the illnesses were acquired domestically. European Union data corroborate with the importance of domestic kitchens in the epidemiological scenario of FBD [4]. In Brazil, of the total number of FBD outbreaks reported between 2007 and 2017, 38.3% (n = 2922) of the cases occurred in homes, considered the places with the highest occurrence of FBD [5].

Responsibility for food safety must be taken together. Government agencies must fulfill their role in educating, informing, regulating, supervising, and ensuring fair and reliable business relationships. Besides complying with regulations and protecting food from contamination, producers from different sectors of the economy must provide consumers with clear and accurate information on handling food. Finally, consumers must recognize their role, following the relevant instructions, and adopting appropriate food hygiene measures [6,7].

Nevertheless, the high incidence of FBD in the home environment is possibly related to failures in adopting safe procedures during food handling. Individuals who handle food have a great responsibility in these cases, and they are considered the main ones responsible for foodborne outbreaks. Numerous studies have been developed to explore the food handler’s involvement in controlling FBD [8,9,10,11,12,13,14]. These studies aimed at understanding these individuals’ knowledge, practices, attitudes, behaviors, risk perception, and optimistic bias to develop diagnostic strategies that subsidize effective actions to face FBD. For this purpose, the studies use instruments developed for the target population according to the research proposal, such as specific questionnaires. They usually approach an adult audience, composed of food handlers from food services or consumers from these establishments.

Understanding such factors is essential since research related to consumers in general and food handlers demonstrated that unsafe handling practices are frequent, despite happening through an acceptable level of knowledge [8,9,15,16,17,18,19,20]. There is a need to cover other knowledge areas and age groups [8]. Children and adolescents are important targets for food safety training actions. They are at a stage where their cognitive structures reach their highest development level [21], considered crucial for developing knowledge, skills, and habits, including food safety. Therefore, with early intervention in young people’s education, it is possible to influence future behaviors and habits before they are formed, benefiting the student himself, his family, and people who will be under their responsibility in the future. A solid foundation for healthy behaviors is then established [22,23,24].

Other aspects reinforce the importance of considering them, such as the great potential as agents of transformation [23,24,25], evidence of being ready to fulfill in the preparation of meals [17,22,26,27,28,29] and finally, a tendency to feel more confident in their perceptions of personal risk [24].

Despite concrete evidence about this group’s important role in the area, the developed studies are more dedicated to adult consumers and their food handling practices. The fact reveals the need for more significant investments in studies and actions with this audience [28,30].

Studies directed to understand the difference in the socioeconomic status of risk factors for FBD are essential. Mainly considering the large proportion of FBD originating from the domestic environment [4,5,31,32] and the greatest vulnerability of populations in poverty situations. In this way, it will be possible to strengthen the individual role, including economically less affluent classes, as an essential link in the food supply chain [2,3].

When considering socioeconomic aspects, the studies point out significant factors that impact knowledge and the adoption of good food safety practices, such as level of education, place of residence [33], and location of the school (rural or urban), the latter when it comes to children and adolescents [34]. Other factors, such as low income and low awareness, contribute to a lesser willingness to adhere to an environment compatible with food safety in developing countries. In these countries, the difficulty of access to drinking water, safe cooling, fuel cost for cooking or adequate reheating of food, high environmental temperatures, inadequate sanitation, and low education levels discourages adopting appropriate practices [35,36,37].

Therefore, the connection between food safety education, school subjects, and the home environment becomes essential [38]. However, it is observed that studies with individuals in the school stage [8,9,10,11,12,13,14] present heterogeneity of methodologies, often adapted from other audiences, and not all of them present evidence of validation. They do not focus on economically vulnerable populations, and there may be bias not being applicable in adverse realities. Pawlowski [39] emphasizes that an instrument’s development must necessarily include the target audience to formulate health programs and policies based on reality. In this case, challenges inherent to low-income individuals need to be considered to ensure that their specificities are addressed [39].

Interventions for younger consumers and investment in consumer education can reduce FBD, and savings are expected of up to ten dollars for every dollar invested [40]. In this sense, this research aims to construct and validate a structured questionnaire to assess knowledge, practices, and risk perception related to food safety in low-income students aged 11 to 14 years. The results serve as a starting point to design educational actions that aim to improve the population’s food safety in which food challenges are faced daily.

2. Materials and Methods

This exploratory research was conducted in the Federal District (FD)—Brazil and it was approved by the Ethics Committee of the College of Health Sciences of the University of Brasilia—CEP/FS UnB (CAAE nº 02033218.0.0000.0030). To participate in the study, three schools were selected, by convenience criteria, within a sample of the Project “Healthy eating and the school food production chain: what happens in public schools in the Federal District (FD)”. The schools selected classes according to the schedule availability. The students’ participation took place by presenting the terms of consent and free and informed consent.

In the form of three questionnaires, the instrument was developed for students aged 11 to 14 years. The environment of public schools was chosen to meet the criterion of low income since, according to the Programme for International Student Assessment—2015 (PISA) [41], this target audience has the lowest ESCS (Index of Economic, Social, and Cultural Status) in Brazil.

A quality instrument’s development requires well-defined steps and rigorous procedures that guarantee reliable indicators [39,42]. The methodology chosen for the study recommends a process with the inclusion of specific tasks and methods carried out in a determined time sequence. These steps indicate the instrument’s psychometric strength and the guarantee of the constancy of its content with the constructs.

Thus, the development of the instrument was based on the model described by Pasquali [43]. The theoretical procedures respected the following steps: (i) systematic search in the databases; (ii) conducting and analyzing focus groups; (iii) questionnaires development; and (iv) questionnaires analysis (Figure 1).

2.1. Systematic Search in Databases

In addition to the database registered in Figure 1, the following documents were used to design the preliminary version of the questionnaire: Brazilian food safety resolutions based on texts from Codex Alimentarius [44]—RDC 216 and IN 16 [45,46]; Base Nacional Comum Curricular (BNCC)- Common Base National Curriculum [47], Protection Motivation Theory [48], Five Keys to Safer Food [49], and instruments developed by Da Cunha, Stedefeldt and De Rosso [50], De Andrade et al. [8], and Haapala and Probart [17].

2.2. Focus Groups (FG) Development and Analysis

Theoretical procedures, the focus of this study aims to obtain validity guaranteed by semantic and construct analyses. Thus, it was necessary to define the instrument’s properties, using the focus group methodology [51]. The method offers the possibility of examining students’ understanding and perceptions about food safety and verifying the appropriate language to address the target audience. It considers the view of different subjects and social contexts for less accessible information, which in many cases can only be raised through qualitative techniques [18,51,52].

Structured with an average of nine participants per group, 40 min in length, and heterogeneous concerning sex, the FGs were held in schools in the urban and rural areas of the public-school system in FD, which serves the low-income population [41]. The focal group conduction was carried out by a moderator with two transcribers’ participation, following a pre-established script of 21 questions contemplating the Five Keys to Safer Food (1—keep clean; 2—separate raw and cooked; 3—cook thoroughly; 4—keep food at safe temperatures; and 5—use safe water and raw materials) [49].

The focus groups were transcribed and analyzed with the content analysis technique and by the IraMuTeQ Software (Interface de R pour les Analyses Multidimensionnelles de Textes et de Questionnaires) 0.7 alpha [53,54], aiding in the questionnaires development. The software’s used followed the instructions in the manual provided by Camargo and Justo [55], and Descending Hierarchical Classification (DHC) analysis was performed.

2.3. Questionnaires Development

In preparing the items, the importance of assessing the knowledge, practices, and risk perception was considered. The transmission of the knowledge with low application in the practical context, associated with the low perception of risk, possibly results in unsafe practices [56]. From this premise, the dimensions assessed were defined.

Determining knowledge and food safety practices among consumers, in general, are of scientific interest, as exemplified by a study by Meysenburg et al. [18]. The study shows that unsafe handling practices are frequent, although they happen through an acceptable level of knowledge [18]. The repetition of these practices points to a significant gap that can be filled by developing educational actions capable of altering the handler’s risk perception [8].

Risk perception plays a role in risk management and control in anticipating crises and supporting people in prevention strategies [57]. Another vital phenomenon included and considered in the risk perception was the optimistic bias [58]. The concept of incorporated optimistic bias is represented by the belief that individuals, who in most conditions, visualize their chances of experiencing health and safety problems as inferior to those of their peers (co-workers, friends, and family) [8,59,60].

Therefore, investigating the relationship between practice and knowledge, permeated by risk perception, will contribute to the elucidation of educational strategies aimed at an age group with little knowledge in this field [28,30].

Three possible responses were used to measure the “knowledge” construct, “yes”, “no”, and “I do not know”. One point was assigned for each correct answer, and zero points were assigned to the wrong answer or “I do not know” [16]. Concerning the “practices” construct, a three-point frequency scale was used—“never”, “sometimes”, and “always”.

As for the “risk perception” construct, the first version’s Likert scale was based on a study developed by Swaney-Stueve et al. [61]. The study proposes a new model of pictorial scale—the emoji scale. Thus, a five-point scale was associated with emojis, ranging from extremely low to extremely high.

2.4. Questionnaires Analysis

2.4.1. Semantic Analysis

The validity procedures started with the semantic analysis of the first version of the instrument, containing three questionnaires, separately addressing knowledge, practices, and risk perception. This stage was developed with the lowest level of skill of the target population: students aged between 11 and 12 years in a FD school. The “brainstorming” technique was used, and the steps described by Pasquali [43] allowing student participation with suggestions for reformulating issues that are not understood. It was performed with this age group to ensure that the items were understandable to the group with the lowest skill, and therefore understandable for the most skilled group.

2.4.2. Construct Analysis (Round 1 and 2)

For construct validation, 22 specialists were available to analyze the first version of the instrument. Of these, 16 participated in the analysis of the second version. For the selection of judges, at least one of the following items was used: (i) minimum master’s degree or, (ii) research development in the areas of the studied constructs or, (iii) performance in the area of food safety or, (iv) performance in the area of school meals. This validation aimed to verify the adequacy of the latent attribute(s) behavioral representation, providing information on the representativeness and clarity of each item of the questionnaires [43].

Initially, experts received the necessary information and guidance on the instrument’s evaluation process, including possible participation in other rounds. All instructions for filling out the Consent Form were provided. After consent, they could access the instrument to facilitate its full view and the link that directed them to the SurveyMonkey^® platform.

The items were evaluated for simplicity, clarity, relevance, credibility, variety, and evaluation of the corresponding construct. The knowledge questionnaire was also assessed for balance and the Likert scales to measure responses adequately. For the correct completion of the evaluation, the constructs’ constitutive definition and the items’ adequacy criteria were provided. After the judges’ participation, their answers and considerations were transcribed to the Microsoft Office Excel^® 2007 program.

The Content Validation Index (CVI) was used to evaluate the judges’ contributions [62], which allows observing the judges’ level of agreement on each item. The item’s continuity was limited to a CVI more significant than or equal to 80% (≥0.80) [43]. Three types of CVI were applied: Item-CVI (I-CVI), Questionnaire-CVI (Q-CVI), and Scale-level CVI (S-CVI). Thus, the first refers to the values obtained from each item that makes up the questionnaires according to the criteria of simplicity, clarity, relevance, credibility, and the possibility of evaluating the corresponding construct. The second refers to the values obtained referring to the balance criteria of the knowledge questionnaire and variety criteria of the three questionnaires. Finally, the third refers to the scales used to measure the responses to each questionnaire.

2.4.3. Elaboration of the Final Questionnaires

The final instrument was developed based on the answers obtained in the previous step. Additionally, the four researchers’ consensus for the judges’ positions was also considered, with three dietitians, and one statistician, all postgraduate.

3. Results and Discussion

The construction process resulted in an instrument with three questionnaires that separately contemplate each dimension studied, with 11 items of knowledge, 11 items of practice, and five risk perception items (Appendix A). The steps carried out allowed the development of an instrument that contemplates the main aspects of food safety covered in the Five Keys to Safer Food [49], within a perspective specific to the reality that will be applied. When considering the reality and particular challenges inherent to the studied public, successful communication on food safety is possible within a diagnostic perspective to carry out effective interventions [63]. For Slovic (1987), those who promote and regulate health must understand how people think and respond to risk. If there is no such understanding, the formulated policies may be ineffective.

The biggest challenge during the research was to make the language simple and straightforward. The same difficulty was reported in developing the “EQ-5D-Y” instrument for quality of life-related to children and adolescents’ health. The instrument was an adaptation of the “EQ-5D-3L”, intended for adults, to suit the new audience. The process resulted in changes, especially in the wording (including illustrations, titles, and response options), to optimize item comprehension and improve data quality [64,65].

In a study to assess the psychometric properties of a Japanese version of the Dutch questionnaire on eating behavior for children, there was a review process by teachers. The idea was to ensure that the final version covered the students’ cognitive development and avoided any incomprehensible writing [66].

In this study, specialized judges in the education area were crucial due to the difficulty of adapting the language for children and adolescents. The contributions allowed to create an appropriate wording according to the audience’s stage of cognitive development.

The results of the steps required to prepare the questionnaires are detailed below.

3.1. Search in Databases

The studies in the literature allowed us to obtain essential information for the subsequent steps. However, some points stood out for the characterization of the instrument. When writing the items, the students’ ability to think logically was considered, according to Piaget’s development in which they find themselves—formal operative [20] and Pasquali’s [43] criteria for understandable and objective writing.

The three questionnaires that make up the instrument had an item structure similar to that developed by Da Cunha, Stedefeldt and De Rosso [50], and De Andrade et al. [8]. The decision on using the Five Keys to Safer Food [49], based on the possible lack of information on the topic by students, proved to be easy to understand and familiar to students. Additionally, these are more disseminated and more accessible information to teachers and guardians of children. The complementary search at BNCC [47] was important to verify the syllabus required for students aged 11 to 14 years from the Brazilian public education network. From the information obtained, it was possible to establish the expected level of understanding of the topic.

The risk perception questionnaire was based on concepts from the World Health Organization [67], from ISO 31000:2009 [68], and a study developed by Slovic [57]. The inclusion of the optimistic bias [8,16,58,60] brought the possibility of broadening the discussion on children’s topics. Some researchers suggest that children are also susceptible to the phenomenon of optimistic bias [60,69], but the scientific literature is still scarce.

The scale for measuring the risk perception questionnaire responses was based on the WHO document [2]. The risk perception questionnaire assessed the relationship between FBD symptoms and their severity and intensity based on the content used.

3.2. Conducting and Analyzing Focus Groups (FG)

The four FG (n = 36) allowed the language structure’s adjustment, including in the questionnaires terms and forms of communication mentioned in the discussions. The study design made it possible to include different perceptions and knowledge since social inequalities exist between urban and rural scenarios. The integration of the two realities enabled the understanding of the universe experienced by students in different contexts.

The content analysis of the discussions that emerged in the focus groups identified the language that the public understands, the knowledge of greater and lesser mastery, the habits, and different views related to food safety. This information supported the construction of the items. The units of record (UR) evoked are shown in

Table 1. Themes, units of record (UR), and relative and absolute frequencies of the evocations of the focus groups conducted with low-income students (n = 36) ages 11 and 12 (FD—Brazil).

Themes—Keys to Safer Food	Units of Record	FG1		FG2		FG3		FG4		Total by UR
		F	%	F	%	F	%	F	%	F	%
Keep it clean	Hand hygiene/Bacterial contamination	17	26	23	36	11	17	12	21	63	39
Separate raw and cooked foods	Cross-contamination	1	100	0	0	0	0	0	0	1	0
Cook the food well	Proper cooking	0	0	1	100	0	0	0	0	1	0
Keep food at safe temperatures	Food storage	2	20	2	20	1	10	5	50	10	6
Use safe water and safe raw materials	Sanitation of fruits and vegetables	0	0	2	8	15	62	7	30	24	15
	Safe/unsafe food	15	25	13	21	22	36	10	18	60	40

, where it was possible to observe the most discussed Keys to Safer Food by analyzing the statements related to each recording unit [49] and the most relevant evocations (

Table 2. Keys to Safer Food and selected individual evocations related to the focus groups’ food safety conducted with low-income students (n = 36) with school ages of 11 and 12 (FD—Brazil).

Keys to Safer Food	Evocations
Keep it clean	Hand hygiene/Bacterial contamination
	“If we do not wash our hands, we can infect the food, and we will eat the infected food, and the bacteria will infect us from the inside. It is always good to wash your hands if you touch the floor or something dirty.”
	“Bacteria are tiny animals. If I touch the floor, it is full of bacteria there. Bacteria are everywhere.”
	“You put the soap in your hand. We have to wash our hands up to the elbow. ”
Separate raw and cooked foods	Cross-contamination
	“If you combine the moldy food with other foods, the clean one can get mold from the one that you put together.”
Keep food at safe temperatures	Food Storage
	“You can keep the whole papaya in a warm place; when opened, you can store it in the refrigerator.”
	“If we are going to save food to eat today or tomorrow, we will store it in the refrigerator. If we are going to save it for a week or later, it is better to save it in the freezer.”
Use safe water and safe raw materials	Checking food for consumption
	“From the smell of fish if it is spoiled fish. Auntie, if it is food that comes in the bag, you can see the date on the label when it expires or because of the smell like beans.”
	“There are some rotten. Others get softer sometimes.”
	“When it expires. When it does not smell good. When the fly is climbing on top of that bad smell. Like milk. It stinks. The egg stinks when it gets rotten.”
	“Expiration of food is when you want to eat something, and it has been in your cabinet for a long time, and you hardly eat it, and it has passed its expiration date, and if you eat it can give you a stomach ache or even kill the person.”
	Safe/unsafe food
	“Unsafe food is fried, soda, fats, sweets, snacks.”
	“Unsafe foods are those that have pesticides. If we eat food that has pesticides for several years, we will die. Get stomach pain and diarrhea.”
	“Safe food is that there is no poison, that there is no animal.”
	“Safe foods for me are those that do not have pesticides and those that we grow.”
	“There are things that smell bad, but it is good to eat. Some look bad, but they are good. Some are bad, but they are good.”
	Sanitation of fruits and vegetables
	“Some foods have to put on bleach, like lettuce and tomatoes. And there are some foods that we have to boil.”
	“You have to soak the fruit, and then put a little bleach in it, then let it soak, to remove most of the microbes.”
	“I do not eat a thing with bleach in my food.”
	“Bleach? That is crazy, dude?”

).

The results obtained in the content analysis corroborate the findings from the DHC analysis of the IRaMuTeQ Software. From the DHC, the corpus, which is the text composed of the FG’s coded transcripts, was categorized into six classes, named according to the central narrative addressed in the discussions (Figure 2).

The UR hand hygiene/bacteria contamination has the highest number of evocations (39%) in content analysis and DHC analysis (46.9%). Students demonstrated knowledge about the importance of hand hygiene and the frequency and way of performing the procedure, understand that the human body is a source of contamination for food, and associate it with bacteria.

In the BNCC [47] for students aged six to 14 years old in Brazil, these subjects are present in life and evolution. At the age of six, body hygiene habits (such as washing hands before eating) and their relationship to health maintenance should be discussed. At the age of nine, the student must have appropriate attitudes to prevent diseases associated with them, based on knowledge of transmitting some microorganisms (viruses, bacteria, and protozoa). Hand washing is a widely discussed topic, and students have more access to it. A study developed by Osei Tutu et al. [19] to assess students’ knowledge and food safety practices, aged between seven and 21 years, showed that most students responded correctly about hand washing.

Then, the UR safe/unsafe food stood out (40%). Considering that the “three” and “four” classes resulting from the DHC correspond in content to this registration unit, it is noted that they stood out similarly (36.7%). Different perspectives were observed. When asked how to check when the food is fit for consumption, they responded by checking the expiration date on processed foods’ packaging and the food’s appearance and smell. Research with domestic food handlers has shown that some determine that food is safe because of its appearance or smell, both unreliable indicators for food safety [18].

There were differences between the participants. Many related the concept to nutritional terms for the concept of safe and unsafe food. A study developed by Gavaravarapu et al. [52] obtained similar results. Many teenagers initially confused the concept of safe food with nutritious food, listing many foods such as vegetables, cereals, milk, and meat as safe foods.

However, a large portion related the unsafe food to one that shows signs of deterioration, that is out of date and contains pesticides. The consequences of unsafe food consumption and the association between food consumption with pesticides and death were listed. Finally, in an urban school, the food was considered safe when planted and grown in the domestic environment, as there is no use of pesticides. It is noted that students have the perception that pesticides cause deleterious effects, listing symptoms such as stomach pain, diarrhea, headache, eye pain, and desire to sleep.

The concern shown in the students’ speeches regarding the importance of pesticide-free consumption comes in line with the significant and active role that consumers assume in the face of the movement towards achieving sustainable production and consumption. The United Nations’ National Goals reinforce this movement for the Sustainable Development Goals (SDGs) [70] and the Food Guide for the Brazilian Population [71]. The Guide shows that an adequate and healthy diet derives from a socially and environmentally sustainable food system, encouraging the domestic cultivation of organic foods and exchanging between neighbors to expand access to a greater diversity of foods. Such practice leads to the production of tasty foods and protects the environment and health.

Consumers have been looking for organic foods, stimulated by health and sustainability benefits [72,73]. Consequently, children and adolescents are influenced by the environments in which they are part of. According to Bronfenbrenner’s Ecological System Theory, different levels influence the construction of these individuals’ lifestyles, emphasizing the importance that the environment has on them. Thus, family (parents, siblings), school (teachers, colleagues), shared activities, culture, history, customs, laws, and the economic system can influence their education and choices [74,75,76].

A relevant point was food consumption with signs of deterioration, common among rural school students. Thus, an instrument to investigate this habit is needed. It is noted that individuals who recognize that these foods are considered unsafe still consume such foods. PISA -2015 [41] points out that Brazilian students from rural schools had the lowest ESCS index. This lower socioeconomic level may imply less knowledge about the harms of spoiled food and the low availability of food inputs, leading to unsafe food consumption.

Furthermore, studies have pointed out that food and nutritional insecurity are more prevalent in rural areas, characterized by lower family income and low education, contributing to establishing this situation [67,77,78,79,80]. Similar data were found in a study conducted with students in China [9]. It was observed that a large majority knew that expired foods could not be consumed. Only a small number still thought that expired foods could be consumed after boiling or heating or as long as they looked good. In this case, the consumption of unsafe food, expired food, may be related to their belief that appearance is more important than the expiration date information.

The consumption of unsafe foods due to a deterioration process or expired dates causes short-term and long-term consequences. These can be chronic kidney diseases, neurological disorders, reactive arthritis, and irritable bowel syndrome. Foods containing mycotoxin are also worrisome due to their carcinogenic potential [81].

Note that the most explored keys were keeping clean and using safe water and safe raw materials in both analyses. However, topics that make up these keys, such as sanitation of the environment and utensils and drinking water, were not discussed in the focus groups. The keys: separate raw and cooked food, cook the food well, and keep food at safe temperatures obtained less space within the narratives. However, it was possible to observe cross-contamination knowledge involving fungi and food shelf life at different storage temperatures. The most debated topics are related to the direct and more usual handling of food, such as handwashing and checking food before eating. The absence of some topics and the relatively impoverished speeches of others can be attributed to the absence of more direct questions about the focus groups’ subjects. Moreover, there are items related to cooking and cleaning responsibilities, which may not be part of their usual routines.

As for the sanitation of fruits and vegetables, the participants recognize the importance of adequate sanitation, but a large portion is unaware of the process that comprises three stages: washing, disinfection or sanitization, and rinsing. The disinfection or sanitization stage was the discussion point that caused the most divergence of opinions from bleach statements. This product has sodium hypochlorite as its active ingredient. This step is recommended by the Brazilian legislation [45,46], and the products must be regularized in the competent agency of the Ministry of Health, the National Health Surveillance Agency (ANVISA). Consumers have access to this information through the Food Guide for the Brazilian population [82].

Food handlers and consumers in a study developed by De Andrade et al. [8] asked whether washing vegetables and soaking them in water with vinegar is enough for this food to be safe for consumption. They obtained 57.8% and 40%, respectively. Given these data, it is inferred that the knowledge of Brazil’s appropriate process may be a deficit in different audiences. Gavaravarapu et al. [52] showed that in the developed FGs, there was also recognition of the importance of cleaning fruits and vegetables and reported that raw foods should be washed carefully before consumption. However, there is no report on how hygiene should be carried out.

Finally, the concepts of gravity [67] and vulnerability [48] became evident in the evocations.

“Unsafe food can cause illness, even death, diabetes, infection, a coffin.”

“The child gets more ill when he eats something bad; his intestines are more sensitive; the intestines of the adult are firmer.”

3.3. Questionnaires Development (Questionnaires Analysis)

From the information obtained with the previous steps, the first version of the instrument was prepared. The questions were distributed among the three dimensions: 12 items of knowledge, 12 practice items, and 10 risk perception items.

Three sessions were held for the semantic analysis stage, with four students each, heterogeneously concerning sex. A construct was worked on in each session. The terms that students understood poorly were replaced by suggestions made by them or by other terms used during the explanation in this step. It was noted that very long questions hinder the correct understanding of the item. Subsequently, it was found that such questions did not meet the criteria of simplicity and clarity [43].

In the construct analysis stage, 22 judges agreed to participate, 59% have a master’s degree (n = 13), 32% have a doctorate degree (n = 7), 4.5% post-doctorate degree (n = 1) and 4.5% specialization (n = 1). The judges’ group comprised a pedagogue and a psycho-pedagogue to contribute specifically to the language’s adequacy.

In the first round of the construct analysis, 28 items were approved (82.4%); they presented I-CVI values ≥ of 0.80. It is noted that all items have relevance and credibility. The variety criterion was met in the three questionnaires with a mean Q-CVI = 0.94. As for the balance criterion, Q-CVI = 0.91 was obtained. The average of the I-CVI values for each item in each questionnaire is shown in

Table 3. Average I-CVI values of the items of the versions submitted to the construct analysis of each questionnaire that make up the instrument for assessing knowledge, practices, and risk perception related to food safety for low-income students (FD-Brazil).

Items	I-CVI
	Risk Perception		Knowledge		Practices
	V1	V2	V1	V2	V1	V2
1	0.93	0.99	0.84	1	0.93	0.98
2	0.84	0.95	0.77	0.9	0.93	0.96
3	0.86	0.98	0.79	0.88	0.88	0.86
4	0.86	0.93	0.85	0.99	0.85	0.98
5	0.88	0.94	0.76	0.89	0.89	0.99
6	0.87	1	0.82	1	0.85	0.9
7	0.91	1	0.79	1	0.92	0.98
8	0.84	1	0.9	0.9	0.95	1
9	0.94	-	0.86	0.93	0.88	0.96
10	0.88	-	0.79	0.95	0.84	0.99
11	-	-	0.84	0.93	0.78	0.96
12	-	-	0.93	-	0.87	0.93

. It should be noted that all items, regardless of approval, were adjusted for clarity and simplicity.

Regarding the Likert scale evaluation, the scale used in the risk perception questionnaire was approved with S-CVI = 0.82, and the scale of the practice’s questionnaire presented S-CVI = 0.77, not approved.

After rewriting, removing items, and replacing the Likert scales used according to the judges’ suggestions, the questionnaires was resent for further evaluation. Regarding the scale used in the “risk perception” construct, as emojis are very useful in evoking emotions and attitudes [83], they can confuse the correct measurement of responses since students can associate with feelings and check the emoji that reflects their mood. Thus, it was replaced by another five-point scale, ranging from “no chance” to “100% chance”. Another five-point scale replaced the Likert scale of the “practices” construct, ranging from “never” to “always”. A study by Dalmoro and Vieira [84] points out that the three-point scale is less reliable and less able to demonstrate the interviewee’s opinion.

In the second assessment of the construct analysis, 16 judges agreed to participate. All items obtained I-CVI ≥ 0.80. The variety criterion was met in the three questionnaires with a mean Q-CVI = 0.94. As for the balance criterion, Q-CVI = 0.94 was obtained. The average I-CVI values for each item in each questionnaire are shown in Table 3. The two Likert scales obtained S-CVI = 0.94.

After completing the specialists’ second evaluation, the researchers responsible for the study discussed the items resulting in the third version of the questionnaires. They analyzed the need for modifications to obtain the final version. They included items deemed pertinent and restructuring sentences to better understand and modify the Likert scale of the risk perception questionnaire.

As seen in the semantic analysis stage and according to some judges’ positions, students could have difficulties understanding the concept of risk. Thus, it was decided to break down the risk perception items in their two dimensions—gravity and probability [67]. The term “chance” was used for the probability dimension because they are synonymous in the semantic context, and the epidemiological definition is not considered [85].

As the two risk dimensions were worked on separately in the risk perception questionnaire, there was a need to develop two scales to measure the responses (Appendix A). The probability dimension was measured using a five-point Likert scale associated with the ruler. The severity dimension was measured using a four-point scale associated with the figures that characterize symptoms at each severity level. Items 1.1 and 2.1 were coded from “one” to “four”, with “one” being “no severity” and “four” being “high severity”. The optimistic bias will be obtained by subtracting the score from item “two” (2 and 2.1) by the score from item “one” (1 and 1.1). Positive and significant differences indicate the trend of optimistic bias [14]. E.g., if in questions (1 and 1.1), the respondent obtained a score of 5 and 4 respectively, the final score was 1. In questions (2 and 2.1), he obtained scores of 4 and 2, with a final score 2. When checking the optimistic bias, it turns out that 2-1 presents a positive difference, being possible to identify the phenomenon.

The entire process of restructuring the questionnaires is shown in Figure 3.

4. Conclusions

The food safety scenario revealed in the discussions, and systematic database search made it possible to understand the addressed dimensions and constructs. Those were knowledge, practices, and risk perceptions, providing an instrument that considered the public’s reality and perspective, especially the issue language, to be simple, straightforward, playful, and understandable. The achievement of the objective was possible through a rigorous validation process. This process resulted in a reliable and appropriate instrument consisting of three questionnaires that helped determine a starting point for the formulation of public policies aimed at the universe of low-income children and adolescents. This study’s limits are represented by the fact that it does not include verifying the reliability coefficients (Conbrach’s alpha and temporal stability) and validity (regression analysis and internal consistency analysis) of the items that make up the instrument. However, this is planned for future studies.