Assessment of the Application of Content and Language Integrated Learning in a Multilingual Classroom

Round 1

Reviewer 1 Report

See the detailed comments in the attached file. The research goals are laudable and the results interesting, but the exposition and structure are confusing. A clearer distinction has to be made between the background to the study and the study proper.

Comments for author File: Comments.pdf

Author Response

Thank you for your comments. We made edits according to your recommendations.

A clearer distinction has to be made between the background to the study and the study proper. - We have added this information

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear Authors

This topic is very interesting for higher educators and institutions.It shows individual capacities of multilingual learners.There are some issues and parts of your paper that should be enhanced so I suggest the follwoing:

1)broader literature review is neded in the literature review-there are so many EU reports concerning multilingual learning issues

2)Discussion must be improved-there is no reference in this parts and no author with whom you "discuss" your results. You should relate your findings with others who did similar reserach.

3)Present your results more clear-what exactly you received

4)In Conclusion part there should be more practical isuues and recommendation for teachers, teaching institutions and further reserach in this area would be welcome.Also Limitations of your study are needed.

Author Response

Thank you for your comments. We made edits according to your recommendations

1)broader literature review is needed in the literature review-there are so many EU reports concerning multilingual learning issues - We have added the most important EU researches.

2)Discussion must be improved-there is no reference in this parts and no author with whom you "discuss" your results. You should relate your findings with others who did similar reserach. - We have added this information

3)Present your results more clear-what exactly you received - We have added this information at the end of the Results and Discussion parts

4)In Conclusion part there should be more practical isuues and recommendation for teachers, teaching institutions and further reserach in this area would be welcome.Also Limitations of your study are needed. - Limitations of the study are presented in the Discussion part. In Conclusion we added the future perspectives of our study.

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear authors,

Congratulations! Your paper is well-written and discusses an interesting topic!

Author Response

Thank you very much!

Author Response File: Author Response.pdf

Reviewer 4 Report

The subject of the article is very relevant and interesting in the domain of multilingual language learning and use. The linguistic context is quite focused, but it could be more clearly present at the beginning of the article. Some sentences seem to be too general, and the writers seem to assume that the readers are familiar with the Russian linguistic context. The introduction is quite well written but there are some repetitive sentences, that is, the same information is given several times. However, on some occasions, there is not enough sources to support the arguments (see at the end of this review). The research questions are well formulated, even if the third one requires modification. The description of the corpus needs some modifications, because at this moment the description is not quite coherent. In the abstract, the number of participants is N=48, on page 5 N=46, and on page 6 N=47. The section Materials and methods should be rewritten, there is a lot of repetition on the on hand, but some descriptions need more detail on the other hand (e.g., 5 groups of 25-35 people participated in preliminary testing; how many in fact were chosen, did the participants take twice the theoretical test, was the test the same etc.). The empirical part is quite well conducted, but more information is needed on the control group and content of the survey (pages 9-10).

In reporting of the findings, the qualitative part requires some precisions to increase the transparency and thus the reliability of the findings; for instance, the negative opinions need more attention. There were only 8 participants whose answers to the interview where reported. When presenting the qualitative findings, the link to the quantitative findings would give a more precise view on the students’ perceptions and the real success in the tests. Those students who did not succeed so well in the tests had more negative comments, but I wonder if you refer to those students with low scores on two tests or also to those who had low scores in one of the tests (language vs. professional discipline).

To sum up, I think that this study could contribute well to the field of CLIL studies; it gives valuable information on CLIL language learning in multilingual classroom during professional communication. It enhances oral proficiency by combining several foreign languages to learn international business issue. Please see the attached file for more detailed comments.

Comments for author File: Comments.pdf

Author Response

Thank you very much for your comments! Here you can find the answers.

p. 1. line 34: “continue to fall short of expected levels”. Please clarify this sentence, it is not general knowledge. In which context? In all countries or in Russia?  We meant that students' foreign language proficiency in Russia is worse than in Europe.

p. 1, lines 34-37: “With an obvious decrease in the problems associated with the availability of learning a foreign language within educational institutions, the effectiveness of language education is still unsatisfactory” – it seems that this sentence is not logical. Please verify the meaning of this sentence. We meant that despite the fact that there is a lot of learning hours of foreign language at university, the level of proficiency is still not satisfactory. 

p. 1, line 42: “a modern approach” – contemporary? Yes, it is. we changed the word.

p. 2, lines 45-47: Please add some resources when you talk about European learning and teaching contexts. Added.

p. 4 line 48; “its implementation in the curricula of schools and non-linguistic universities” – please add here the particular context, i.e., in Russia – in this part, make it clear, what is specific to Russian context and not in all countries (too not generalize too much) We explained it in the paper (line 48).

p. 2, lines 61- 63 & lines 74-75: some repetitive sentences Corrected.

p. 2, lines 68-72: You could perhaps use here also the term multilingual and trilingual learners (you use the term once on page 11). You could explain more clearly that the point was to learn Spanish and international business at the same time, English was the teaching language as the students came from different linguistic background – their L1 were different, teaching was on their L2, and they were learning a L3. Done.

1.1.Literature review – if you have a chapter 1.1., you should also have a chapter 1.2. – reconsider the numbering of your chapters Corrected.

1.1.1 Content and Language Integrated Learning and its forms
- p. 2, lines 84 and 87: Content and Language Integrated Learning (CLIL) is given on line 84, it suffices to give the abbreviation on line 87 Done.

1.1.2. Multilingualism
- p. 3, lines 117-119 and p. 4, lines 156-157 you give too different definitions of the multilingualism. You could specify them for instance by adding “on individual level” and “in multilingual classroom” to show that there are too definitions but on different context (individual vs. social) Done.

p.3, lines 131-132: “Teaching foreign languages was often viewed as teaching one teacher for each foreign language” – please verify the sentence structure. Changed. (In teaching foreign languages, it has often been assumed that one teacher teaches only one foreign language.)

- p. 3, lines 132-133: “It was relatively rare to find people teaching two or more foreign languages [34].”- there are countries like Finland where it is normal to teach two or more foreign language. Please contextualize your argument.  Added.

- p. 4, lines 163-164 “multilingualism is also forms “part of the discursive regimes that students must perform in the 21st century must perform” – You have changed the text here, please verify that the quoted sentence is identical to the original text. Yes, it is correct, thank you.

- p. 4, lines 188-189: “In the Scandinavian countries” – do you refer to all the Scandinavian countries or only to Norway? We referred to Norway and clarified it in the text.

- p. 4, lines 191-198: The reference to Norwegian context seem quite far from your focus. The reference is not listed in the reference list and its form is not the same as elsewhere. Multilingualism is one of the key elements in educational policies in European countries (Multilingualism: The language of the European Union (europa.eu), About multilingualism policy | Education and Training (europa.eu). You might want to argue on this basis. We considered the example of the Norwegian context as one of the options of multilingualism implementation in which students rely on languages learned before in the process of learning new languages, considering at the same time similarities and differences of these languages.

- p. 5, lines 210-211 “Analysis of the literature showed that the use of content language integrated learning for a multilingual group of students has not been studied” – there is surely on gap but see for instance the references I listed at the end of the document. Please modify the sentence.  Done.

-  p. 5, line 216: students’ N is 46, in the abstract 48, on page 6 the N is 47 – please verify the number of the participants and be consistent Corrected.

- p. 5, lines 218-219: on page 2 you explain in a clearer way why these students learn Spanish (L3) by using English (L2) which it is not their first language (L1). The language background of these students could be explicitly mentioned, because it has an impact on their language proficiency, for instance, if their L1 is another Romance language. Already teaching International business in English might be considered an CLIL context depending on the students’ language proficiency in English. Please try to argument more clearly and give some more information on students’ L1s. We added more information students' L1. Teaching International business in English is not considered as CLIL as it is EMI.

- page 5, lines 218-229 there is some repetition, the same content is given in several sentences in different paragraphs. Please rewrite this part.  Corrected.

- p. 5, line 235: “in a specific algorithm” A specific educational approach? Teaching method? Educational model? Yes, we meant teaching method. Corrected.

- p. 5. Figure 1: Could you please add the stages in the Figure and the tasks also, for instance, familiarization with and translation of the new vocabulary The information on the stages is indicated under the Figure. Information about tasks is added.

- p. 5, lines 238-239: Please add some concrete examples of the words that the students searched for the task. This would increase the text coherence and would give more information about the tasks. Added.

-  p. 6 line 241: “translate it into their native language ("flipped classroom")” – please add before the classroom session, because not the concept of flipped classroom might be new to some readers Done.

-  p. 6, lines 242, 258 & 273: electronic educational platform Moodle, the online platform Moodle – the online platform could be used in all cases, and it is enough to determine it once and then use just the word Moodle Added.

-  p. 6, line 244: both languages – English and Spanish? Yes, in it.

-  p. 6, lines 246-247: “During the lesson, students discuss in English (possibly the use of Spanish) vocabulary” – they only discuss the Spanish words in English or also the English words – please clarify this part They discuss all vocabulary and professional terminology (both English and Spanish) that was unclear for students. we clarified it in the text.

-  p. 6, third stage: Could you give a concrete example of the task/vocabulary/subject? Added.

-  p. 6. lines 258-259: “The electronic educational platform Moodle presents video and audio materials” – please rephrase the sentence, the platform contains these materials, does not present them Corrected.

-  p. 6 line 270: (5 groups of 23-25 people) – it is not quite clear what this means, please explain in more detail what is meant. Did you have over 120 students of which you chose about 46-47-48 students for the experiment and the control group? Please clarify and verify the number everywhere in the text. Yes, we had over 120 students for pre-experiment testing and chose just 47 of them for the experiment.

-  p. 6, line 286: The professional discipline test – was this test done only in the experimental group? If so, could you explain why you did not test the control group also? Furthermore, if the experimental group was tested twice, was it the same test or a different one? Please give some more information on this test. The testing was for both groups. It is indicated in the text of the paper and the results of both groups are presented.

-  p. 7, the Third research question: please rephrase the question. Done.

- p. 7, Table 1: Could you please add the N values in the table? Is it so that the two first lines is related to both groups (scores on testing) and the two others (interviews and survey) only to two the experimental group and there also to only a part of the group? We added N values.

- p. 8: Could you elaborate on the results of the writing proficiency in more detail? It seems interesting that all other competences improved more in the experimental group but not writing. Did the traditional group work differently so that the writing competence is somewhat focused? What is the traditional teaching method? How is it different from the new model? Added (line 306-309)

- p. 9, 325-330: How many questions did they have in the pre-test (20?) Where these questions in the final testing (25 closed questions) the same kind of questions? Could you give some examples? We did not make a pretesting on the International business discipline. We added an example of a question. 

- p. 9, lines 337-339: “learning model consisted of 5 statements and was proposed to the experimental group of students. In this part, the students were asked to rate ten statements according...” – please modify and clarify this part Thank you. Corrected.

- p. 9 Table 4: “The current learning model provides me with different learning styles and can make my learning more fun” – this question might be a little bit problematic as you ask two different things here. Students might agree with the first part of the question but not with the second part. We agree and will take it into account in our further research.

- p. 9, lines 347-349: “11 people agreed to the interview and the subsequent processing of this data. Among them were students with different academic performance. For a fair assessment, 8 students were selected from the volunteers” – From the group of 24 (?) students in the experimental group, you chose finally 8 participants who made the interview? Why did you not take all the 11 students? It is not quite clear what you mean with the fair assessment. Please try to clarify this point.  The clarification was added (line 343-344).

- p. 10 Could you give some original answers of the students as examples? We added several examples.

-  p. 10, line 363: “students with poor academic performance” – What does academic performance mean here? The Spanish task or the professional discipline task or both?  Both.

-  p. 10, lines 374-375: “As a result, 5 out of 8 students expressed satisfaction with the passed course. 2 students remained indifferent. One student spoke negatively”. You wrote just above that students with poor academic performance gave more negative feedback. Does this concern just this one student or more student? Could you please reconsider how to explain this result? We added clarification (line 353-355)

- p. 10, line 392: “The use of multilingual education was very popular among students.” – Do you refer here to the 8 students who you interviewed, or to all the group of 24 (?) students. If so, could you explain how did you end up with this conclusion? It is now quite evident form the results you have given in the text. We clarified it in the paper.

-p. 10, line 393: “at the first time” – at the beginning of the course? We changed the phrase.

- p. 11, line 410: the writing skills improved more in the traditional group. Do you have an explanation here for this result? Does the traditional teaching focus more on the writing tasks? Added.

- p. 11, lines 415-416: “he results of testing students in the course of international business (professional discipline), the experimental group coped better than the control group”– on page 9, in Table 3, you can see that the control group was better than the experimental group. Please verify this result/sentence. It was a mistake, we verified it.

-  p. 11, lines 423-424: “A s a result of the survey and interviews with students, it was revealed that students evaluate this model as effective” – please mention here how many students answered the survey and how many were interviewed. Added.

-  p. 11, line 437: “trilingualism” – it is the first time you mention this concept. You could use this already earlier in your text when you describe the corpus. We decided not to introduce a new concept and changed it to "multilingualism"

Author Response File: Author Response.pdf

Reviewer 5 Report

The article is important for the debate about CLIL in Russia 

Author Response

Thank you very much!

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

See the attached file

Comments for author File: Comments.pdf

Author Response

Thank you very much!

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear Authors

Unfortunately you didnt respond to any of my suggestions and recommendations. I cannot approve this paper in this form

Author Response

We made corrections according to your recommendations. Unfortunately, we do not understand what exactly is wrong.

Author Response File: Author Response.pdf

Reviewer 4 Report

The subject of the article is very relevant and interesting in the domain of multilingual language learning and use. The authors have made most of the suggested modifications, which rendered the article more coherent and the argumentation more convincing. I am also very satisfied with all the added concrete examples from the corpus. The text is easy to read and to follow the focus and the argumentation. I still have some small suggestions of modifications, which rely on the authors’ response to the reviewer.

To sum up, I think that this study could contribute well to the field of CLIL studies; it gives valuable information on CLIL language learning in multilingual classroom during professional communication. It enhances oral proficiency by combining several foreign languages to learn international business issue.

Suggestions of modifications (5) - see the attached file.

Comments for author File: Comments.pdf

Author Response

Thank you for your comments!

1) p. 1. line 34: “continue to fall short of expected levels”. Please clarify this sentence, it is not general knowledge. In which context? In all countries or in Russia? We meant that students' foreign language proficiency in Russia is worse than in Europe.

→Could you add this explanation sentence in your article? I would make your point even clearer.

We added this sentence.

p. 1, lines 34-37: “With an obvious decrease in the problems associated with the availability of learning a foreign language within educational institutions, the effectiveness of language education is still unsatisfactory” – it seems that this sentence is not logical. Please verify the meaning of this sentence. We meant that despite the fact that there is a lot of learning hours of foreign language at university, the level of proficiency is still not satisfactory.

Could you add this explanation sentence in your article? I would make your point even clearer.

Added.

Could you put the paragraphs together and add an explicative sentence / connector / connective clause according to what you explain above, like

Added.

Could you mention this link explicitly, like
p. 6, line 275-266 “Based on the most similar results in preliminary tests, we chose a part of the participants and created 2 groups.”

Done. 

I now understand would you mean here, but the problem is that you just explain above the preliminary test, then the language test taken two times in both groups and then the theoretical foundation test (= discipline test). You could add here that this test was only done at the end of the course in both groups. At this point, this explanation is not coherent with your answer.

We clarified in the paper that this test was only done at the end of the course in both groups. 

Author Response File: Author Response.pdf