Students’ Perceptions of Gained and Lost Value: A Case Study of a Summer School That Had to Suddenly Move Online

Abstract

The COVID-19 lockdown induced a sudden migration of traditionally presential learning activities to online domains, as was the case of inter-institutional summer schools. This research corresponds to a case study in which our organization had to reformulate, in less than three months, one of its traditional summer schools while trying to keep the original goals. Through qualitative and quantitative surveys, we aimed at identifying the impact of our reformulation through students’ perception of gained or lost value regarding four topics: (a) online teaching, (b) pre-recorded business cases, (c) online social events, and (d) technical solutions. By analyzing these four topics with emphasis on participants’ knowledge and learning experience, we identified some “tensions” leading to loss of value (i.e., belonging, performing, and organizing). These tensions suggest that future reformulations should be conducted considering students’ backgrounds and motivations.

1. Introduction

The teaching–learning process of more than half of the world’s student population was affected by the sudden and unplanned shutdown of facilities [1] after COVID-19 was declared a pandemic by the World Health Organization (WHO) on 11 March 2020. On short notice, educational organizations were forced to make crucial managerial decisions to reconfigure their traditionally face-to-face activities, including internships [2], conferences, and summer schools [3,4]. While previously seen as an option or as a fashionable way to deliver lectures and training, the “e-learning” method became the lifeline for higher education during the pandemic [5]. As a result, webinars, for example, increased 300% in 2020 compared to 2019 [6].

The unplanned reconfiguration of the educational field, referred to as a “COVID-19 emergency online learning model” [1], “pedagogical shift” [7], “online transition” [8], or “emergency remote teaching” [9,10], was marked by strategic managerial decisions at organizational (e.g., funding institutions and partner universities) and individual levels (e.g., teachers, course designers, and students). Generally, in this research we describe a case study in which our organization, a post-graduate unit within the University of Rennes 1, France, had to react, at the start of the pandemic in 2020, to reformulate in less than three months one of its most acknowledged educational activities, that is, a two-week international summer school on the topic of “smart cities”, which would have normally hosted a learning community of around 50 students, with varied academic backgrounds and from all over Europe. Specifically, we assess the impact of the resolutions taken to reformulate the event so that the original goals of the European leading coordinator, EIT Digital, could be attained.

Managerial decision making in “fast-changing environments” constitutes a large body of inquiry. As an example, some researchers have worked on the notion of “improvisation” [11,12,13], which accounts for a resolutive action when no plans are available [14,15]. “Improvised decisions” are often approached from two points of view: decision makers, that is, how members behave (individually, interpersonally, organizationally) towards an unexpected situation; and the effects or tensions.

While keeping distance from this so-called “improvisation” perspective, in the present study we put emphasis on two aspects. First, we focus not on the decision maker, but on the user (this is, students attending the educational activity), considered as the actor who was mostly affected by the managerial decisions of our organization. Although all summer-school stakeholders are in some way impacted by reactive reformulations (e.g., instructors [5]), students as users are particularly vulnerable since they have little or no formal negotiation power in decisions [16]. Second, we focused on the possible tensions produced by the reformulation [17] and that can be recognized from students’ evaluations.

To understand how this organizational change impacted users, we analyze how the proposed (new and emergent) functionalities deployed along the abruptly-shifted-to-online summer school were assessed or perceived by the students after they finished their virtual learning activities. Such assessment is conceptualized and analyzed in terms of negatively or positively perceived value: if the online functionalities are positively assessed, value is gained; if some of its features are negatively evaluated, value is lost.

Among the reactive actions taken by our organization, four main decisions can be found: (1) defining, selecting, and collecting pre-recorded (or asynchronic) business case presentations instead of having real-time face-to-face conferences from industry experts; (2) offering online coaching sessions instead of presential workshops; (3) preparing online social events instead of presential socialization activities; and (4) using the most convenient technological solutions (communication platforms) for delivering the activities. By analyzing these actions, we aim to learn about their impact on students by exploring perceived value from two dimensions, that is, didactics and pedagogics, where the former is related to knowledge production and content learning, and the latter to experience or learning conditions [18]. The objectives leading our research are:

General objective: Learn about the impact of reactive decisions (1. preparing pre-recorded business case presentations; 2. offering online coaching sessions; 3. preparing online social events; and 4. defining the most convenient technological solutions) to migrate the summer school to an online domain.

Specific objectives: (a) Identify how students assess or perceive value in the reformulated summer school in terms of didactics (i.e., knowledge and content learning) in relation to the four managerial decisions; (b) identify how students assess or perceive value in the reformulated summer school in terms of pedagogics (i.e., “learning experience”) in relation to the four managerial decisions.

The content of this article has been organized as follows: firstly, we present our theoretical framework, specifically in relation to the role of knowledge and experience in the assessment of value; secondly, we describe our methodology; and thirdly, we present and discuss our main results.

2. Theoretical Framework

2.1. Assessing Decisions: Perceived Value through Didactics and Pedagogics

In our case study, students’ knowledge (didactics) and students’ experience (pedagogics) are used as proxies to account for value that has been perceived as gained or lost in the reformulated activity.

2.1.1. Didactics Dimension: A Knowledge Perspective

We explore the perception of value through the dimension of didactics, conceptualized here as the complementarity between prior knowledge and disciplinary knowledge. Disciplinary knowledge has been defined as the curricular knowledge (co)constructed between students and instructors, whereas prior knowledge has been defined as “the whole of a person’s actual knowledge that: (a) is available before a certain learning task, (b) is structured in schemata, (c) is declarative and procedural, (d) is partly explicit and partly tacit, and (e) is dynamic in nature and stored in the knowledge base” [19] (p. 4699).

By exploring the knowledge dimension, we establish that students’ prior or ongoing academic training may lead participants to give a certain value assessment (positive or negative) in relation to a reformulated event. This idea has its roots in at least three common claims found in the educational field: (a) students’ prior knowledge may affect learning performance [20,21]; (b) designing engaging lessons, taking into account students’ interests, may encourage more active learning participation [22]; and (c) differentiated instruction, according to each student’s background and readiness, may have a positive impact on student achievement [23,24]. Thus, students’ knowledge background would play an important role when assessing their learning activity.

2.1.2. Pedagogics Dimension: Experience

Pedagogics, the second dimension used to explore the perception of value, is materialized in terms of students and teachers’ interactional experience in context. Recent literature framed in the pandemic context has paid attention to experience mainly through three aspects that are commonly assessed by students and teachers: learning practice, feelings, and accessibility. In relation to the learning practice, time devoted to online activities has received considerable attention. For instance, it has been suggested that classes should be short, below 30 min [7], which is consistent with another study which, based on the emotions of instructors, revealed that the optimal course time in a distance learning environment should not exceed 20 min [25]. In this sense, longer classes may lead to a negative assessment, as reported by [8], where only 35% of pupils from schools in Delhi considered online classes to be as effective as face-to-face classes. These critical assessments have led scholars [26,27] to provide some guidelines to improve the online teaching practice, for example, by (a) incorporating collaborative activities, emergency plans for unexpected situations, and a mixture of synchronous and asynchronous events or activities; (b) providing students with flexibility (“compassionate flexibility” [9]) since they may be experiencing different realities; and (c) designing different types of assessments for evaluated tasks.

In the experience dimension, it is also possible to find research accounting for the assessment of feelings or psychological attitudes from students or teachers involved in online activities. For example, it was found, in a sample of 100 students, that only 50% felt that they possessed adequate digital skills to use online learning tools, whereas 33.8% thought that their digital skills were inadequate [8]. Similarly, one survey applied to 126 university students in Pakistan showed that 71.4% felt qualified to use a computer/laptop, and that only 10.3% felt more motivated with online learning than with conventional learning [27]. Interestingly, a study showed how nursing students perceived the pandemic and what their personal experiences were while studying during the COVID-19 crisis, highlighting their fear of infection within clinical settings [28].

As for academics’ feelings, scholars [10] found, in a sample of 1148 British university teachers, that preparedness and confidence were assessed differently according to disciplines. For instance, academics from computer sciences felt mostly prepared (66%) and confident (75.8%), while languages teachers felt less prepared (30.4%) and less confident (48.7%). In this context, educators felt stressed and anxious when they were forced to rapidly move their teaching online due to the COVID-19 physical distancing measures [5]. Among the reasons identified, limited eLearning readiness [29] and lack of prior training in the use of technology [30] can be found in the literature. The sense of lacking the right skills may lead to feelings of frustration, inhibited learning, and, of course, a perception of online learning activities as ineffective. Indeed, researchers have found that pupils described online courses as individualizing learning, limiting interaction with others, and contributing to a sense of isolation from instructors [31].

Literature on accessibility, on the other hand, has been focused on students’ assessment of access conditions to online sessions [1,32,33], proposing some action lines for specific university programs [34]. Accessibility is one of the key principles of online learning [35]; however, important gaps in and between countries are still found [36]. From a critical standpoint, there is evidence about the challenges faced by rural learners in South Africa when adjusting to the new mode of learning, characterized by predominant use of online systems and low-tech applications [37]. Although empirical evidence about the pros and cons of distance learning for rural and urban students is still missing [38], studies claim that the new online context has excluded many rural learners from teaching and learning due to a lack of resources to connect to the internet and low-tech software [37]. Thus, accessibility, specifically in relation to the ease of handling online-teaching applications, may be a key aspect for assessing the impact of improvised learning activities.

2.1.3. Tensions

The identification of assessed value (e.g., of a particular decision) may reveal “tensions” among individuals and the organization. The framework of “Organizational Tensions” [17] offers a categorization of “paradoxes” or “tensions”, that is, learning, belonging, performing and organizing [17] (p. 383), among stakeholders facing a sudden decision. Since we focus on students as users, we draw mainly on the last three. For belonging, identity fosters tensions between the individual and the collective and between competing values, roles, and memberships (for instance, some students would claim a “student identity,” while others would claim a “professional identity”); for performing, plurality fosters multiple and competing goals and strategies as stakeholders seek divergent success (for example, some students would expect to learn about entrepreneurship, while others would expect to have fun); and for organizing, structuring and leading foster collaboration and competition (for example, students would not feel comfortable with arbitrarily defined teamworking or other decisions; or some participants would prefer using alternative digital resources, other than those proposed by the organization).

3. Materials and Methods

In the next section, we describe the methodology that will allow us to learn about the impact of our four reactive decisions.

3.1. Context in a Nutshell

The summer school event, organized by the Université de Rennes 1 and coordinated at a European level by EIT Digital, was held online in August 2020. This event corresponds to a mandatory module, which is worth 4 ECTS (112 h of student work) in the EIT Digital curriculum (Master and Doctoral programs). On the topic “Unleashing the Power of Circular City Data”, the general objective of the activity consisted in providing students with knowledge on innovation and entrepreneurship.

The abrupt change in the educational scene impacted heavily on the several stakeholders in charge of organizing the summer school. Several meetings were held between the European general coordinators (in charge of EIT Digital) and the international partners to reformulate the event as coherently as possible; however, the novelty of the situation, the multiples emergencies to be handled by the universities at that time, and the reduction of human resources due to the confinement created an environment in which no clear guidelines were available. Uncertainty about the online summer school execution prevailed for more than three months, a period in which the general coordinators were hesitant not only about the financial feasibility, but also about the satisfaction rates, which would normally correspond to key metrics for promoting future versions of the event. Since the budget had been already allocated and answers were strongly demanded by partners (rumors about cutting off funding had started to circulate), the general coordinators decided to change the dates of the summer school to the third week of August 2020, a date in which summer holidays would come to an end for most participants. Despite partners’ objections, the decision was irrevocably made.

3.2. Participants

The summer school normally hosts external (non-EIT) Master and PhD students, leveraging diversities (i.e., teams mixing EIT with non-EIT students and maximizing cultural differences within teams while keeping an even gender balance). On this occasion, 32 students were enrolled in the program, but only 26 participated in this research. Of these 26, 15 were part of a computer-related master’s program in the EIT Digital consortium, while the rest were externals with other backgrounds. Thus, two groups of participants were finally distinguished: firstly, the ‘computer science’ group (CS), made up of 18 students, including those with an academic background in electronics and computer science (i.e., autonomous systems, IoT engineering, cybersecurity, and so on); and secondly, the ‘social science’ group (SS), composed of 8 students, including those with a background in management (economics or public policy) and human sciences (law, linguistics).

3.3. Decisions to Achieve a Shifted-to-Online Summer School

The imperative need to shift the summer school to a fully online setting engaged our organization with an unprecedent process of adaptation and rapid decision making. In terms of the production of the event, the reactive decisions primarily consisted of four main reformulations. Next, the four main decisions that shaped the shifted-to-online summer school are described.

3.3.1. Decision 1: Reshaping Business Cases

The first decision consisted in defining, selecting, and collecting recorded (or asynchronic) business case presentations to replace real time face-to-face coaching interventions. Without COVID-19, business cases would have consisted of industrial challenges presented in conference mode by a company worker or a start-up owner, who would have supported students until the end of the event. Normally, this type of presentation receives a lot of attention from students, as they are performed in a lively and asymmetric atmosphere. In the “new normal”, the selection process was radically different. Our organization unit decided to contact as many business case providers as possible, independently of the entrepreneurial domain (as long as their innovations remained in the topic of the event). Due to home confinement, many entrepreneurs rejected our invitation since, as they explained, they did not have the conditions to participate. Indeed, many of them were teleworking with their kids at home. After contacting at least twenty national and international entrepreneurs, eleven answered positively. Broadly related to applications in mobility, business cases ranged between 30 and 40 min and varied between environmentally oriented and everyday-challenge oriented (

Table 1. Description of pre-recorded business cases.

Business Case	General Description
Wipsea	AI-based technology that recognizes wildlife in urban zones for preservation purposes
DriveTrust	Dash camera that identifies drivers’ behavior behind the wheel
Vipo	AI-based technology that provides customers with clothing recommendations
Imagine	Technology applied to building structures to monitor their health status
Keolis	Project explaining how public transport can be improved in Rennes
Wi6labs	IoT technologies for solving energy consumption issues
YoGoKo	Project explaining how citizens could improve city transport flux by providing data
Car Free-floating	Research on a car sharing model
Panga	IoT technologies for solving energy consumption issues
Rudi Platform	Open data platform for developing technical solutions
Fabcity	Project explaining how citizens could improve city services in Rennes

).

The activity switched from a synchronic format, in which students could normally exchange information with business case providers, to an asynchronic version, in which interaction was deleted. The final product was a pre-recorded video session in which the presenter was interviewed by one of the organizers.

3.3.2. Decision 2: Offering Online Training Sessions

The second decision consisted of offering online coaching sessions instead of presential classes. Normally, experienced coaches are hired to train students in the field of innovation and entrepreneurship, using their own methodology. In these live training sessions, instructors strive for highly interactive discussions with students, and participants have the possibility to interact among themselves effectively. In a normal context, students and coaches could maintain flexible discussions at lunch breaks, for example. Contrarily, in the new normal, training sessions changed drastically. Interactive discussions were limited since most training sessions adopted a monologue style and student groups were defined a priori and kept for the whole event. From a contractual perspective, the organization decided to change the contract terms and ask the already-hired coaches to deliver online lectures (without knowing whether they had enough experience behind the camera or using conference software). Despite the dead-end request, coaches decided to take the challenge, with the support of the organizing team. Finally, the online courses, delivered 6 h a day online, corresponded to: Identification of a problem, Technological watch, Definition of a value proposition, Training for intermediate presentation, User Experience definition, Strategic business plan, Public presentation and pitch, and Business model definition. The online delivery may have led to an inevitable loss of non-verbal communication, a situation which has been reported as a significant challenge for students as well as for teachers. For example, one study [39] showed that lack of non-verbal communication may trigger low student engagement in the activity or poor interactivity. From the side of instructors, the online learning context gets particularly challenging as it often turns ‘faceless’ when participants turn off their cameras [32,40]. This loss of non-verbal meanings may have an impact on the “psychological closeness between the student and the teacher” [33] (p. 8).

3.3.3. Decision 3: From Social Events to Online Socializing Activities

The third decision consisted of preparing online social events instead of presential socialization activities. In normal times, social events would have been one of the main motivations to attend the summer school. Events, such as dinners at lively restaurants and weekend tours around the hosting city, would have allowed students to network and obtain information about potential European universities to finish their master’s degrees. In the context of COVID-19, these social events had to be replaced somehow. The management team decided to organize three online social events in which participants had the opportunity to play a game and solve trivia while socializing with other classmates at the end of the coaching sessions. To minimize the negative impact on students, social events were thought to last no more than one hour.

3.3.4. Decision 4: Choosing Technology Solutions

Finally, the fourth decision consisted in choosing the most convenient and practical technological solution (communication platform) for administrating the learning activity. In normal times, platforms were not part of the activity, not even for sharing information. In fact, information was shared through highly elaborated brochures that contained all planification details. In the reformulated version of the summer school, a technological platform had to be used to deliver the online lessons and share content. After revising several alternatives, Zoom was chosen for training sessions, mainly because of its simplicity and its ‘breaking room’ option, which allowed students to work in groups in separated spaces. For organizing content and asking for deliverables, Moodle was chosen, mainly because it was supposed to be intuitive for both students and instructors. Other platforms, such as Teams, were discarded by the organization as they did not seem very intuitive at a first glance.

3.3.5. Decisions’ Assessment via Didactics and Pedagogics

The assessment of the four managerial decisions was explored through didactics and pedagogics. From the didactics dimension, we explore how the interrelation between students’ prior knowledge and students’ new knowledge may play a role in the assessment of the summer school activity. Prior knowledge, on the one hand, is understood as knowledge acquired before the summer school, and it has been explored according to three categories: academic background (i.e., knowledge acquired by students through their former university training); professional background (i.e., knowledge acquired through professional working experience); and personal background (i.e., knowledge based on personal interests or preferences). On the other hand, the role of new knowledge in the assessment of the summer school event was explored through the assessment of lectures or training sessions. From the pedagogical dimension, assessment is explored through the student’s experience according to the main structural components of the activity: pre-recorded business cases, online teaching, social events, and technological solutions.

3.4. Collection of Data

The 26 students, after completing the summer school, were asked to answer two complementary surveys (Appendix A) whose main purpose was to identify how they assessed the activity. Including open-ended qualitative questions, the first survey (designed by the authors) aimed to collect data about the learning experience (i.e., use of online educational resources and collaborative work). The second questionnaire (designed by EIT), including quantitative questions using a 5-point Likert scale (1: Very Poor, 2: Poor, 3: Fair, 4: Good, 5: Excellent), aimed at collecting specific evaluations about the event, such as content, program, and organization. For this, mean values were calculated for both groups of participants (CS and SS). The analysis of qualitative data consisted of a codification process conducted, iterated, and revised by the three authors.

3.5. Ethical Considerations

The 32 attending students were invited to voluntarily and anonymously (if they wished) answer two surveys, one from the EIT institution and another from our research team, on their participation experience. They were orally informed (before, during, and after the event) that their answers would be used for promotional and improvement purposes, in the case of the EIT survey; and for academic purposes to understand the impact of the use of online resources, in the case of our research team. Before taking the surveys, students were clearly informed about the purpose (i.e., to collect their feedback for promotion and research), its anonymous character, and the inexistence of risks. After this information process, 26 students finally participated in the activity. In compliance with the European Regulation, specifically the General Data Protection Regulation (GDPR) applied to research in humanities and social sciences, all subjects’ personal data obtained from the surveys (for example, names) underwent a process of “irreversible anonymization”, after which individuals can no longer be re-identified.

4. Results and Discussion

In this case study, we have aimed to learn about the impact of the reactive decisions taken to reformulate an international summer school. To meet this general purpose, we have explored how students assessed the reconfigured event in relation to the four managerial decisions.

Table 2. Decisions, reformulation, and students’ assessment according to didactics and pedagogics dimensions.

Decision	Main Reformulations (x → y)	Users’ Assessment (A Selection)
		Didactics: Knowledge Production and Content Learning	Pedagogics: Learning Conditions
1: Reshaping the business cases	Professionals from the industry, presenting in synchronic mode → Pre-recorded business cases administered asynchronically.	Students with technical and social backgrounds positively assessed certain types of business cases, i.e., environmental-related cases with concrete applicability.	In general, participants assessed the recorded business cases neutrally, although some participants evaluated them positively in terms of their clarity (for non-engineers).
2. Offering online training sessions	Instructors hired for offering synchronic classes → Change of contract terms, pushing coaches to teach online classes instead of face-to-face sessions. Interactive discussions between instructor and student in and out of the classroom → Disappearance of lively discussions in the online modality.	Academic and professional backgrounds helped CS to “analyze technical questions”, “decide the feasibility of a project”, and “use standard techniques and diagrams to understand the market”. In addition, some learned new aspects of entrepreneurship. SS students explained that their academic background helped them think in a relational way, e.g., identifying stakeholders involved in innovation projects. Some learned new aspects that would help them develop a business idea.	Pitch training sessions, which rely heavily on nonverbal communication, were negatively assessed by students. Online sessions were assessed negatively in terms of workload (classes were held from 9 a.m. to 4 p.m.).
3. From social events to online socializing activities	Socialization events, such as evening dinners and weekend visits. → Online games and cultural visits.	-	Social events were assessed neutrally. Students expected to achieve a high level of interaction with their mates. Short duration of social events was positively assessed by students. However, if previous coaching sessions did not finish on time, social events were considered too exhausting and were negatively assessed.
4. Choosing technology solutions	No technological solutions for administrating sessions or sharing information. → Use of technological platforms (Zoom/Moodle).	Given their academic backgrounds (familiarity with the apps), students positively assessed the use of Zoom and Moodle.	Students missed interaction elements of the physical version. Zoom and Moodle did not fulfill all students’ needs; alternatives were found to share documents and to maintain more fluent communication (WhatsApp, Facebook Messenger, Telegram, Google Drive).

summarizes our main results.

4.1. First Decision

From the point of view of didactics, students’ prior knowledge as well as personal interests were decisive to assess the value of the recorded business cases. For example, CS participants with an engineering background were inclined to positively evaluate, specifically, cases targeting every day and environmental challenges through artificial intelligence. They tended to judge almost all of them in terms of ‘fair’ to ‘good’ (3–4 in the Likert scale), except for DriveTrust and Wipsea, which were assessed more positively, from ‘good’ to ‘excellent’ (4 and 4.1, respectively). DriveTrust, a dash camera that identifies drivers’ behavior behind the wheel, may have been positively valued since it involved people’s safety issues. As a CS student stated, “I’m really interested in improving safety while driving”. “Citizenship concern” may also explain why CS students positively assessed Wipsea, an AI-based technology that recognizes wildlife in urban zones for preservation purposes.

The perception of value among students with a background in social sciences (SS) differed from that of their counterparts. Although SS participants also assessed Wipsea (4) and DriveTrust (4.3) positively, the reasons changed. As for the latter, for instance, one of the students referred to its “clarity”, claiming that “DriveTrust was very practical and accessible, even for me who have no technical background”. Another technology positively assessed by the SS students was Keolis (4.4), the business case provider in charge of public transport in several cities across Europe. Like DriveTrust, which targeted security issues, Keolis was positively assessed on the basis of a potential impact on students’ life. As one SS student stated, “Keolis is about an everyday-life topic, that is to say transportation. Something necessary and not to be left, and something that needs to be improved to have a better ecosystem”. This preference for projects with an evident “social impact” (see [41,42]) was, of course, reflected in the assessment of YoGoKo (4.3), a BC that mainly consisted of explaining how citizens could improve city transport flux by providing data. SS students also positively valued Fabcity: while CS students showed no enthusiasm about this proposal, SS participants felt positively engaged with it, probably because most of the challenges presented were related to everyday topics that matched their academic or professional interests, for example, how the city trash could be collected more efficiently or facing threats like climate change [43].

The assessment analysis also revealed cases in which background was not decisive, apparently because of a “belonging tension” between the “student” (interested in technical details) and the “citizen” (interested in solving a daily problem). Both entrepreneurial projects explaining the potential of IoT technologies for solving energy consumption issues, Wi6Labs and Panga, were assessed just neutrally, maybe because they mostly focused on the technology details rather than its applicability. This technology-focused description could explain why the open data platform Rudi was assessed neutrally by the participants. Rudi, unlike the other BCs, referred to the potentialities of the platform for developing technical solutions, but no direct impact on people’s life was described. An alternative explanation for the low perceived assessment would be its context-dependency (it shared data about the French city of Rennes), which could have prevented international students from feeling more engaged with this case. This belonging tension should be overcome if the purpose is to implement a community-based learning in environmental education [44].

From the pedagogical dimension, it is true that the teacher–student interaction was severely affected; however, the reformulation boosted interaction among students in a different way. Indeed, the use of recorded business cases allowed students with diverse backgrounds to find, within the case portfolio, at least one interesting challenge and propose (and defend) it to the workgroup, which would then decide which business problem would be developed and presented during the pitch day at the end of the event. This discussion activity, carried out during the afternoon of the first day of the summer school, became an argumentative practice that could be further explored by organizers.

Recorded business cases might have been neutrally assessed because of an “organizing tension”. Some participants pointed out disconformity regarding the topics covered by the business cases, arguing that some of them were not related to the overall subject of the summer school, that is, Unleashing the Power of Circular City Data (“A huge difference in the name of the course and what I found in the brochure before enrolling led me to think that the course was focus mainly on creating a link between technologies and policies applying it to cities. The course had a totally different thematic good also, I learned a lot, but it differs from my initial expectations”). Indeed, the decision of including a wide variety of cases, ranging from highly applied technologies to open-source databases, was mainly based on experts’ availability during the pandemic crisis.

A dimension that will have to be further explored is related to video duration. Although our qualitative survey did not present comments on this topic, students’ assessment could have been influenced by this aspect. Indeed, research has shown that students who watch short videos perform better in terms of cognitive achievement [45]. In this sense, another study found that students prefer watching video content of 5–6 min [46].

4.2. Second Decision

From the point of view of didactics, despite participants’ prior knowledge or discipline of origin, it was possible to observe that perception of value depended heavily on their academic background. For example, our analysis showed that CS students claimed that, during the online summer school, their expertise allowed them to assess the “technical specifics” of a given project, while social science students declared that their former training allowed them to have a “general view” of a given innovation project. CS students explained that their previous courses, as well as their professional experience in some cases, helped them to “analyze technical questions”, decide the “feasibility” of a project, apply the “technical expertise to another field”, use “standard techniques and diagrams to understand the market”, and put into practice “design thinking”. SS students, on the other hand, explained that their former academic background helped them to “think about the other stakeholders involved in the projects”, “use soft skills to quickly grasp the unknown contexts” (in relation to hypothetical scenarios), “present and brainstorm in the creation process”, and “work under pressure”.

From the pedagogics dimension, the learning experience was mainly assessed by participants in relation to coaches and classmates, courses in general, and thematic training sessions. As for the former, students assessed professors’ commitment, joy, and homogeneity. As for commitment, for example, one student claimed that “Coaches were awesome. I did not expect that level of involvement in our projects. That was nice”.

Regarding joy, students, probably having the physical version in mind, expected amusing and relaxing activities, as expressed in the following comment: “I thought it wouldn’t be tiring, was supposed to be a lot of fun while studying and doing a project. And the reality is we are going to build up our project from day 1 with intensive sessions every day”. The remark about “intensive sessions” referred to the fact that coaches did not give students the possibility to conduct autonomous work; instead, professors lectured from 9 a.m. until 4 p.m., with only a one-hour lunch break (a situation that could have led to dissatisfaction in relation to sessions’ durability, as reported by [6]). Thus, an effective learning mode should diminish lecture time and increase autonomous tasks.

As for homogeneity, participants, specifically CS students, expected to find teammates with a similar engineering background, as expressed in the following comment: “I did not expect most of the people from my team were from outside EIT. Many of them did not have technical scientific background, so it was difficult for them to participate”. Indeed, when assessing their group work, it was possible to find a “belonging tension” among those with and without technical background, as in the following comments: “Most of the team members did not have a technical background, so it was difficult to communicate some of the ideas” or “I did not expect most of the people from my team were from outside EIT. Many of them did not have technical/scientific background, so it was difficult for them to participate”. Contrarily, research [47] has shown positive results related to online teams with diverse backgrounds (i.e., participants with multidisciplinary and multicultural competences), specifically when participating in a game-based learning activity. The key might have been working in a collaborative game, since, as shown by [48], the effectiveness of a team depends on the generation of interdependence between its members to solve a problem. Thus, a gamification approach, with different team groups each day, may be adopted in future summer schools to leverage students’ different backgrounds.

As for courses in general, CS and SS participants showed positive, although slightly different, kinds of assessments. Some CS students, for example, appreciated new knowledge according to practical purposes, as can be observed in the following statements: “I discovered some new aspects of entrepreneurship that would be useful in the future”, “Personally from the summer school I am taking away lots of technical tips and tools which I could use to start a start-up as successfully as possible”. Thus, students valuated the summer school courses based on an entrepreneurial projection. Unlike computer science participants, SS students appreciated the courses from a novelty perspective, as can be observed in the following statements: “New skills in regard to Business Management and administration” and “It was fun to learn about the steps of developing a business idea”. Regarding the relation between “interest” and “prior knowledge”, research has suggested that students with little knowledge in a certain field were likely to have the highest interest in novelty [35]. A performing tension may be identified from the fact that there were students aiming to learn about new business concepts, while others aimed to become entrepreneurs.

Thematic training sessions, the devices for producing new knowledge, were highly appreciated by participants. The most valued training session, by both types of students, was ‘Definition of a value proposition’, which broadly consisted of presenting the main characteristics of a potential technological solution. Despite the general positive assessment, there were a couple of courses that led to some disagreement: ‘Training for intermediate presentation’ and ‘Public presentation and pitch’. These sessions aimed at providing students with the necessary skills to communicate their entrepreneurial projects. While SS highly appreciated the contents (4.1 and 4, respectively), CS students showed little enthusiasm (3.7 and 3.8, respectively). It is difficult to find a reason to explain this difference among participants, but one possible cause could be related to the proximity that the course had to their expertise, which would suggest the existence of a performing tension in which CS, unlike SS, may be not interested in learning communicative skills and other non-technical aspects.

4.3. Third Decision

In terms of pedagogics mainly, social events tended to be well assessed, as shown in the following comments: “Scape Game was fun, it allowed us to communicate with students from the other teams that we don’t see very often” (since workgroups were fixed). Despite communication being a positive value, students who had attended other summer schools expected to achieve a high level of interaction with their peers, as indicated in the next comment: “it is difficult to achieve the same level of interaction online as in real life”. This performing tension is interesting since it raises the question about its social purpose. Based on our experience as organizers, we argue that summer school social events are not about working in teams or solving a problem collaboratively, but about seeing new places and making social ties and networks [3] to draw on in their second year of their master’s studies. In relation to the possibility of making social links, similar research exploring the effects of moving an onsite summer school to online [3] showed that the creation of an asynchronous virtual poster gallery allowed students not only to share their research, but to interact and comment on other students’ work through Google Docs.

Although socialization activities tended to be positively assessed (normally between 3 and 4), there was one event that was poorly evaluated, with 3.1 and 2.2 from SS and CS participants, respectively. Regarding this result, three possible causes can be found. Firstly, this event took place after a long pitching session that finished over scheduled time. This situation, which, by the way, led to a low evaluation of the course itself, may have predisposed students to assess the social event poorly. Secondly, this social event consisted in a (recorded) virtual tour which required no user interaction. Since students expected to meet new people, the absence of interaction could have played a negative role. And thirdly, the social event required a fast internet connection, which, in the COVID-19 context, was detrimental for most of the students (according to various testimonies).

4.4. Fourth Decision

In terms of didactics, given their academic backgrounds, CS and SS students positively assessed the use of Zoom (as shown by [49]) and Moodle. Most of the participants had experience in working with both Zoom (“I was using Zoom before the Summer School, that’s why I did not have much technical issues”) and Moodle (“I’ve been using Moodle in the university for several years now, so I’m familiar with Moodle”).

From pedagogics, students assessed the selected technologies in multiple ways. Regarding the use of Zoom for lectures, both CS and SS students assessed the platform in positive terms, as they felt confident with the technology. Evaluations, however, changed drastically when it came to “teamwork”. While SS students were quite satisfied with Zoom for working with peers (4.6), CS participants were more critical (3.5), which accounts for an organizing tension. As can be observed in the next comments, one common trigger of dissatisfaction among students was related to the fact that they missed certain interaction elements of the physical version, such as meeting people from other groups (“there was no problem with teamwork and coaching on zoom although we couldn’t interact much with people outside our group”), watching people’s expressions (“of course it is not like real life so unfortunately for the coaches, it was difficult to see our reaction, even us when we present, we cannot see the reactions”), and networking (“although I think in general the whole experience was positive, I was missing some face to face networking and interaction after the sessions”). Most of these reactions may be related to the fact that students turned off their cameras during the training sessions. This is a social phenomenon that has received a lot of academic attention lately, identifying various reasons, for instance, personal appearance, people in the background, poor internet connection [40], anxiety, fear of being exposed, shyness, and desire to maintain privacy [50]. Social interactions play a valuable role in group work, impacting students’ perceptions about the learning experience [34]. Thus, the lack of social interactions in Zoom, at least as expected from presential contexts, would explain why engineering students showed lower levels of satisfaction with respect to the platform use.

From a technical perspective, participants experienced connection problems. Indeed, as a major concern, participants referred to the fact that, once they left a breakout room for some reason, it would not be possible for them to join the session again without the permission of the teacher, which often resulted in long dead times (“Although the only tiny thing is that sometimes when we are assigned to breakout rooms and quit by accident, if the teacher is not in main room then we have to wait a lot to be resigned”). Although described as “tiny”, the problem with Zoom breakout rooms, which occurred because instructors were not familiar with this function, may have negatively impacted student’s e-learning experience. Indeed, technical difficulties in online settings can have a negative effect on learners’ satisfaction [51], which was reflected in the 3.5 score obtained from computer science students.

Regarding Moodle, this platform was generally assessed in positive terms, probably because it turned out to be quite familiar to the students, as shown in the next comment: “I’ve been using Moodle in the university for several years now, so I’m familiar with Moodle”. Moodle performance, it is worth noting, depends on how the different functions have been set up. The Assignment resource, for example, was negatively assessed because of a misalignment among professors when collecting students’ tasks, as can be observed in the comment “I do not really feel that the Moodle was really useful, (…), some teachers preferred that we send the work by mail, but others used the Moodle repository”. This perceived lack of coherence (an organizing tension) in the way professors used the platform features is interesting, as it is an observation that would not be formulated in relation to the presential version. In fact, in the face-to-face version, each professor decides, along with their students, the most convenient method to collect the assignments, which, by the way, can change from one day to another depending on the onsite dynamics. Thus, it can be deduced that whenever there is confusion about what is required by teachers, it is highly likely to lose value [52].

Another organizing tension is related to the fact that most participants were pushed to find other platforms to fill some of the ‘digital gaps’ of the Moodle platform, for example, to exchange and store documents easily (“In our group we used Google Drive which is practical to share our work”). As noticed in the former comment, Google Drive was highly regarded by the participants as the best tool for filling this digital gap, assessed at 4.9 by SS and 4.5 by CS. When asked if they used another communication tool, participants revealed that they used WhatsApp, Telegram, and even Facebook Messenger, which reveals how important it is to maintain social proximity among teammates. The relevance of these tools, specifically Google Drive and WhatsApp, was reported by a study [53] that found that the use of these technologies in teaching–learning settings encourages active learning, increases motivation, and mixes elements of instruction with other interactive elements (e.g., visual and auditory aspects).

Finally, based on our analysis and the tensions found among participants, in

Table 3. Perception of gained and lost value according to managerial decisions.

Decision	Users’ Assessment
	Gain Value	Lost Value
1: Reshaping the business cases	Inclusion of environment-related business case (e.g., related to transport and nature) Inclusion of business cases that were presented or explained clearly enough for those without engineering background	Inclusion of technology-focused cases Inclusion of context-dependent cases Inclusion of cases not related to the summer school topic (circular city data)
2. Offering online training sessions	Participation of engaging coaches Sessions allowing students to apply their computer and social science knowledges	Long and overly intensive training sessions Activities implying teamwork among students with no affinity among them (fixed workgroups)
3. From social events to online socializing activities	Activities not lasting too long (1 h approx.) Opportunity to interact with classmates from other groups (at least in one event)	Time of day was too exhausting (after 4 p.m.) Delayed social event Social event demanding fast internet Social event demanding no interaction (recorded virtual tour)
4. Choosing technology solutions	Use of familiar tools (Moodle and Drive)	Low interaction potential of used apps Use of Zoom’s breakout rooms (difficult to return to an ongoing meeting in case of lost connection) Moodle as submission platform without coaches’ preparation (instructors used email instead)

we synthesize the aspects that triggered a perception of gained and lost value according to the reformulation decisions regarding business cases, training sessions, online events, and technological solutions. Gained value was identified when assessments ranged from “fair” to “good” (3–4), and “good” to “excellent” (4–5). Lost value was identified when assessments ranged from “very poor” to “poor” (1–2), “poor” to “fair” (2–3), and “fair” (3).

5. Conclusions

Our research allowed us to understand the impact of the decisions facing the online shift and identify those aspects that triggered a positive or negative assessment from CS and SS students. We believe that increasing those aspects with a positive valuation and reformulating those with a negative assessment may help organizations remotivate their staff in a new summer school iteration. In the case of positive assessment, our research allowed us to find positive aspects such as considering business cases related to solving environmental challenges, hiring engaging coaches able to integrate students with different backgrounds, organizing short social events, and using well-known platforms. In addition, our research also allowed us to identify “belonging”, “performing”, and “organizing” tensions leading to a perception of lost value, such as working with technology-focused and context-dependent business cases, finding no direct link to the general topic of the event, doing teamwork activities among students with no intellectual affinity, attending pre-recorded, delayed and broadband-dependent social events after training sessions, using Zoom breakout rooms for teaching, and using Moodle as an assignment submission platform (without preparing coaches to do so).

Scholars interested in extending research on the perception of value in the frame of shifted-to-online educational activities, such as summer schools, may take into account, for example, coaches delivering the modules, since they also made important decisions to adapt their work. Another topic that may be considered in further research is the analysis of those aspects leading to lost value and, subsequently, impacting effective learning, for example, teamworking among students with no intellectual affinity. Last but not least, future studies may be also interested in digging into gained values, for instance, how to boost the positive effects of business cases related to environment challenges.

The results of this research may be of interest to those institutions that are currently turning traditionally face-to-face events into hybrid models as palliative alternatives for learning deficit produced after almost one year of massive lockdown.