Continuance Use of Cloud Computing in Higher Education Institutions: A Conceptual Model

Abstract

Resource optimization is a key concern for Higher Education Institutions (HEIs). Cloud Computing, as the recent generation in computing technology of the fourth industrial revolution, has emerged as the main standard of service and resource delivery. As cloud computing has grown into a mature technology and is being rapidly adopted in many HEIs across the world, retaining customers of this innovative technology has become a challenge to the cloud service providers. Current research trends on cloud computing have sought to study the acceptance or adoption of technology; however, little research has been devoted to the continuance use in an organizational setting. To address this gap, this study aims to investigate the antecedents of cloud computing continuance use in HEIs. Hence, drawing on the prior literature in organizational-level continuance, this research established a conceptual model that extends and contextualizes the IS continuance model through the lens of the TOE framework (i.e., technological, organizational, and environmental influences). The results of a pilot study, conducted through a survey with information and communications technology (ICT) decision makers, and based on the proposed conceptual model, indicate that the instrument is both reliable and valid, and so point the way towards further research. The paper closes with a discussion of the research limitations, contribution, and future directions.

1. Introduction

Cloud computing (CC) is increasingly becoming a springboard for digital innovation and organizational agility. Higher education institutions (HEIs) are facing the problems with the increasing of participants, growing need of IT and infrastructure, education quality of provision, and affordable education services [1,2]. With the high rate at which IT technology changes, resource management optimization is a key concern for HEIs [3], not least because on-premise systems can only operate effectively when they receive adequate initial funding and resources, as well as dedicated and systematic maintenance regimes [4,5]. Institutions looking to compete in the new world need a flexible yet comprehensive digital transformation blueprint that integrates various technologies across the institution with CC being at its foundation. CC, as the current generation in computing technology of the fourth industrial revolution (IR 4.0), has emerged as the main standard of service and resource delivery [6], in which it has become an excellent alternative for HLIs to support cost reduction, quality improvement and, through this, educational sustainability [7] by providing the required infrastructure, software, and storage as a service [3]. Thus, CC has been adopted rapidly in both private and public organizations, including HEIs [3,8,9]. As the fifth most frequently used utility after gas, electricity, water, and telephone lines [10], the CC tracking poll from the International Data Corporation indicates that USD 370bn in CC will be used by 2022, corresponding to an increase in 22.5% in terms of 5-year compound annual growth [11].

However, while the subscription model of cloud services contributes to the growth of the overall market and makes it accessible for HEIs, a new set of challenges have arisen. This research addresses one such challenge that the cloud service providers face. The possibility of making such a decision to discontinue a cloud service provider is exacerbated by the low cost of switching between applications [12] and in general the competitive markets [4]. Therefore, the conceptualization of CC service in HEIs changes to a decision on ‘continuance’, rather than ‘adoption’.

Moreover, in subscription models offered via cloud-based education systems, it is possible for HEIs to switch vendors if they perceive greater benefits elsewhere. Thus, it is essential to understand the conceptual differences between adoption and continuance [13]. Hence, research on CC continuance have practical and artifact-specific motivations. Furthermore, theoretical research on organizational-level continuance is also scarce [14], particularly in HEIs [9,15]. Typically, continuance research has been undertaken at the individual user level; however, organizational continuance decisions are often made by senior IS executives or others in the organization who may not be intense users of the service in question [14]. For many of these executive decision makers, a strong influence may be attributed to factors that are insignificant for individual users (e.g., lowering organizational costs or shifting a strategic goal) [16].

Thus, to contribute to the body of knowledge on the organizational-level continuance, this study draws on the prior literature in organizational-level continuance to establish a conceptual model that extends and contextualizes the IS continuance model to improve our understanding of the determinants of CC continuance use in HEIs through the lens of the TOE framework (i.e., environmental, organizational, and technological influences). Following the establishment of the conceptual model [15], the model was validated by conducting a pilot study with senior decision makers, all of whom were asked about aspects of their organizations relating to CC services. Therefore, this study sought to address was the following: “What constructs influence the organizational-level continuance of CC in HEIs?” To address this question, our research relies on a positivist quantitative-empirical research design. In this research, the unit of analysis (i.e., a basic element of observation indicating who or what the researcher has generalized) [17] is the organization, and the organization-level phenomenon will be observed by individuals involved in organizational CC subscription decisions at an organization [17].

In general, we contribute to research in different ways. First, the most important contribution of this research constitutes to the body of knowledge within the IS field surrounding continuance phenomenon. In practical settings, many are concerned with reducing capital IT expenses [3,18,19] and IS services allow client organizations to select from various services that they can continue using or discontinue using [8,9]. Therefore, conducting research that focuses on the continuance of IS will play a critical role in theory and practice. Second, we develop a conceptual model that provides a clear perspective through which HEIs can answer the question related to their use of CC services: “should we go, or should we stay?” Third, the results of the full-scale research will assist IT decision makers in CC when seeking to optimize institutional resource utilization, or to commission and market CC projects. As a case in point, the results can serve as guidelines that cloud service providers will use to focus their efforts towards retaining customers. These can also be leveraged by clients to guide routine assessments over whether the use of a specific CC service should be discontinued. Fourth, the study is expected to contribute to developing the literature in the best available organizational-level continuance models for HEI settings. Last, in providing a model for CC continuance use, we provide a new explanation for organizations’ continuance use of novel technologies. Measuring the model constructs not only reflectively but also formatively would add little to the practical contribution of the study. Thus, further quantitative, and qualitative research about the conceptualized model and its relationships is needed.

The structure of the rest of this study is as follows, a literature review is given in the next section. Theoretical models are then identified and analyzed, a conceptual model for exploring continuance of CC in HEIs is proposed. In turn, the method is explained, and the preliminary results of the study are presented. Finally, the study’s results are discussed, their implications are examined, and the contributions of the research are outlined.

2. Background and Related Work

As a term, CC is defined diversely in the literature. In this paper, the definition of CC used is comparable to NISTs [6], which regards CC as the set of aspects that are common across all CC services. Hence, from the perspective of this paper, CC relates to the applications and shared services involved in the surveyed institutions through subscription-based models, whereby shared data servers or application activities are accessed.

In HEIs, CC has been identified as a transformative technological development [3]. This is because CC benefits from rapid IT implementation, especially for research, which compares favorably when considered against legacy software systems. Additionally, CC solutions can be exploited to assist in implementing socially oriented theories of learning, as well as cooperative learning [20]. CC resources can be used to create e-learning platforms, infrastructure, and educational services through the centralized provision of data storage, virtualization, and other facilities [21]. With these considerations in mind, CC, for certain HEIs, is essential, and many institutions rely on the technology to reduce costs, remain competitive, and satisfy learner and teacher requirements [22]. The accessibility and transparency of CC services mean that HEIs can utilize existing knowledge to their mutual benefit [23].

To examine CC use in HEIs, a systematic literature review (SLR) was undertaken. The following electronic databases were included in the literature search: Web of Science, IEEE Xplore, ScienceDirect, Scopus, ACM Digital Library, Emerald, and Springer [9]. Additionally, the following search terms were entered into each of these databases: (cloud OR “cloud computing”) AND (adoption OR usage) AND (education OR teaching OR learning)). The literature search revealed that many studies had been published in this area, and that the rate of publication had been increasing for the past few years. IS researchers have tended to investigate CC use in HEIs from the perspective of individuals [7,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49] or from the perspective of organizations [50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65].

For the last three decades, many researchers have sought to evaluate the success of information systems (IS). Theoretical and practical contributions have been discussed, many of which indicate that the factors underpinning IS success are multidimensional. Furthermore, several studies have contributed to CC continuance use in highly different settings [14,66,67,68,69,70,71,72,73,74,75,76]. Nonetheless, the factors that drive institutions to continue or end a CC subscription have yet to be clarified [14,16,66], specifically in HEIs [9]. Given that almost all CC service models rely on subscriptions [6], this is an unexpected finding. Therefore, the purpose of the current research was to assess the main factors that HEIs consider when deciding whether to continue their use of CC services. In

Table 1. Literature on CC Continuance.

	Level of Analysis		Adoption Phase		Theoretical Perspective					Type
	IND	ORG	PRE	POST	ISC	ISS	ISD	TOE	OTH	EMP	THEO
[14]		√		√		√	√			√
[72]		√		√	√			√		√
[75]		√		√					√	√
[76] *		√	√	√	√				√	√
[71] **	√	√		√					√	√
[77]		√		√		√				√
[45] *	√		√	√						√
[78]		√		√		√	√			√
[79]		√		√		√		√		√
[70]		√		√	√						√
[80]	√			√	√	√				√
[49]	√			√	√					√
[14]		√		√		√	√			√
SUM	4	10	2	13	5	6	3	2	3	12	1
This Research		√		√	√	√		√

Legend: IND = Individual; ORGA = Organizational; PRE = Pre-Adoption; POST = Post-Adoption; ISC = Information System Continuance; ISS = IS Success Model; ISD = IS Discontinuance Model; TOE = Technology–Organization–Environment Framework; OTH = Others; THEO = Theoretical/Conceptual; EMP = Empirical. * Study examines adopters’ and non-adopters’ intention to increase the level of sourcing; thus, it is categorized as adoption. ** Study examines adopters’ intention at individual and organizational levels; thus, it is categorized as an individual.

, evidence pertaining to the continuance use of CC is synthesized.

Life Cycle of an Information System

This study belongs to the well-established stream of literature that has examined the phenomenon of “technology adoption”, which was initially defined by Rogers [81] as a five-step process. Since then, a range of models has been developed to extend Rogers’ preliminary work [82,83,84]. According to some researchers, adoption is a multi-phase process rather than a binary decision [85,86,87,88], and for some theoreticians, adoption occurs over seven rather than five stages [89,90]. However, most researchers agree that technology adoption operates across the following five stages: awareness, interest, evaluation, trial, and continuance. Several researchers advocated a four-phase model, which involved initiation, adoption, decision, and implementation (e.g., [81,91,92]). However, certain studies have concentrated their empirical understanding of technology adoption into a single phase (e.g., adoption or pre-/post-adoption) [14]. This perspective is consistent with the wide-ranging stages of technology adoption that have been investigated in previous studies [93,94,95] (see Figure 1). Certain studies have reported that, in terms of the methods that are available in technology adoption research, many are limited because they do not differentiate between changes in the significance of factors in the different phases of adoption [96]. Therefore, opportunities, as well as suitable research settings for exploring continuance as the last phase of adoption, have been limited.

In Table, previous studies have adapted various IS theories and empirically analyzed CC in different contexts from an individual or organizational viewpoint in the pre-adoption or post-adoption (i.e. continuance use) phase. However, no empirical study was found measuring the continuance use of CC in HEIs. Therefore, the main contribution of this study is to develop an instrument and conceptualize a model to measure the continuance use of CC in the context of HEIs. Beside this context-specific contribution, our study also reduces the gap of related to organizational IS continuance research.

A range of theories have been used to study adoption (e.g., UTAT or TAM), continuance (e.g., ISC), and discontinuance at the individual level. Contrastingly, scholars have used theories such as TOE, DOI, and social contagion to examine adoption from an organizational perspective. Additionally, the ISS and ISD models have been leveraged to investigate continuance and discontinuance respectively, at the organizational level.

Table 2. Life Cycle of An IS with Different Theoretical Approaches.

Life Cycle Phases	Adoption	Usage	Termination
User/organization Transformation	Intent to adopt	Continuance usage intention	Discontinuance usage intention
End-user state	No user	User	Ex-user
Individual Level-based theories	TAM [98], and UTAT model [99]	ECT [100], which has taken shape in the ISC model [101]
Organizational Level-based theories	TOE framework [102], DOI [103], and Social Contagion [104]	ISS model [105]	ISD model [16]

Legend: IS = Information System, TAM = User Acceptance Model, UTAT = Unified Theory of Acceptance and Use of Technology, ECT = Expectation Confirmation Theory, ISC = Information System Continuance model, ISS = Information System Success model, ISD = Information System Discontinuance model, TOE = Technology–Environment–Organization Framework, DOI = Diffusion of Innovation theory, ITMAP = Information Technology Post Adoption Model.

provides an overview of the various theoretical approaches that have been used in the literature to examine the lifecycle of an IS. Dissimilar to studies that have focused on the individual level, those that have addressed continuance and discontinuance at the organizational level are few and far between [9,14,16,97].

After initially adopting an IS, a user decides to either continue or discontinue the IS adoption. Contrastingly, this is unlikely to retire or replace their on-premise CC services [106]. Nevertheless, since almost all CC services involve a subscription model, and since this study’s focal point is the issue of continuance, the aim of this study is not to examine factors that influence the use of CC services in HEIs. Instead, the main area of focus in the current study is the set of constructs that contribute to IS continuance. An implication of this is that it is possible to evaluate success and system performance, which is dissimilar to the pre-adoption phase in which only expectations can be used to estimate usage. This also enables the integration of post-adoption variables as predictors of IS adoption continuance, thereby exerting far-reaching impacts on model development.

3. Theoretical and Conceptual Background

In this section, we present the theoretical and conceptual background, focusing on the prior literature in organizational-level continuance to extend and contextualize the IS continuance model to improve our understanding of the determinants of CC continuance use in HEIs.

3.1. IS Continuance Model

The IS Continuance (ISC) model [101] was developed based on expectation confirmation theory (ECT) [100]. The model has been used widely in the field of marketing to examine the impact of user satisfaction on a user’s intention to continue their adoption of a technology [107,108]. As Figure 2 indicates, IS continuance behavior in the ISC model is informed by post-consumption variables, namely, the perceived usefulness and satisfaction. Bhattacherjee [101] applied several theoretical changes to fine-tune ECT theory to the ISC model.

The first of these changes relates to the pre-consumption antecedents of confirmation, namely perceived performance, and expectation. Specifically, both antecedents were removed from the model, and the researcher’s rationale for doing so was that their influences are addressed in other constructs (specifically, confirmation and satisfaction). The second change relates to the addition of an ex-post expectation variable, namely perceived usefulness. Noteworthily, ex-post expectation is critical in IS services and products, principally because expectations tend not to remain stable over time. Consistent with previous studies on initial IS use [98,109], the study conducted by Bhattacherjee [101] reported that perceived usefulness may continuously influence subsequence IS continuance use decisions. Resultantly, perceived usefulness was considered a novel determinant of satisfaction. The third change is that, regarding the usefulness–intention connection originally developed in TAM [98], the ISC model suggests that this may exist not only in the original use context but also in the continuance context. This is linked to the fact that continuance intention in humans can be attributed to a sequence of adoption decisions that are not related to factors such as timing or the behavioral stage [110]. Hence, perceived usefulness should have a direct impact on IS continuance intention to have an indirect effect on IS continuance intention via satisfaction. In the literature, the ISC model has primarily been employed to examine continuance use from an individual perspective. However, the model has been extended for organizational post-adoption studies (e.g., [72]); consequently, it is associated with a substantial level of external validity. Besides, the ISC model has been used to explain the continuance phenomenon in the education context (e.g., [111,112,113]).

The focal point of the ISC model is an individual’s continued acceptance of technology, whereas the purpose of this study is to address organizational continuance use. However, several researchers have extended the model for organizational post-adoption context (e.g., [72]). As suggested by the TOE model, a critical point of difference between organizational and individual continuance use settings is that the technology adoption decisions made by organizations are typically informed both by technology factors associated with individual beliefs (e.g., satisfaction) and by organizational factors such as external threats and opportunities. Therefore, to fine-tune ISC model to address the research problem, it is necessary to supplement factors in the organizational continuance context, especially those relating to organizational and environmental settings [72]. Given that researchers need to choose a theory or model that is appropriate for the research setting (in this case, continuance use at the organizational level), this study substitutes the perceived usefulness construct of the ISC model with logical reasoning. Perceived usefulness is generally considered the most relevant technological factor that informs IS post-adoption behavior in the ISC model, and a range of studies, including [72,114], have employed it as a baseline model. Nevertheless, the theory of planned behavior (TPB) [115] suggests that net benefits ought to be viewed as behavioral belief (that is, perceived usefulness) [116]. For this reason, the net benefit construct taken from the IS success model is used instead of perceived usefulness, thereby achieving an effective fit with the research setting.

3.2. IS Success Model

According to Gable, Sedera [67], the positive outcomes arising from IS are the ultimate “acid test”, and so the question that should be addressed is that of whether IS has been beneficial for the organization [117,118,119]. Other questions of interest are whether IS is worth retaining, whether it requires modification, and the impacts that it will deliver in the future. Hence, a continuance decision relating to IS can be regarded as something that is informed by IS success at several levels [14].

Numerous studies have been conducted on IS success (ISS), and the ISS model [105], as well as a revised version of this model [120], have been used extensively in the literature to explore the issue [121]. The use of ISS model [105] in this study stems from the following considerations: firstly, the model has been used in various research settings [14,67,72,122,123]; secondly, it is easy to communicate the results of the model due to the systematic nature of the included dimensions; and thirdly, the model is not narrowly applied as a framework for measuring success, and so it has a high level of external validity.

The success dimensions chosen by a researcher must be based on the research setting and research problem; therefore, in this study, two dimensions are removed based on logical reasoning: namely, use and service quality. In terms of the use construct, it has been critiqued in the literature for several reasons [67,124,125,126,127], particularly in terms of the way it performs an intermediate function that lies between quality and impacts, and as such does not operate as a measure of success [128]. Additionally, the adoption of system use construct of IS success has been identified as unsuitable in previous studies of system performance [14]. Regarding the service quality construct, this was removed for the following reasons: firstly, it is comparable to the complete idea of IS success in several ways, where system and quality constitute a “functional” aspect, while the effects constitute the “technical” aspects in the “operational” IS (in this case, the system is considered a set of services); and secondly, in order to assess the service provider’s services, a narrow perspective of service quality is involved. In CC service research, an assessment of this kind would be an antecedent rather than a measure. Given these considerations, several constructs, namely information quality, system quality, and net benefits, are integrated into the conceptual model of this study.

3.3. IS Discontinuance Model

Additional factors that influence organization persistence, particularly in the context of CC use in HEIs, were identified in this study for an explanation of CC continuance use in the organization-level IS post-adoption context. This resulted in the inclusion of system investment as an organizational construct, as well as technical integration as a technological construct [16]. Each of these constructs relates to the commitment that an organization has to a subscription-based technology (e.g., CC) [14].

Regarding technical integration, this study sought to examine whether the features of a technology impacted continuance decisions. Specifically, in view of the principal objective of CC, the study investigated the degree to which HEIs would benefit from the sophistication provided by the features of CC services. However, the positive impacts of the features of the technology are not assured in many cases. For example, certain organizations may not have the expertise needed to exploit the complexity of the technology [129,130,131], meaning that they cannot integrate it into the overall functioning of the firm. As a result of this, the organization would be compelled to discontinue their use of CC services [14,78].

In terms of the system investment variable, this—as a source of behavioral persistence—has often been referred to as a “sunk cost” in the literature [132]. Among managerial personnel, it is common to invest continually in an area despite reasonable evidence for not doing so. The notion of a sunk cost becomes relevant when the cost of acquisition can be regarded as a capital expenditure (CapX). System investment studies have assessed the role played by CapX in the formulation of computer software prices in the context of switching between software solutions [133], as well as the impact on succeeding decisions in terms of IS outsourcing [134]. In the case of CC, system investment is a noteworthy variable because of the technology’s low barriers to entry and minimal overhead costs [135]. In view of this, it is not unreasonable to view CC services as any other utility that can be turned on and off at will [135,136]. However, it is critical to recognize that many CC services are typically associated with significant implementation costs. A key implication of this is that system investment is fundamental in CC continuance in HEIs.

Regarding competitive pressure, this refers to the pressure that an institution’s leadership may feel regarding the performance-related abilities that its competitors are gaining through the exploitation of CC services (e.g., an increase in student assessment outcomes due to the use of CC platforms) [72,137,138].

3.4. TOE Framework

In the context of the TOE framework [102], it is possible to divide the constructs determining behavior related to CC continuance into the following contextual areas: firstly, organizational context; secondly, technology context; and finally, environmental context. However, it is notable that the framework itself does not include information about these constructs. In terms of the effect of the technology context on CC adoption behavior, this refers to the set of technology-related factors that feed into an organization’s decision to adopt an innovative IS [139]. As for the organizational context, this is concerned with the way in which various factors affect IS adoption behavior. These factors include available resources, opportunities for collaboration, profile characteristics, peer influence, internal communication, organizational culture, formal and informal linking structures, human resources quality, firm size and scope, and the internal social network. Finally, the environmental context demonstrates that an organization’s IS adoption is significantly impacted by constructs that lie outside of its direct control (e.g., competitors, government regulations, and supply chains) [102]. In view of these considerations, it is clear that the TOE framework can play an effective role in identifying non-technology-level factors that have not been considered in other studies on consumer software (e.g., constructs relevant to external circumstances) [140]. Additionally, the TOE framework helpfully interprets the notion of adoption behavior based on the following technological innovations: firstly, innovations applied for technical tasks (i.e., type 1 innovations); secondly, innovations relating to the business administration (i.e., type 2 innovations); and thirdly, innovations integrated into core business procedures (i.e., type 3 innovations) [141].

Along with the technological and organizational variables of continuance use, which were derived from models of IS continuance, discontinuance, and success, constructs were identified that affect organizational and environmental persistence, especially insofar as they relate to CC in HEIs. As a result of this process, collaboration was identified as an organizational variable [42,142,143,144], while regulatory policy [145,146,147] and competitive pressure [72,145] were identified as environmental variables. Collaboration tasks lie at the heart of HEIs, and collaboration can be conceptualized as the ability of CC services to facilitate communication among stakeholders [42,142]. In the case of digital natives, CC services play a vital role in effective collaboration [148,149].

Table 3. Mapping matrix of model constructs from ISC, ISS, ISD, and TOE.

Theory/Model	Technology/Dependent Variable	Source	Constructs/Independent Variables
			Satisfaction	Confirmation	Technology				Organization		Environment
					Net Benefits	Technology Integration	System Quality	Information Quality	System Integration	Collaboration	Regulatory Policy	Competitive Pressures
ISD	Organizational level information System discontinuance intentions.	[16]				√			√			√
ISC	Information system continuance.	[101]	√	√
ISS	Information system success.	[105,120]			√		√	√
ECM & TOE	Enterprise 2.0 post-adoption.	[72]	√	√								√
TAM	Continuance intention to use CC.	[75]
ISC & OTH	Disruptive technology continuous adoption intentions.	[76]	√									√
ISS	CC evaluation	[77]	√		√		√
ISS & ISD	Cloud-Based Enterprise Systems.	[78]			√	√	√	√	√
ISC	SaaS-based collaboration tools.	[45]	√	√
ISC	CC client-provider relationship.	[70]	√	√
ISC	Operational Cloud Enterprise System.	[152]			√		√	√
OTH	Usage and adoption of CC. by SMEs	[143]								√
TOE	Knowledge management systems diffusion	[153]									√
TCT	Information technology adoption behavior life cycle	[154]	√	√
ISC	Wearable Continuance	[155]	√	√	√

Legend: TOE = Technology–Organization–Environment Framework; ISC = Information System Continuance Model; ISS = IS Success Model; ISD = IS Discontinuance Model; TAM = Technology Acceptance Model; TCT = Technology Continuance Theory; OTH = Others.

presents a mapping matrix for the continuance use constructs and theories, each of which has been obtained from the extant and related literature [150,151].

4. Research Model and Hypotheses

A robust theory of organizational level continuance of CC is yet to be developed [14]. Therefore, based on the theoretical and conceptual background outlined previously, this research used a method that complements and contextualize existing constructs in the IS continuance model through the lens of the TOE framework. We extended the IS continuance model [101] using constructs from dominants models in innovation organization-level IS post-adoption research which are IS success model [105,120] (i.e., net benefits, system quality, and information quality) and IS discontinuance model [16] (i.e., technical integration, system investment, and competitive pressure). To keep our research model coherent and relevant, we identified additional contextual constructs from the literature as constructs to predict continuance use of CC in educational context (i.e., collaboration and regulatory policy). To structure our model, we took a technological–organizational–environmental approach by applying the lens of the TOE framework [102] to our research model (i.e., Technology context: net benefits, system quality, information quality, and technical integration; Organizational context: system investment, and collaboration; and Environmental context: regulatory policy, and competitive pressure). We also formulated related hypotheses to clarify our research agenda, emphasize research areas that need further investigation, and acquire requisite knowledge on CC continuance use. Figure 3 provides an overview of the original IS continuance model and this study’s proposed extensions. The model is grounded at the organizational level of analysis [156], and the smallest unit of analysis is an individual CC.

Based on a positivist, deterministic philosophical paradigm, a priori assumptions in the form of hypotheses were established for later statistical analysis to facilitate model validation. The propositions focus on the link between the independent variables encompassing the IS continuance model, IS success model, IS discontinuance model, and TOE framework, and the dependent variable, namely CC continuance use.

The relationships among perceived usefulness and confirmation, continuance intention, and satisfaction, as noted by Bhattacherjee [101] in the context of system acceptance, are relevant for investigating CC continuance in HEIs. In this study, perceived usefulness was substituted by net benefits, which is a cognitive belief relevant to IS use [157]. In the context of TAM, perceived usefulness is considered a user’s belief towards system usefulness [98]. Whereas in the organizational context, net benefit is considered a belief about the degree to which IS promotes organizational objectives. This definition is aligned with other organizational-level definitions [157,158]. We thus propose the following hypotheses:

Hypothesis 1 (H1). 

An institution’s satisfaction level with initial CC adoption positively influences its CC continuance use.

Hypothesis 2a (H2a). 

An institution’s extent of confirmation positively influences its satisfaction with CC use.

Hypothesis 2b (H2b). 

An institution’s net benefits from CC use positively influence its satisfaction with CC use.

Hypothesis 3a (H3a). 

An institution’s net benefits from CC use positively influence its CC continuance use.

Hypothesis 3b (H3b). 

An institution’s extent of confirmation positively influences its net benefits from CC use.

The relationships among system quality, information quality, and continuance intention [120] in the context of IS success can also be applied to CC continuance use in HEIs. Prior studies have examined the relationships among system quality, information quality, and satisfaction [77,159,160,161,162,163,164,165]. Hence, it follows:

Hypothesis 4a (H4a). 

System quality positively influences an institution’s satisfaction with CC use.

Hypothesis 4b (H4b). 

System quality positively influences an institution’s CC continuance use.

Hypothesis 5a (H5a). 

Information quality positively influences an institution’s satisfaction with CC use.

Hypothesis 5b (H5b). 

Information quality positively influences an institution’s CC continuance use.

The relationships among technical integration, system investment, and discontinuance intention [16] can also be applied to CC continuance use in HEIs. Thus, we propose:

Hypothesis 6 (H6). 

Technical integration positively influences an institution’s CC continuance use.

Hypothesis 7 (H7). 

System investment positively influences an institution’s CC continuance use.

Presently, the success of HEIs depends in large part on effective collaboration. This is noteworthy because, by leveraging CC, it is possible for HEIs to exploit new modes of communication between key stakeholders [42,142]. Digital natives, many of whom are students within HEIs, now require the Internet to undertake daily tasks [148,149], and also to participate in online group activities (e.g., socializing, group studying, and so on) [166]. In order to satisfy student requirements, it is necessary to practitioners within HEIs to understand the various ways in which knowledge and content can be delivered to them [167]. In view of this, it is important to know what types of expectations students have, and to understand how technology can be leveraged or incorporated into teaching activities to meet these expectations. Hence, it is not unreasonable to suggest that the competitiveness of a HEI depends on its utilization of novel technology to satisfy student needs, and to enable streamlined collaboration and communication [168]. Taking the context into account, we thus predict:

Hypothesis 8 (H8). 

The collaboration characteristics of CC services positively influence an institution’s CC continuance use.

Regulatory policy is another critical consideration that is likely to affect an organization’s decision to use, or to continue using, a technology. One of the reasons for this is because the regulatory policies established by a government play a key role in setting laws relating to the use of certain technologies (e.g., CC) [147,169,170]. For example, the authors of [145,146,147] discussed how regulatory policies have shaped adoption trends in CC in various research settings. Taking the context into account, we therefore hypothesized:

Hypothesis 9 (H9). 

Regulatory policy positively influences an institution’s CC continuance use.

Competitive pressure refers to the pressure that an institution’s leadership may feel regarding the performance-related abilities that its competitors are gaining through the exploitation of CC services (e.g., an increase in student assessment outcomes due to the use of CC platforms) [72,137,138]. In the literature, several scholars have noted that competitive pressure plays a determining role in influencing CC use in multiple research settings [72,153,170,171,172,173]. Taking the context into account, we thus predict:

Hypothesis 10 (H10). 

Competitive pressure positively impacts an institution’s CC continuance use.

The research model will be used to examine CC continuance use at the organizational level in HEIs. Nevertheless, institutions are a key element of the CC ecosystem, in which diverse sets of actors are involved (e.g., government agencies, public organizations, and researchers). The proposed model can be referenced by CC actors in HEIs as a basis for cooperating with stakeholders, which is a prerequisite for the creation and provision of improved products and services.

5. Methodology

A positivism quantitative survey approach is warranted to address the research objectives illustrated through the research questions and the hypotheses. Instrument development, data collection and data analysis mechanisms are discussed in detail below.

5.1. Research Design

This research relies on the positivist philosophical paradigm, as well as the collection and analysis of quantitative data. The rationale for this decision stems from the way in which the approach permits a cost-effective and timely research process [174]. Furthermore, a natural way to address the study’s research questions and test the hypotheses involved using a quantitative survey, since this yielded a direct approach to comparing dependent and independent variables. According to Creswell and Creswell [174], describing causal relationships between variables is only possible when a non-experimental correlation research design with quantitative data is utilized.

Since this study relies on the theoretical foundation of continuance, established guidelines were followed to develop the research instrument (i.e., item formulation, scale development, and instrument testing) [175,176]. Following the model development, a pool of survey items was derived from the literature; then, content validity was tested by examining the extent to which every item reflected its nominated construct. Additionally, consistent with recommendations reported by Kelley [177] and McKenzie, Wood [178], the expert review evaluation process was undertaken to establish measurement representativeness, clarity, and comprehensiveness. Afterwards, pilot test was undertaken to assess the validity and reliability of the research instrument. Instrument development and data collection were the two main methodological activities undertaken in this research, and these will be described later in this manuscript.

5.2. Instrument Development

Survey research is an essential and complex process used to ensure research objectives are met [179]. Therefore, designing and selecting the correct instrument for survey is fundamental as it should be answering the research questions on what is to be measured and how it is to be measured—in this case, the construct validity and construct reliability, respectively [179,180].

In this study, both reflective and formative measures were used to test the research model, as shown in

Table 4. Constructs and definitions.

Constructs	Definition	Literature Sources	Previous Studies
Net Benefits (Formative)	Extent to which an information system benefits individuals, groups, or organizations.	[105,116,120]	[14,77,78,152]
System Quality (Formative)	Desirable features of a system (e.g., reliability, timeliness, or ease of use).	[105,116,120]	[14,77,78,152]
Information Quality (Formative)	Desirable features of a system’s output (e.g., format, relevance, or completeness).	[105,116,120]	[14,77,78,152]
Confirmation (Reflective)	Extent to which a user in a HEI feels satisfied when the outcomes are consistent with (or exceed) their expectations or desires, or when the outcomes are inconsistent with or below their expectations or desires.	[101,189,190]	[45,72]
Satisfaction (Reflective)	Psychological state that results when the emotion linked to disconfirmed expectations is paired with the user’s previous attitudes towards the consumption experience.	[101,191]	[45,72,76]
Technical Integration (Reflective)	Extent to which an information system depends on intricate connections with different technological elements.	[16,192]	[14,78]
System Investment (Reflective)	Resources, both financial and otherwise, that the institution has applied to acquire, implement, and use an information system.	[16,193,194]	[14,78]
Collaboration (Reflective)	Extent to which CC application supports cooperation and collaboration among stakeholders.	[195,196]	[42,142,143,144]
Regulatory Policy (Reflective)	Extent to which government policy supports, pressures, or protects the continued use of CC applications.	[147,169,170]	[145,146]
Competitive Pressure (Reflective)	Pressure perceived by institutional leadership that industry rivals may have won a significant competitive advantage using CC applications.	[170,171,197]	[72,145]
Continuance Intention (Reflective)	Extent to which organizational decision makers are likely to continue using an information system.	[16,101]	[45,72,76]

. Formative measurements were taken of net benefits, information quality, and system quality, mainly because formative measurement gives rise to actionable and specific concept attributes [181], that is specifically interesting from a practical viewpoint. In the context of formative measurements, a single indicator’s weight is used to draw practical insights about the criticality of certain details, thereby generating information that guides practical enforcement in terms of the system characteristics (e.g., “overall system quality is high” (reflective) vs. “system is easy to use” (formative)). Dissimilar to the formative constructs (e.g., system quality, net benefits, and information quality), the purpose of which is to assess an information system’s success, it is possible for the reflective constructs to be given historically. Hence, the measurement of these constructs involved well-validated reflective scales [16]. In the case of the formative instrument, this was developed based on the guidelines of Moore and Benbasat [175]), which was combined with recent processes in scale development [182,183,184]. For the formative measures, the objective was to achieve mutual exclusivity and parsimony, and to identify one measure that would be the most appropriate for inclusion in the model. As a case in point, parsimony and accuracy are critical considerations for all measures in a formative model, particularly since every dimension and measure is essential. Consequently, there should only be a small level of overlap, and no unnecessary measures or dimensions should be present. Such attention is considered vital in selecting the tentative measures.

It is worth drawing attention to the fact that the questionnaire scales were adapted from the prior literature (i.e., from well-validated studies on the IS continuance model [101], IS success model [105,120], IS discontinuance model [16], and TOE framework [102]) (See Appendix A). The instrument’s feasibility, consistency of style and formatting, readability, and linguistic clarity [185,186], were evaluated in interviews with academic researchers (n = 2) with experience in questionnaire design. Their feedback on the general design and measurement scales were requested for improving the usability of the questionnaire. A content-based literature review approach recommended by Webster and Watson [187]) was used for instrument conceptualization and content specification, in which constructs have been clearly defined (See Table 3). The next stage involved producing an item pool, the purpose of which was to represent every aspect of the construct without overlap [183]. Notably, the elimination of a measure from a formative indicator model risks leaving out a relevant part of the conceptual domain (or, for that matter, changing a construct’s meaning). This is because the construct is the set of all indicators [188], and also because maintaining irrelevant items will not have the effect of introducing bias into the results when examining the data with PLS [181]. In view of this, every dimension that was identified was retained and changed into an item.

The questionnaire comprises three parts: firstly, a preamble; secondly, a demographic section; and finally, a section on the constructs relating to continuance use of CC in HEIs. For the first section, we applied the key informant approach [198] in which two questions were used to eliminate participants from the sample: first, checking for participants whose institutions had not yet adopted CC services; and second, checking for participants who do not participate in the ICT adoption decision. Participants eligible for inclusion in the study, they can complete the next two sections of the questionnaire. In the second section, demographic data about each institution’s age, faculty, student population, years of CC service adoption, and type of CC service model were gathered. A service provider variable was also measured based on asking the respondent who their CC service provider was (e.g., Oracle, Microsoft, Google, Salesforce, and Amazon, among others). Notably, the service providers did not affect the final dependent variable. In the third section of the questionnaire, each item sought to address an aspect of the research question, particularly measuring the information of constructs that led towards an organizational continuance use. These included satisfaction (S), confirmation (Con), net benefits (NB), technical integration (TE), system quality (SQ), information quality (IQ), system investment (SI), collaboration (Col), regulatory policy (RP), and competitive pressure (CP). As shown in Appendix A, a 5-point Likert scale, ranging from “strongly agree” to “strongly disagree”, was used to measure each item. Additionally, SAT items were measured on a 5-point Likert scale with different options (e.g., 1 = very dissatisfied to 5 = very satisfied; 1 = very displeased to 5 = very pleased; 1 = very frustrated to 5 = very contented; and 1 = absolutely terrible to 5 = absolutely delighted).

Further, the instrument development process took into consideration the debate surrounding the practice of gathering perceptual data on both the dependent and independent variables from a single respondent [199]. In this debate, a central issue is that of whether the practice may result in excessive common method variance (CMV). Nevertheless, some studies indicate that CMV is a greater problem for abstract constructs (e.g., attitude) when compared to concrete measures (e.g., those linked to IS success in this research) [200]. Besides, it has been noted in the literature that the constructs of IS success are not highly susceptible to CMV [201]. Moreover, CMV is not a major concern for formative constructs because the items do not have to co-vary [14]. Furthermore, in the process of operationalizing the research instrument, CMV can be further reduced by neglecting to group the items from reflective constructs under the associated construct headings [199,200].

5.3. Data Collection

A continuance decision within an organization must be reached unanimously, and so the choice made in this research to use an individual as a representative of an organization (even inside a team) is reasonable [14]. Given the adopted survey methodology, individuals reported on organizational properties. Hence, ensuring that each participant had the requisite authority and knowledge to contribute data was critical. Therefore, the key informant approach [198] was utilized in this research. In the introductory section of the questionnaire, participants were informed that the study only sought to recruit key decision makers within organizations. Furthermore, the participants were asked directly to withdraw from the study if they were not directly involved in their institution’s decision to continue using CC services.

For data analysis, descriptive statistics were used to examine data from the first and second sections of the questionnaire. The survey respondents had been recruited through online and offline distribution channels to reach participants with the questionnaire. An invitation from the institutional email with reminders where sent online to 50 people, 34 completed and submitted the questionnaire with a response rate of 68%. Of the 12 surveys handed over face to face, only four surveys were completed and returned successfully with a response rate of 33.3%. This indicates that face to face method consumes more time and effort than the online approach. For most scholars, a pilot study sample size of 20–40 is reasonable [202,203,204,205,206], and so our pilot study’s reliability statistic was based on 38 completed questionnaires.

5.4. Data Analysis

Cronbach’s Alpha and composite reliability (CR) tests were used to measure instrument reliability. Each test was undertaken using statistical package for the social sciences (SPSS). For the purpose of validating the measurement and structural model, structural equation modelling (SEM) was applied to the pilot data with SmartPLS 3.0 [207]. A variance-based technique was used to analyze the structural model, and this decision was made for several reasons: firstly, the partial least squares (PLS) method is effective for small-to-moderately sized samples, and it provides parameter estimates even at reduced sample sizes [208,209]; secondly, PLS is viable for exploratory research [210], particularly when examining new structural paths in the context of incremental studies that extend previous models [211], or when the relationships and measures proposed are new or have not been extensively examined in the prior literature [212,213]; and thirdly, the variance-based approach in PLS is effective for predictive applications. Therefore, since the study’s objective was to identify the factors underlying organizational-level CC continuance use (i.e., not to examine a particular behavioral model), PLS was a suitable choice [214].

5.5. Prototype Development and Evaluation

Drawing on the literature and theories of the IS continuance model, IS success model, IS discontinuance model, and TOE framework; the present study purposes a continuance use measurement prototype (i.e., system/application) to validate the research model.

In this phase, a prototype is developed, and evaluation is conducted through a survey. The main purpose of the prototype development is to apply and validate the proposed model through the evaluation and of the level of match between the model and the prototype in the real world. The prototype developed for the present study is in accordance with the processes proposed by Sommerville [215]. The processes involved in this phase are shown in Figure 4 and detailed below.

5.5.1. Establish Prototype Objectives

The objectives of developing the prototype are: (i) to assist cloud service providers and ICT decision makers at HEIs for evaluating the continuance use of CC services in HEIs; (ii) to provide a guideline on the vital requirements needed for ensuring a successful use of the CC service in HEIs.

5.5.2. Define Prototype Functionality

The fundamental criterion to quantify the success of a software system is the extent to which it pleases its customers [48]. A software/system requirements specification (SRS) is an explanation of a proposed software system that meets different kinds of stakeholder needs. The system requirements specification (SRS) suggest two main requirements of a system: functional and non-functional requirements. The functions relevant for the development of the prototype are derived from the literature. Accordingly, specific features of the prototype that are significant to the proposed prototype should listed and elaborated when full implementation in carried out.

5.5.3. Develop Prototype

The prototype will be built using the React JS web application framework. It is used collaboratively with Hyper Text Markup Language (HTML), Cascading Style Sheets (CSS), jQuery, Node js web server and MySQL Database. A flow chart design use case diagram, content, and navigation structure as well as data dictionary will be designed to build the prototype.

5.5.4. Evaluate Prototype

This research will implement the user acceptance test to validate the proposed model by evaluating the overall usability and acceptability of the prototype. A structured survey will be conducted at the end of the development process once the prototype is fully developed. The acceptance test used is based on the Perceived Usefulness and Ease of Use (PUEU) instrument by Davis [98] which has been based on the Technology Acceptance Model (TAM) [216,217]. Perceived usefulness and perceived ease of use are hypothesized to be fundamental determinants of user acceptance and system use [98,218]. A sample of respondents consisting of ICT decision makers at HEIs will be selected to participate in this survey.

According to Faulkner [219], studies to evaluate a prototype of a novel user interface design reveals severe errors quickly and, therefore, often require fewer participants. The literature suggests that 3 to 20 participants provide valid results [220]. PUEU consists of 12 questions with 7 scales from unlikely (1) to likely (7). To analyze the PUEU test, descriptive analysis (mean, standard error, median, mode) using SPSS will be carried out. The questions of the survey are divided into two parts. The first part is on demographics while the second on prototype perceived usefulness and ease of use. The PUEU test has been widely implemented to explain on the overall user acceptance towards an information technology (e.g., [221,222,223,224,225,226]).

6. Preliminary Results

A pilot study was conducted to increase the consistency of the measures used throughout the research. Noteworthily, the main objective associated with a pilot study is to ensure the validation of the initial instrument, and to identify any inconsistencies that could undermine the accuracy of the results [227].

6.1. Validity and Reliability of the Survey Instrument

Face validation, content validation, and a pilot study were undertaken to ensure the validity and reliability of the survey instrument. Face validity refers to the procedure of assessing an instrument based on feasibility, consistency of style and formatting, readability, and linguistic clarity [185,186]. In this study, face validity was evaluated in interviews with academic researchers (n = 2) with experience in questionnaire design. Regarding content validity, this refers to the extent to which a survey instrument measures what it intended to measure [177,178,227], and an expert panel was assembled to test it in this study. Although only three experts are the minimum for a content validity panel [228,229], five were recruited for the present study (three academics involved in IS and CC, as well as two industry practitioners). Online and offline distribution channels were used for the questionnaire in order to assess the constructs, and the measurement of items also took place with 4-point scale suggested by Davis [230]. A textbox was provided for every question, thereby ensuring that the participants had enough free space in which to provide comments. The participants identified areas for potential changes (e.g., “re-word this sentence”), and to quantify the relevance of every item, the average congruency percentage (ACP) [231] was computed. Noteworthily, the threshold value of ACP recommended by DeVon, Block [186] was achieved (i.e., 90%). In this regard, it is worth drawing attention to the fact that the questionnaire’s items were adapted from the prior literature of the adopted theoretical models (i.e., the IS continuance model [101], IS success model [105,120], IS discontinuance model [16], and TOE framework [102].

Reliability testing involved measuring each construct’s attribute based on the Cronbach Alpha reliability test statistic, and this decision was made owing to the statistical test’s widely accepted status in the literature in terms of internal reliability (i.e., the ability of an object of measurement to yield the same results on independent occasions). Several studies have inspected the minimum threshold for Cronbach Alpha as being above or equal to 0.7, while values less than 0.6 have been inspected to have a lack of reliability [232]. Cronbach Alpha values are given for every construct, as well as inter-item correlation values, in

Table 5. Reliability Statistics.

Construct	No. Items	Min Inter-Item Correlation	Max Inter-Item Correlation	Cronbach’s Alpha
CC Continuance Use	3	0.656	0.828	0.907
Satisfaction	4	0.684	0.81	0.916
Confirmation	5	0.124	0.795	0.78
Net Benefit	13	-	-	0.916
Technical Integration	3	0.711	0.788	0.891
System Quality	13	-	-	0.927
Information Quality	7	-	-	0.928
System Investment	3	0.67	0.731	0.836
Collaboration	5	0.504	0.742	0.899
Regulatory Policy	5	0.398	0.821	0.894
Competitive Pressure	4	0.577	0.772	0.86
All Items	65		0.862	0.913

. According to Briggs and Cheek [233], inter-item correlations in a reflective measurement scale offer data pertaining to the scale’s dimensionality. Furthermore, mean inter-item correlation is distinct when compared to a reliability estimate because it is not impacted by scale length. Resultantly, it provides a less ambiguous sense of item homogeneity. When the mean inter-item correlation does not exceed 0.3, this suggests that the item is not correlated with others in the same construct in a strong way [234]. At the same time, inter-item correlations that exceed 0.9 are indicative of multicollinearity issues [235]. For Cronbach Alpha values that exceed 0.7, this suggests favorable internal consistency in terms of the items on the scale [236]. Specifically, Cronbach Alpha values that exceed 0.9 are considered “excellent”, while values exceeding 0.8 and 0.7 are considered “good” and “acceptable”, respectively [237]. For values exceeding 0.6, these are considered “questionable”, while values lower than 0.5 are considered “unacceptable” [237]. In this study, relatively only one of the inter-item correlations fell below the recommended value of 0.3, meaning that the implications associated with their elimination from the survey instrument were considered. Given their weak correlations with other items in the measurement of confirmation construct, one item was removed. Additionally, because the resulting Cronbach Alpha values would not have increased after the deletion of the item, the item was retained (see Table 5).

Drawing on partial least squares (PLS) analysis, the validity of the reflective and formative models was evaluated based on the recommendations of Hair Jr, Hult [208], which are discussed below.

6.2. Reflective Measurement Model Evaluation

Assessment of the reflective measurement model involved taking an estimate of internal consistency, along with convergent and discriminant validity (see

Table 6. Quantitative Assessment of Measurement Model (Reflective).

Cloud Computing Continuance Use (Reflective)	Loadings 1	AVE 2	CR 3
CCCU1	0.896	0.845	0.942
CCCU2	0.961
CCCU3	0.898
Confirmation (Reflective)	Loadings	AVE	CR
CON1	0.505	0.544	0.852
CON2	0.622
CON3	0.876
CON4	0.786
CON5	0.832
Satisfaction (Reflective)	Loadings	AVE	CR
SAT1	0.872	0.798	0.941
SAT2	0.887
SAT3	0.917
SAT4	0.897
Technical Integration (Reflective)	Loadings	AVE	CR
TE1	0.923	0.824	0.934
TE2	0.876
TE3	0.924
System Investment (Reflective)	Loadings	AVE	CR
SI1	0.91	0.799	0.923
SI2	0.871
SI3	0.901
Collaboration (Reflective)	Loadings	AVE	CR
COL1	0.88	0.716	0.926
COL2	0.775
COL3	0.835
COL4	0.867
COL5	0.87
Regulatory Policy (Reflective)	Loadings	AVE	CR
RP1	0.809	0.697	0.92
RP2	0.84
RP3	0.859
RP4	0.853
RP5	0.812
Competitive Pressure (Reflective)	Loadings	AVE	CR
CP1	0.776	0.748	0.922
CP2	0.904
CP3	0.878
CP4	0.895

¹ All Item Loading > 0.5 Indicates Indicator Reliability; ² All Average Variance Expected (AVE) > 0.5 as indicates Convergent Reliability; ³ All Composite Reliability > 0.7 Indicates Internal Consistency.

). The reliability of the instrument was acceptable, with reflective factor loadings in excess of 0.505 (i.e., greater than the recommended level of 0.5) [209]. CR was also acceptable, where every construct’s value was greater than 0.852 [238].

In the case of convergent validity, this was computed as the average variance extracted (AVE) for every construct, and it was greater than 0.5 in each case [239]. Every square root for the AVEs was greater than the corresponding latent variable correlation, thereby indicating a satisfactory level of discriminant validity (see

Table 7. Discriminant Validity (Fornell-Larker Criterion).

Latent Construct	CCCU	COL	CP	Conf	IQ	NB	RP	SI	SQ	SAT	TE
CC Continuance Use	0.919
Collaboration	0.85	0.846
Competitive Pressure	−0.524	−0.541	0.865
Confirmation	−0.201	−0.277	0.397	0.737
Information Quality	−0.615	−0.579	0.381	0.328	formative
Net Benefits	0.719	0.736	−0.592	−0.617	−0.733	formative
Regulatory Policy	0.409	0.472	−0.821	−0.306	−0.378	0.513	0.835
System Investment	0.903	0.807	−0.458	−0.251	−0.578	0.695	0.383	0.894
System Quality	0.838	0.789	−0.593	−0.481	−0.782	0.877	0.438	0.776	formative
Satisfaction	0.473	0.54	−0.468	−0.58	−0.812	0.794	0.462	0.481	0.805	0.894
Technical Integration	0.894	0.739	−0.479	−0.256	−0.696	0.647	0.396	0.809	0.794	0.504	0.908

).

6.3. Formative Measurement Model Evaluation

The three-step process suggested by Hair Jr, Hult [207] was used to evaluate the formative measurement model (see

Table 8. Quantitative Assessment of Measurement Model (Formative).

Redundancy Analysis, Assessing Multicollinearity, Significance and Contribution
Net Benefits (formative)	VIF	t-values	Weights	Loadings
NB1	3.615	0.604	0.125	0.633
NB2	3.276	1.32	0.313	0.802
NB3	6.695	1.653	−0.489	0.648
NB4	3.098	1.561	0.346	0.71
NB5	2.202	1.618	0.23	0.535
NB6	1.942	0.801	0.134	0.691
NB7	5.617	1.785	0.608	0.876
NB8	3.896	0.632	0.146	0.609
NB9	3.236	0.988	0.214	0.641
NB10	1.51	0.932	0.158	0.45
NB11	6.653	1.149	−0.391	0.721
NB12	2.053	0.231	−0.039	0.594
Net Benefits (Reflective)	F2
Redundancy Analysis	0.763
NB13
System Quality (formative)	VIF	t-values	Weights	Loadings
SQ1	4.197	0.115	−0.019	0.75
SQ2	3.715	0.854	0.141	0.626
SQ3	2.57	1.397	0.165	0.794
SQ4	2.182	1.222	0.134	0.72
SQ5	5.351	1.203	0.204	0.867
SQ6	5.582	0.199	−0.035	0.726
SQ7	2.615	2.399	0.262	0.771
SQ8	1.3	0.711	−0.065	0.184
SQ9	4.435	1.392	0.239	0.768
SQ10	1.92	0.046	0.005	0.707
SQ11	3.749	1.26	0.156	0.659
SQ12	2.434	0.692	0.075	0.82
System Quality (reflective)	F2
Redundancy Analysis	0.784
SQ13
Information Quality (formative)	VIF	t-values	Weights	Loadings
IQ1	3.122	0.366	−0.079	0.664
IQ2	3.232	0.995	0.228	0.874
IQ3	2.84	1.569	0.348	0.839
IQ4	3.78	1.787	0.436	0.874
IQ5	4.753	0.838	0.219	0.831
IQ6	2.928	0.017	−0.004	0.727
Information Quality (reflective)	F2
Redundancy Analysis	0.884
IQ7

). At the outset, convergent validity was examined, which refers to the degree to which a given measure is positively correlated with other measures in the same construct [207]. In order to test convergent validity, redundancy analysis can be performed [240]. In this study, every construct was associated with satisfactory convergent validity, and the path coefficients ranged from 0.763 to 0.884 (i.e., greater than the recommended level of 0.7) [207]. In the second step, collinearity issues were addressed by computing variance inflation factors (VIFs) for every indicator [241], which can identify multicollinearity problems among the measures. Most VIFs in this study did not exceed the recommended level of 5 [242], but several were unreasonably high. Nevertheless, because formative measurement models are based on the regression of the formative construct against its measures, the stability of the measures’ coefficients is not only affected by the strength of the intercorrelations but also by the sample size. Given that the present process is concerned with testing the initial model, these indicators will not be considered until full-scale data are used to test the model.

For the final step of the three-step process, indicators are evaluated in terms of their significance and relevance, specifically by utilizing the initial research model. A range of formative indicators were not significant at the level of 10%. This is consistent with expectations because, as noted by Cenfetelli and Bassellier [243], the greater the number of indicators, the higher the likelihood that the indicators will not be significant. This is because each indicator competes to account for the variance in the target construct. Mathieson, Peacock [181] used in their study a range of formative indicators to assess perceived resources, and four out of seven were identified as insignificant. Additionally, Walther, Sedera [14] noted that system quality is associated with three significant indicators, while information quality and net benefits only have one and two significant indicators, respectively. Critically, however, lack of significance for an indicator should not be viewed as suggestive of a lack of relevance. If an indicator’s outer weight is not significant, but its outer loading is considerable (i.e., greater than 0.5), it is necessary to interpret the indicator not as absolutely important but rather as important [207]. In the study undertaken by [207], it was noted that, if the theory-driven way the construct is conceptualized is strongly supportive of the decision to keep the indicator (e.g., based on the judgements of specialists), then it ought not to be eliminated in most cases. An additional issue relates to the phenomenon of negative indicator weights [243]. Interpretations should not view these as items that negatively impact the construct; rather, they should be considered as correlated to a higher degree with indicators of the same measure as opposed to the construct they measure.

7. Discussion and Conclusions

The goal of this research paper was to propose a conceptual model that extends and contextualizes the IS continuance model to improve our understanding of the organizational level continuance of CC in HEIs. Drawing on the prior literature in organizational-level continuance (i.e., IS success and IS discontinuance models), the proposed research model extends the IS continuance model by the following constructs: net benefits, system quality, information quality, technical integration, system investment, collaboration, regulatory policy, and competitive pressure through the lens of TOE framework. The results of a pilot study, conducted through a survey with ICT decision makers, and based on the proposed conceptual model, indicate that the instrument is both reliable and valid, and therefore pave the way for further research.

The pilot study’s results lend valuable support to the model constructs and instrument in assessing CC continuance use at HEIs. Additionally, analysis with Cronbach’s Alpha indicates that the model constructs have satisfactory reliability [238], while convergent and discriminant validity for each of the constructs has been established using AVE and CR tests. Based on the feedback received from the pilot study’s participants, as well as from the statistical analysis, a revised survey instrument was devised for the full-scale research. An assessment of the reflective measurement model was undertaken by computing internal consistency, discriminant validity, and convergent validity. In each case, the instrument was associated with satisfactory results. Formative measures were evaluated based on convergent validity, collinearity issues, and significance and relevance, revealing satisfactory performance in terms of convergent validity. As for collinearity issues, these were examined by computing the variance inflation factors (VIFs) for every indicator, most of which did not exceed the maximum threshold of 5 [242]. Certain VIFs were greater than the required value, but because formative measurement models are based on regression (which is informed both by measure intercorrelations and sample size), these indicators will be considered when using full-scale data for the model. Additionally, indicators were examined in terms of significance and relevance by considering the initial research model.

7.1. Theoretical Contributions

This study provided empirical literature within IS, especially CC; in addition, this study provided assessment for CC adoption in HEIs, and more revitalization of the CC and intent of decision makers to utilize CC in HEIs.

One of the most contributions of this research constitutes to the body of knowledge within IS field surrounding continuance and discontinuance. This study provides an extensive model that extends and contextualizes the IS model using three dominant models in organizational-level continuance (i.e., IS success model, IS discontinuance model, and TOE framework) to improve our understanding of the organizational-level continuance of CC in HEIs. Furthermore, the results of the full-scale research will aid in developing a context-specific model that can be exploited to examine CC continuance in HEIs. Furthermore, it will play a valuable role in extending the academic literature on CC with empirical evidence. In addition, the study will form the basis of a final model that could be applied to examine continuance with other novel technologies within the sphere of education, as well as in other sectors. Finally, the study is expected to contribute to developing the literature in the best-available organizational-level continuance models for HEI settings, as well as other similar domains.

7.2. Practical Implications

From the practical implications’ viewpoint, this study provides potential implications for practitioners and cloud providers. Given that almost all CC service models rely on subscriptions [6], the current research purposed a conceptual model to assess the main factors that HEIs consider when deciding whether to continue their use of CC services. In practical settings, many are concerned with reducing capital IT expenses [3,18,19], and IS services allow client organizations to select from various services that they can continue using or discontinue using [8,9]. Therefore, conducting research that focuses on the continuance of IS will play a critical role not only in theory but also in practice. Furthermore, measuring the model constructs not only reflectively but also formatively would add little to the practical contribution of the study.

Besides, this study provides potential implications for decision makers. The research results will also assist IT decision makers in CC when seeking to optimize institutional resource utilization, or to commission and market CC projects. As a case in point, the individual weightings associated with the model’s constructs can serve as guidelines that software vendors will use to focus their efforts towards retaining customers. These weights can also be leveraged by clients to guide routine assessments over whether the use of a specific CC service should be discontinued.

7.3. Limitations

This study has some limitations that will bring about the focus of subsequent research. First, similar to organizational studies, possible bias to the results may occur from representing individual views rather than a shared opinion within the HEIs. This can be addressed if the constructs in the full-scale study are used for longitudinal evaluations rather than just cross-sectional evaluations, then the client organizations will be able to learn about critical “pain-points”. Second, the proposed model is intended for organizational-level usage. However, CC ecosystems involve operationalizing constructs such as IT infrastructure availability and computer sophistication [244,245]. Future research will have to take additional perspectives to understand continuance on an organizational level. Finally, the proposed model contextualized the IS continuance model based on previous literature in organizational-level continuance to better suite HEIs. However, future research may have to consider contextual constructs to understand continuance on HEIs.

7.4. Future Research Directions

The model and instrument development activities covered in this paper are the first step towards a larger initiative that seeks to examine the factors affecting the organizational continuance use of CC in HEIs. The validated instrument that was piloted in this study will be used to gather data from a large sample of IT decision makers in Malaysian HEIs. The study will analyze the impact of every construct in the proposed model on the continuance use of CC, and their significance in the model will be validated based on a test of the proposed hypotheses, where SEM and maybe other analytical approaches (e.g., Artificial Neural Network) will be used. A final model will be generated, which is expected to have considerable value in future organizational-level continuance studies of various technologies in the coming years (e.g., CC continuance in SMEs and government departments). Therefore, further quantitative and qualitative research about the conceptualized model and its relationships is recommended. A prototype should be developed and evaluated to validate the research model. A user acceptance test using the Perceived Usefulness and Ease of Use (PUEU) instrument will conducted and demonstrated the overall feasibility and acceptability of the prototype. As a final phase of the research design [180], conclusions should be drawn from the final model through the interpretation of results confirmed by empirical analysis and the evaluation of prototype developed.

A world with current modern technologies that keep evolving in which organizations and individuals keep adopting the evolved technologies requires further assessment for its sustainability and success use. Thus, researchers are required to always examine these innovations by investigating the continuance use of future technologies. In this regard, the fourth industrial (IR 4.0) revolutions provide a dialectical, intricate, and intriguing opportunity to all aspects of our life, in which the society would be changed for the better. As a case in point, education in the IR 4.0 era (Education 4.0) is driven by technologies such as artificial intelligent (AI), augmented reality (AR), Internet of things (IoT), big data analysis, CC, 3D printing, and mobile devices, which can promote the way of teaching, research, and service and change the work area from task-centered to human-based [48,153,246,247,248,249,250,251]. Many of the IR 4.0 technologies have been adopted and used in many sectors; therefore, further investigations on the continuance use of those technologies may gain the attention of the researchers. Finally, future research on how technology advancements could leverage learning and teaching process to achieve sustainability in HEIs is recommended.