Towards Developing a Framework to Analyze the Qualities of the University Websites

Abstract

The website of a university is considered to be a virtual gateway to provide primary resources to its stakeholders. It can play an indispensable role in disseminating information about a university to a variety of audience at a time. Thus, the quality of an academic website requires special attention to fulfil the users’ need. This paper presents a multi-method approach of quality assessment of the academic websites, in the context of universities of Bangladesh. We developed an automated web-based tool that can evaluate any academic website based on three criteria, which are as follows: content of information, loading time and overall performance. Content of information contains many sub criteria, such as university vision and mission, faculty information, notice board and so on. This tool can also perform comparative analysis among several academic websites and generate a ranked list of these. To the best of our knowledge, this is the very first initiative to develop an automated tool for accessing academic website quality in context of Bangladesh. Beside this, we have conducted a questionnaire-based statistical evaluation among several universities to obtain the respective users’ feedback about their academic websites. Then, a ranked list is generated based on the survey result that is almost similar to the ranked list got from the University ranking systems. This validates the effectiveness of our developed tool in accessing academic website.

1. Introduction

With the rapid growth of information and communication technology (ICT), having a website is inevitable for any organization, especially for a university. University website is one of the primary resources for the prospective students when they seek information about academic programs and decision making process of the university [1,2,3,4,5,6]. These websites can be good sources of reference for general information about the university and thus can be used to empower users to learn about the university easily and can also provide different facilities to the students [7].

A dynamic website can also help with registration procedure, payment, digital library facility etc. Moreover, these websites can be used to create a globally competitive advantage to attract prospective students [8]. Thus, the quality of a university website should be a key concern and seeks for special attention [1,9,10].

1.1. Background

On the report of University Grants Commission (UGC) of Bangladesh, there are currently 46 public, 98 private and 03 international universities running in Bangladesh. Most of these universities maintain their own websites; however, quality of these is not up to the mark [11].

According to the Ranking Web of Universities, there are only 02 universities from Bangladesh that have secured positions in the top 100 South Asian university websites; these are Bangladesh University of Engineering and Technology (BUET) (40th position) and University of Dhaka (DU) (45th position). This scenario reveals the dissatisfactory quality level of the academic websites in Bangladesh. Furthermore, there has been almost no research work done to find out the weakness of the academic websites of Bangladesh. The only work found in this regard is in [11].

There are various existing methods for evaluating website quality, some of these can be found in [6,11,12,13,14,15,16,17,18]. Most of them focus on usability and accessibility, html page, aesthetic design, page size but not focus on the contents of information. Content of information plays vital role in making a bridge between website user and university authority [19,20]. In this manner, we have considered content of information as an important factor, and selected three major attributes for evaluating university websites in our proposed mechanism.

1.2. Contributions

The contributions of this research work are listed as follows:

• In our proposed framework, three key quality attributes were considered, such as content of information, website performance and loading time.
• In addition, there are multiple factors considered under content of information, such as university vision and mission, faculty information, online course registration etc.
• Along with this, we have conducted a questionnaire-based survey with the respective university students for obtaining their satisfaction level about their university website.
• We also performed statistical analysis on the feedback of survey documents to get a clear insight about the websites of several universities from the point of view of its users.
• Finally, we compare the survey results with the results of our developed system; this comparison validates the effectiveness of our tool.

Overall, our research in this paper aims to fill the aforementioned gaps by developing an automated web-based tool to evaluate the quality of the websites of the universities in context of Bangladesh. Our system is dynamic in nature, it can take any academic website URL as input for processing and generate result. This tool can also perform a comparative analysis among several university websites.

1.3. Outline of Paper

The rest of the paper is organized as follows. Section 2 provides a brief review of related work. Section 3 discusses in detail about the system architecture and design. Section 4 presents implementation. Finally, Section 5 concludes the paper and recommends the future directions of this work.

2. Related Work

Good quality of a website has a direct and positive effect on its users’ satisfaction [21]. According to prior studies, there are multiple factors influencing the quality of a website, such as interface design, navigation, information content, loading time, usability, security, and so on [4,22,23]. When assessing the quality of any website, researchers choose one or more factors according to the context of their research. There are two mostly applied methods of assessing the quality of websites; these are- using automated tools and by collecting direct opinion of users (questionnaire-based survey). Both of the two methods are equally important and effective in evaluating web quality. Survey based method can pick up the actual satisfaction level of user’s, whereas, a software application can access the internal factors (i.e., page load time, broken link etc.) of that site easily.

In [24], Khandare et al. evaluated the usability of an engineering college website using three automated tools namely: Website Grader, SEOptimer and Qualidator. The authors in [25] also used Website Grader to evaluate websites in context of tourism field. They recommended automated evaluation over human judgement because human judgement can be biased.

A hybrid tool is proposed in [26] to assess the usability of e-commerce website using AHP (Analyical Hierarchy Process) and ANFIS (Adaptive Neuro Fuzzy Inference System). They proposed a fuzzy based Quality Index Evaluation Method to gauge the design quality of a website. Fuzzy-DEMATEL theory and Fuzzy trapezoidal number technique are used to build this automated tool. To verify the tool, it has been tested on several academic, informative and commercial websites.

In [27], Jayakumar et al. developed Website Quality Assessment Model (WQAM), a framework for assessing the quality of e-learning website on the basis of four high-level quality metrics such as accuracy, feasibility, utility, and propriety. These quality metrics are obtained through a Questionnaire Sample (QS).

Zahran [28] discussed the classification of the evaluation process into two type: web evaluation and website evaluation. He suggested some criteria to select the proper assessment method.

Almahamid et al. [7] showed an analysis from the perspective of a lecturer about the factors that influence a lecturer to use their university website. They developed an integrated model of TAM (Technology Acceptance Model) model and D&M model to assess the website of Middle East University. Perceived usefulness (PU) and perceived ease of use (PEU) are the required factors according to their findings.

In [29], the authors used a web crawler to assess the quality of website to explore the relationship between hospitals and health systems’ website quality and their patient satisfaction levels. The authors of [30] investigated that there is a positive correlation of academic performance of an institute with the quality of its website. Giraud et al. [31] highlighted the potential of a tool based on filtering redundant and irrelevant information, which allows reducing the cognitive load of users with blindness and improving interface usability.

Much research have been performed using statistical evaluation of website quality as well. For instance, in [32], Medyawati and Mabruri tried to identify the service quality of two banking websites providing e-banking services using a questionnaire-based analysis on the users of e-banking services. They considering Accessibility, Interaction, adequacy of information, Usefulness of content, Lifestyle and Personality as quality measuring factor.

Andalib et al. [1] presented a survey work conducted by including all potential users of the website of Payame Noor University, Iran. They involved 387 participants who answered some questions based on four factors including efficiency, accessibility, achievement and security. Their analysis revealed that efficiency and accessibility influence user’s trust and satisfaction positively.

In [33], the authors presented a questionnaire-based evaluation to assess the academic website of an Indonesian university website, whether it meet the acceptability criteria of usability testing. The survey was conducted with a questionnaire of 17 questions and filled by 95 respondents. Finally they opined that the target website was easy to use, though there were scope to improve its usability.

There are some multi-method approaches of website quality evaluation, where the researchers performed statistical evaluation along with other assessment task. For example, EL-firjani et al. [34] proposed a usability evaluating technique for any web-based systems. The U.A.E. airline website was used for their case study purpose. They performed a comprehensive evaluation by arranging a ‘task and time’-based plan. They assigned some tasks (registration, ticket purchase etc.) to 05 participants and assigned some time to complete these tasks. Based on successful completion within time, the authors measured the usability of that website. Along with this, they validated their technique with the result obtained from a questionnaire-based point of view of the users.

A similar kind of research is presented in [35], where authors aimed to test the usability of the library website of Sulaimani Polytechnic University, Iraq. They involved three users of that website and assigned them four tasks (e.g., search for a particular book etc.). The number of completed tasks and time to complete these were monitored and used to evaluate that site. The participants also rate the site to express their satisfaction level. Finally they gave 06 recommendations to improve the website.

Islam et al. [11] applied a hybrid method of both questionnaire-based survey and automated tools using html toolbox and web page analyser to judge the academic websites of Bangladesh. They selected websites of 20 universities and a total of 200 participants took part in their research work. They tried to find out the weakness of these websites and gave some suggestions for improvement. They claimed their work as the first initiative in this research area in context of Bangladesh.

In [3], the authors performed a statistical evaluation to observe the relationship between web usability and web presence of five Turkish universities. They carried out two methods- a user testing to measure user performance on some selected tasks and a questionnaire with 20 participants to explore user’s satisfaction. They concluded that academic websites with a higher web presence are most likely to meet the users’ need.

3. System Architecture and Design

The proposed work has two distinct parts—automated evaluation and statistical evaluation. Both the parts are elaborated below.

3.1. Automated Evaluation

Our developed automated framework has four basic modules: 1. Data Collection Module 2. Data Preprocessing and String Matching Module, 3. Analysis Module and 4. Output Module. The overall structure of our tool is shown in Figure 1.

3.1.1. Data Collection Module

Data Collection module performs the tasks of extracting necessary data from the websites of the universities. We used Selenium for web crawling. Selenium is a Web Browser Automation Tool of web applications and can interact with browsers. Along with this, we used ChromeDriver, an open source tool, to open the browser and load the referenced page. We consider document-oriented database for storing crawled data as the data are in semi-structured format. The architecture of data collection module is shown in Figure 2. In data collection, we extracted information under the attribute Content of Information. Under this attribute, 25 key strings are selected (e.g., university vision and mission, faculty information, etc.) and the system search for these key strings in the desired website. The list of 25 key strings can be found at the end of the paper in Appendix A. A similarity matching task is performed in the next module between the key string and the data stored in temporary database. Here in parallel, chrome driver fetches necessary internal data such as- connect start time, domain complete time, secure connection start time, etc. A total of 20 types of data are used to calculate the performance of the website and the loading time. The list of 20 types of data are attached in Appendix B. At the same time, individual score is generated based on string matching task, loading time and performance of the website. These scores are combined to get the final score. Further details about score generation can be found at Section 3.1.3 (Data Analyzing Module).

3.1.2. Data Preprocessing and String Matching Module

Information stored in temporary database may contain redundancy or noise, that can degrade the system performance. To fix this problem, data cleaning has been performed to remove noise, missing tuples and redundant data. This clean data is then ready to take part in string matching task. Here we find out the matching of information between this clean data of temporary database, with key strings of python dictionary. 25 types of key strings are stored in python directory, such as university vision and mission, faculty information etc. The full list can be found at Appendix A. A score for this matching task is generated, that is called as Count in Algorithm 1. This score reveals how many of our desired information this specific university website maintains. Algorithm 1 demonstrates the steps of preprocessing and string matching tasks. Here, the first loop, continuing from index 1 to 25, fetches Values of each index. In the second loop, the individual Value is taken from Values and converted to lowercase. If Value is matched with any string of temporary database then Count variable is increased by one and exit from second loop. After end of first loop, it returns Total Count for string matching module.

Algorithm 1: Algorithm for data preprocessing and string matching module.

3.1.3. Data Analyzing Module

In this module, we combine all the scores and information counts generated against per website and then determine the final score of each website. We consider the maximum score for a website is 100, where 50% marks is for content of information, 40% is for performance of website and 10% is assigned for web page loading time.

Now, the score for the attribute contents of information for each website = (Total count × 50)/25, where 25 is the number of key strings selected for content of information. For example, in BUET website, total count for contents of information is 23. So the score for this attribute: (23 × 50)/25 = 46 (out of 50)

Then for website performance, we are using two types of equations. One is-

$$value=max(40-time\_variable\ast 4.0,0)$$

(1)

We have considered time_variable = 10 as threshold value. If time_variable >= 10, then value = 0.

This equation is used to evaluate the following time_variables: (connect_start_time, connect_end_time, domain_lookup_end_time, domain_lookup_start_time, fetch_start_time, redirect_end_time, redirect_start_time, request_start_time, response_start_time, secure_ connection_start_time, unload_event_end_time, unload_event_start_time, worker_start_time).

Another equation is-

$$value=max(40-time\_variable/12.50,0)$$

(2)

We have considered time_variable = 500 as threshold value. If time_variable >= 500, then value = 0.

This equation is used to evaluate the following time variables: (dom_complete_time, dom_content_loaded_event_end_time,dom_content_loaded_event_start_time,dom_ interactive_time,duration_time, load_event_end_time, load_event_start_time).

Using these two equations, we got marks for Performance of BUET website = 31.87131999793928 (Out of 40.0)

The quality standard of the website loading time must be less than 10 seconds. If loading time is greater or equal than 10 second, then score for this attribute is 0. Otherwise, score will be [10 - load_time]. So, the equation for marks of loading time is-

$$Marks\_of\_loading\_time=max(10-load\_time,0)$$

(3)

For example, loading time of BUET website is 0.09454989433288574s. So, score for this attribute: max(10 - 0.09454989433288574, 0) = 9.905450105667114 (Out of 10). And the final score of a website is the summation of all individual scores across each attribute.

Thus, the total score for BUET website: marks for content of info + marks of load time + marks of performance = 46 + 9.905450105667114 + 29.013639997690916 = 84.91909 (Out of 100).

3.1.4. Output Module

In Output Module, the system shows the analyzed data in chart and tabular form. It generates results based on two features:

• Details of each attribute for each university website.
• University website ranking.

Detailed results are shown in Result section.

3.2. Statistical Evaluation

Along with automated evaluation, we also prepared a survey document to get the user’s perception of using their university website. For this, we have conducted a questionnaire-based survey among 22 public and private universities in Bangladesh. The questionnaire has two parts: the first part addressed the credentials of participants (Name, University Name, Department, Email) and the second part included 23 questions that are effective in evaluating academic websites in context of Bangladesh. All the questions are selected based on the following features:

• Educational information
• Online facilities (Online learning environment, course registration etc.)
• Faculty member information
• Other helpful information (Scholarship, health care center etc.)
• Navigation and responsiveness
• Visual appearance

At the last portion of questionnaire, the participants were encouraged to put their suggestions regarding the improvement of their university website.

4. Implementation

4.1. Automated Tool

We built a web-based automated tool using Python Scrapy. We have used web crawler, an internet bot for extracting information from our required URLs. The whole system is implemented using the algorithms stated in Algorithms 2–4.

Algorithm 2 is used to retrieve the content of information from any particular university website. Initially, the URL of a university website is provided as input. Then the Web Crawler gets access of the HTML page of that URL using ChromeDriverManager. After that, BeautifulSoup, a python Library, is used to get the source file of HTML page. The source HTML is then parsed and scanned. After that, from the class ‘dropdown’, all unnecessary elements are removed and the rest are stored in a temporary database. Almost similar operation is performed if algorithm gets any path of ’//a[@href]’.

Algorithm 2: Algorithm of Content of Information.

Algorithm 3 keeps track of the start time for a given URL; i.e., when the data stream is started to read, as well as the end time when the data stream is finished reading. Finally, the load time is calculated using the difference between the start and end times.

Algorithm 3: Algorithm for attribute loading Time.

Result: loading time required for each webpage  Input URL;

1. fetch data stream using stream = urllib2.urlopen(URL);

2. calculate the start time;

3. read data stream using “stream.read()” and calculate the end time;

4. calculate: load time = end time - start time;

5. calculate: marks of load time = max (10 − load time, 0);

In Algorithm 4, the performance value of the given university website is calculated using chrome driver. Twenty types of data are considered in this regard.

Algorithm 4: Algorithm for attribute Website Performance.

Result: Performance of website  Input URL;

1. set chrome driver path and other parameters;

2. install and initialize the chrome driver;

3. retrieve the 20 type of data selected for performance calculation;

4. calculate marks on performance attribute;

After calculating all these values using these algorithms, total score per website is calculated using the formula $total\_score$ = $marks\_of\_content\_of\_information$ + $marks\_of\_load\_time$ + $marks\_of\_performance$.

4.2. Questionnaire Based Evaluation

Data Collection Procedure

For performing questionnaire-based evaluation, we have conducted a survey with 22 public and private universities in Bangladesh. For collecting users’ feedback, we have contacted with the students of our selected universities via social networks and E-mail and they were given a brief of the survey purpose. A total of 1820 students took part in this evaluation process, where almost equal number of students participated from each university. All the respondents were under graduate or post graduate level students from different disciplines.

5. Experimental Result

5.1. Result of Automated Tool

Our automated tool generates the analyzed results in two forms-

• Comparative analysis of selected university websites considering each attribute (Contents of information, Performance of website, Website loading time)
• Rank list of selected university websites based on overall score

Comparative analysis of the websites is shown in Figure 3, Figure 4 and Figure 5. In each figure, X axis is for the name of the selected universities, and Y axis shows their corresponding marks. However, the numerical score for each attribute is listed in

Table 1. Rank list of selected university-websites.

University	Total Score	Marks: Content of Information	Marks: Performance	Marks: Load time
KUET	87.36	50	28.19837998	9.159674883
BUET	84.92	46	29.01364	9.905450106
RUET	80.02	42	28.21030003	9.814033747
RU	79.61	42	28.54937997	9.059892654
SUST	78.01	40	28.25218002	9.754738331
BRAC	77.97	40	28.18038005	9.790145159
Bangladesh Agricultural University (BAU)	74.45	34	30.94878003	9.504649401
International Islamic University Chittagong (IIUC)	74.31	36	28.51358002	9.80026722
CU	74.22	36	28.34311995	9.881783724
Daffodil International University	72.97	36	27.46102006	9.505887508
JnU	72.59	34	28.68139998	9.907032013
JU	70.93	34	27.92382002	9.003336878
Uttara University	70.85	34	28.78095999	8.070547581
KU	69.87	32	28.15985999	9.70607543
Port City International University	68.42	30	29.05629998	9.362358093
CUET	64.17	26	28.44676004	9.724467039
City University	63.67	30	25.41607798	8.252358093
DU	61.88	26	26.21411998	9.667788744
Chittagong Medical College (CMC)	56.14	18	28.68057996	9.462923765
AUST	54.69	16	28.76954001	9.921000481

.

Based on the final score of each website, our system generates a Rank List of the selected websites. Table 1 shows that rank list for our selected university websites.

It is observed from Table 1 that the academic website of KUET university obtains highest Total Marks (87.36) and thus gets the top position. Website of BUET is in the 2nd top position, as it received significantly lower score for Content of Information than that of KUET, despite having marginally better loading time and performance.

5.2. Result of Questionnaire Based Evaluation

5.2.1. Analysis of Each Question

Each respondent answered the questions about their university website; the questions are listed in

Table 2. Overall feedback of the questionnaire.

No.	Questions/Statements	Yes(%)	No(%)
Q1	Information on continuous education and training provided by your university website is helpful	59	41
Q2	Does the website provide any Online Learning Environment facility (e.g., Moodle)?	43	57
	Does the university website provide following information? (Q3 to Q8)
Q3	Research and extension activities	79.5	20.5
Q4	Student scholarship	53.33	46.67
Q5	Academic calendar	86.67	13.33
Q6	Transport schedule and related important information	33.33	66.67
Q7	Relevant information and contact details of university health-care center?	20	80
Q8	Student-related social support (e.g., complaint on sexual harassment or ragging)?	13.33	86.67
Q9	Does the website maintain updated faculty member list?	82.35	17.65
	Does it contain the following information about faculty members? (Q10 to Q13)
Q10	Detailed qualification information	64.71	35.29
Q11	Publication list	42.11	57.89
Q12	Field of interest	47.37	52.63
Q13	Updated contact information	31.58	68.42
Q14	Does your university website provide online course registration service?	31.58	68.42
Q15	Does it support online payment of fees (e.g., registration fee, library fine etc.)?	26.32	73.68
Q16	Does it provide facilities for online library management (e.g., book rental/ renew)?	23.81	76.19
Q17	Does the website support multiple languages (e.g., native and English)?	19.05	80.95
Q18	Is the website compatible with the most popular browsers (e.g., Internet Explorer, Firefox, Chrome)?	90.48	9.52
Q19	Is the website visually appealing?	80.95	19.05
Q20	Do you feel good using this website?	52.38	47.62
Q20	Is the website’s loading time satisfactory?	66.67	33.33
Q22	Do you get all updated information/ notices from the website?	33.33	66.67
Q23	Please give your suggestions for improving your academic website quality

. This table shows an overall feedback for each of the questions. There is one more question (question no. Q23) which is for collecting the users’ suggestions concerning for the improvement of their respective academic website. This portion is covered in detail in Section 5.2.2.

A graphical representation of the above mentioned feedback is given in Figure 6.

5.2.2. Recommendation for Website Improvement

In our survey document (question no Q23), we encouraged the respondents to write down their suggestions for improving their university website. Eleven percent of the total respondents answered this question and suggested their opinions. By analyzing all the suggestions we categorized these under six factors. According to the respondents’ opinions, the academic websites in Bangladesh should be improved in the following factors:

• Speed/ Load time
• UI design
• Useful info/ Updated notice board
• Faculty information (Experience, field of interest)
• Online facilities (registration, payment)
• Overall improvement

Here in Figure 7, the pie chart depicts that 26% of students want a more appealing and structured UI design for their university website. Nineteen percent suggested to improve the information section requiring up-to-date notice board. Seventeen percent respondents showed concern about the loading time of the website. Fifteen percent of students want a more organized and informative Faculty Information section. Nine percent suggested online registration and other online facilities. Fourteen percent of students think that their university website needs overall improvement to meet their needs.

5.2.3. Rank List Based on Average Score of Each Website

Each question in the questionnaire is weighted with a score for evaluation. Accordingly, the total score for each university website is obtained by summing up the scores got from each respondent of that university. Then the average score is calculated as follows:

Average score = Total score of a university website/Number of respondents from that university.

Based on the average score, a rank list is generated which is shown in

Table 3. Rank list based on questionnaire.

University	Average Score
BRAC	85.12
BUET	82.99
Daffodil International University	75.27
Port City International University	70.07
International Islamic University Chittagong (IIUC)	69
AUST	68.66
CUET	68.01
City University	68
DU	67.46
KUET	67.03
JU	66.88
RUET	66.08
SUST	64.55
RU	59.79
Uttara University	57.76
Bangladesh Agricultural University (BAU)	50.95
CU	50.92
JnU	50.32
KU	50.16
Chittagong Medical College (CMC)	43.57
HSTU	41.22
Sylhet Agricultural University (SAU)	41.01

. The website of BRAC university has received the highest rating by the students’ evaluation.

6. Conclusions and Future Research

This paper presents an extensive study on quality analysis of the websites of the universities of Bangladesh. University websites are built to provide information and services to its stakeholders. There is almost no research work conducted in Bangladesh to evaluate the quality of academic websites in this country. To meet this gap, we developed a dynamic web-based tool that can assess any university website based on three attributes. The main challenge was to build such a system as dynamic, because of the different HTML page structure of each website. By making the system dynamic, we are able to evaluate any university website rather than some selected websites. This tool can generate a score across each website and we can get a ranked list of our desired university websites based on this score. Based on each attribute, our system also generates a comparative analysis among all the selected websites.

Along with this, a questionnaire-based survey was held to get an insight of users’ perception about their university website. This survey reveals that most of the university websites in our country cannot meet users’ satisfaction.

By conducing this research, we find that there is huge scope to improve the quality of the university websites in Bangladesh. These should be well designed along with future research, considering richness in the content, maintaining updated information and notices, and also improving the loading time and performance of different website activities. We hope that our study will help website designers and future researchers to enhance the quality of their developed sites to a large extent. We expect more follow-up studies regarding this field in context of Bangladesh. In addition, we plan to expand this study by considering websites of the universities all over the world.