Evaluating Methods and Decision Making

A special issue of Information (ISSN 2078-2489). This special issue belongs to the section "Information Applications".

Deadline for manuscript submissions: closed (31 May 2022) | Viewed by 34306

Special Issue Editor

Special Issue Information

Dear Colleagues,

Evaluation is an important phase of the software life cycle because it is the procedure that ensures that the system meets the design requirements. Furthermore, evaluation is often used to choose the software product that is best in meeting the requirements of research or an activity that has to be implemented. Such a procedure depends on multiple criteria, goals, or objectives of often conflicting nature.

Therefore, decision-making methods have been repeatedly used in the evaluation of software in different domains. Given the above, we solicit papers that combine evaluation methods and decision making. We welcome theoretical and empirical contributions, using qualitative or quantitative methods.

Topics of interest include but are not limited to:

- Evaluation Methods;

- Inspection Methods;

- Empirical Methods;

- Hybrid Methods;

- Decision Making:

- Decision support systems;

- Multi-Criteria Decision Making;

- Artificial intelligence and Decision Making;

- Human Computer Interaction;

- Software Engineering

Dr. Katerina Kabassi
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Information is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (8 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

14 pages, 1712 KiB  
Article
Vertical Integration Decision Making in Information Technology Management
by Menekse Gizem Gorgun, Seckin Polat and Umut Asan
Information 2022, 13(7), 341; https://0-doi-org.brum.beds.ac.uk/10.3390/info13070341 - 15 Jul 2022
Cited by 1 | Viewed by 2184
Abstract
Vertical integration, also known as make-or-buy, defines whether activities are conducted by company or provided by external parties. There are different models to support decision making for vertical integration in the literature. However, they ignore the uncertainty aspect of vertical integration. As a [...] Read more.
Vertical integration, also known as make-or-buy, defines whether activities are conducted by company or provided by external parties. There are different models to support decision making for vertical integration in the literature. However, they ignore the uncertainty aspect of vertical integration. As a strategic decision, vertical integration is multidimensional and less frequent. This study contributes a new data-driven model that includes all these characteristics of vertical integration decisions. In this study, a methodology is suggested that benefits from the models in the literature and assesses the results with data obtained from real IT cases. Different methodologies were followed to reach a model that accurately predicts make-or-buy decisions in IT projects at a retail company. Firstly, three different knowledge-based generic models derived from the literature were applied to predict decisions for twenty-one different make-or-buy cases in IT. The highest accuracy rate reached among these knowledge-based models was 76%. Secondly, the same cases were also analyzed with Naïve Bayes using factors originally introduced by these generic models. The Naïve Bayes algorithm can represent the uncertainty inherent in the decision model. The highest accuracy rate obtained was 67%. Thirdly, a new data-driven model based on Naïve Bayes using IT-related factors was proposed for the decision problem of vertical integration. The data-driven model correctly classified 86% of the decisions. Full article
(This article belongs to the Special Issue Evaluating Methods and Decision Making)
Show Figures

Figure 1

28 pages, 8271 KiB  
Article
User Evaluation and Metrics Analysis of a Prototype Web-Based Federated Search Engine for Art and Cultural Heritage
by Minas Pergantis, Iraklis Varlamis and Andreas Giannakoulopoulos
Information 2022, 13(6), 285; https://0-doi-org.brum.beds.ac.uk/10.3390/info13060285 - 04 Jun 2022
Cited by 3 | Viewed by 2293
Abstract
Content and metadata concerning a specialized field such as Art and Cultural Heritage are often scattered throughout the World Wide Web, making it hard for end-users to find, especially amid the vast and often commercialized general content of the Web. This paper presents [...] Read more.
Content and metadata concerning a specialized field such as Art and Cultural Heritage are often scattered throughout the World Wide Web, making it hard for end-users to find, especially amid the vast and often commercialized general content of the Web. This paper presents the process of designing and developing a Federated Search Engine (FSE) that collects such content from multiple credible sources of the world of Art and Culture and presents it to the user in a unified user-oriented manner, enhancing it with added functionality. The study focuses on the challenges such an endeavor presents and the technological tools, design decisions and methodology that lead to a fully functional, Web-based platform. This implemented search engine was evaluated by a group of stakeholders from the wider fields of art, culture and media during a closed test and the insights and feedback gained by these tests are herein analyzed and presented. These insights contain both the quantitative metrics of user engagement during the testing period and the qualitative information presented by the stakeholders through interviews. The above findings are thoroughly discussed and lead to conclusions regarding the usefulness and viability of Web applications in the aggregation and diffusion of Art and Cultural Heritage related content. Full article
(This article belongs to the Special Issue Evaluating Methods and Decision Making)
Show Figures

Figure 1

26 pages, 2080 KiB  
Article
Critical Success Factors Evaluation by Multi-Criteria Decision-Making: A Strategic Information System Planning and Strategy-As-Practice Perspective
by Sehoon Kim
Information 2022, 13(6), 270; https://0-doi-org.brum.beds.ac.uk/10.3390/info13060270 - 25 May 2022
Cited by 2 | Viewed by 7319
Abstract
Strategic information system planning (SISP) is a central process that enables organizations to identify the strategic alignment of their IT portfolio to achieve their business needs and objectives. The extant SISP literature has focused on theoretical and processual aspects and has left methodological [...] Read more.
Strategic information system planning (SISP) is a central process that enables organizations to identify the strategic alignment of their IT portfolio to achieve their business needs and objectives. The extant SISP literature has focused on theoretical and processual aspects and has left methodological ambiguity about how SISP is practiced. This paper contributes to the current knowledge by providing a mixed-methods SISP framework labeled CSF-MCDM for company-wide strategic alignment. The paper conducts a methodological synthesis, embracing an expert-based qualitative approach based on a PEST-SWOT and causal layered analysis to draw the critical success factors of a next-generation business system for an automotive company in South Korea. The derived CSF dimensions and sub-criteria are evaluated by the multi-criteria decision-making model, engaging a strategy-as-practice lens to SISP to enable an integrative analysis of IS strategy formulation, planning, and implementation. The findings reveal the relative strategic priorities of dimensions, the following core activities, and the global priorities for resource distribution planning for IS strategy of the firm. This paper argues that bringing replicability with SISP and diversifying methodological approaches within the organization is substantial. This paper also suggests that future researchers validate the suggested framework for scientific replicability and expand the SISP research stream within the entire IS/IT ecosystem. Full article
(This article belongs to the Special Issue Evaluating Methods and Decision Making)
Show Figures

Figure 1

28 pages, 2530 KiB  
Article
Making Informed Decisions to Improve Restaurant Image Using a Hybrid MADM Approach: A Case of Fast-Food Restaurants in an Island of East Malaysia
by Anath Rau Krishnan, Rizal Hamid, Ronia Yeap Siew Lin, Geoffrey Harvey Tanakinjal and Balan Rathakrishnan
Information 2022, 13(5), 219; https://0-doi-org.brum.beds.ac.uk/10.3390/info13050219 - 22 Apr 2022
Cited by 1 | Viewed by 3801
Abstract
Restaurant image refers to an immediate perception that pops up in a customer’s mind when the name of a restaurant is mentioned. Therefore, it is crucial for restaurants, including fast-food restaurants (FFRs), to evaluate and sustain a positive restaurant image. However, evaluating and [...] Read more.
Restaurant image refers to an immediate perception that pops up in a customer’s mind when the name of a restaurant is mentioned. Therefore, it is crucial for restaurants, including fast-food restaurants (FFRs), to evaluate and sustain a positive restaurant image. However, evaluating and improving a restaurant’s image is challenging, since it counts in multiple service attributes associated with various degrees of unknown priority. Even so, the existing literature is yet to outspread the usage of an appropriate multi-attribute decision-making (MADM)-based approach to specifically evaluate the image of FFRs. Therefore, this research aimed at employing such an approach to evaluate the image of four FFRs on an island in East Malaysia, using various people, processes, and physical evidence attributes. Firstly, an initial list of FFR image attributes was elicited from the available literature. This initial list was then further validated through a two-round Delphi survey involving a panel of ten experts. A questionnaire was then designed based on the finalized attributes, and data collected from a sample of 251 respondents were analyzed using the compromised-analytical hierarchy process (C-AHP) method. The C-AHP results suggest that the strategies to improve an FFR’s image should primarily incorporate the following six attributes: hospitality, employees’ problem-solving skills, employees’ knowledge, food taste, physical cleanliness, and service response time. The FFR at the top of the ranking has the highest performance scores over these same six attributes. Surprisingly, employees’ appearance and restaurant exterior were reported as the two least important image attributes. This research is the first to demonstrate the application of a hybrid MADM-based approach to uncover the weights of FFR image attributes and rank those FFRs by computing their aggregated image scores. Full article
(This article belongs to the Special Issue Evaluating Methods and Decision Making)
Show Figures

Figure 1

19 pages, 471 KiB  
Article
Evaluating Methods for Efficient Community Detection in Social Networks
by Andreas Kanavos, Yorghos Voutos, Foteini Grivokostopoulou and Phivos Mylonas
Information 2022, 13(5), 209; https://0-doi-org.brum.beds.ac.uk/10.3390/info13050209 - 19 Apr 2022
Cited by 6 | Viewed by 3747
Abstract
Exploring a community is an important aspect of social network analysis because it can be seen as a crucial way to decompose specific graphs into smaller graphs based on interactions between users. The process of discovering common features between groups of users, entitled [...] Read more.
Exploring a community is an important aspect of social network analysis because it can be seen as a crucial way to decompose specific graphs into smaller graphs based on interactions between users. The process of discovering common features between groups of users, entitled “community detection”, is a fundamental feature for social network analysis, wherein the vertices represent the users and the edges their relationships. Our study focuses on identifying such phenomena on the Twitter graph of posts and on determining communities, which contain users with similar features. This paper presents the evaluation of six established community-discovery algorithms, namely Breadth-First Search, CNM, Louvain, MaxToMin, Newman–Girvan and Propinquity Dynamics, in terms of four widely used graphs and a collection of data fetched from Twitter about man-made and physical data. Furthermore, the size of each community, expressed as a percentage of the total number of vertices, is identified for the six particular algorithms, and corresponding results are extracted. In terms of user-based evaluation, we indicated to some students the communities that were extracted by every algorithm, with a corresponding user and their tweets in the grouping and considered three different alternatives for the extracted communities: “dense community”, “sparse community” and “in-between”. Our findings suggest that the community-detection algorithms can assist in identifying dense group of users. Full article
(This article belongs to the Special Issue Evaluating Methods and Decision Making)
Show Figures

Figure 1

23 pages, 2531 KiB  
Article
Selected Methods of Predicting Financial Health of Companies: Neural Networks Versus Discriminant Analysis
by Jarmila Horváthová, Martina Mokrišová and Igor Petruška
Information 2021, 12(12), 505; https://0-doi-org.brum.beds.ac.uk/10.3390/info12120505 - 06 Dec 2021
Cited by 10 | Viewed by 2815
Abstract
This paper focuses on the financial health prediction of businesses. The issue of predicting the financial health of companies is very important in terms of their sustainability. The aim of this paper is to determine the financial health of the analyzed sample of [...] Read more.
This paper focuses on the financial health prediction of businesses. The issue of predicting the financial health of companies is very important in terms of their sustainability. The aim of this paper is to determine the financial health of the analyzed sample of companies and to distinguish financially healthy companies from companies which are not financially healthy. The analyzed sample, in the field of heat supply in Slovakia, consisted of 444 companies. To fulfil the aim, appropriate financial indicators were used. These indicators were selected using related empirical studies, a univariate logit model and a correlation matrix. In the paper, two main models were applied—multivariate discriminant analysis (MDA) and feed-forward neural network (NN). The classification accuracy of the constructed models was compared using the confusion matrix, error type 1 and error type 2. The performance of the models was compared applying Brier score and Somers’ D. The main conclusion of the paper is that the NN is a suitable alternative in assessing financial health. We confirmed that high indebtedness is a predictor of financial distress. The benefit and originality of the paper is the construction of an early warning model for the Slovak heating industry. From our point of view, the heating industry works in the similar way in other countries, especially in transition economies; therefore, the model is applicable in these countries as well. Full article
(This article belongs to the Special Issue Evaluating Methods and Decision Making)
Show Figures

Figure 1

16 pages, 1810 KiB  
Article
Application of Multi-Criteria Decision-Making Models for the Evaluation Cultural Websites: A Framework for Comparative Analysis
by Katerina Kabassi
Information 2021, 12(10), 407; https://0-doi-org.brum.beds.ac.uk/10.3390/info12100407 - 30 Sep 2021
Cited by 5 | Viewed by 2566
Abstract
Websites in the post COVID-19 era play a very important role as the Internet gains more visitors. A website may significantly contribute to the electronic presence of a cultural organization, such as a museum, but its success should be confirmed by an evaluation [...] Read more.
Websites in the post COVID-19 era play a very important role as the Internet gains more visitors. A website may significantly contribute to the electronic presence of a cultural organization, such as a museum, but its success should be confirmed by an evaluation experiment. Taking into account the importance of such an experiment, we present in this paper DEWESA, a generalized framework that uses and compares multi-criteria decision-making models for the evaluation of cultural websites. DEWESA presents in detail the steps that have to be followed for applying and comparing multi-criteria decision-making models for cultural websites’ evaluation. The framework is implemented in the current paper for the evaluation of museum websites. In the particular case study, five different models are implemented (SAW, WPM, TOPSIS, VIKOR, and PROMETHEE II) and compared. The comparative analysis is completed by a sensitivity analysis, in which the five multi-criteria decision-making models are compared concerning their robustness. Full article
(This article belongs to the Special Issue Evaluating Methods and Decision Making)
Show Figures

Figure 1

25 pages, 3106 KiB  
Article
Combatting Visual Fake News with a Professional Fact-Checking Tool in Education in France, Romania, Spain and Sweden
by Thomas Nygren, Mona Guath, Carl-Anton Werner Axelsson and Divina Frau-Meigs
Information 2021, 12(5), 201; https://0-doi-org.brum.beds.ac.uk/10.3390/info12050201 - 06 May 2021
Cited by 21 | Viewed by 8287
Abstract
Educational and technical resources are regarded as central in combating disinformation and safeguarding democracy in an era of ‘fake news’. In this study, we investigated whether a professional fact-checking tool could be utilised in curricular activity to make pupils more skilled in determining [...] Read more.
Educational and technical resources are regarded as central in combating disinformation and safeguarding democracy in an era of ‘fake news’. In this study, we investigated whether a professional fact-checking tool could be utilised in curricular activity to make pupils more skilled in determining the credibility of digital news and to inspire them to use digital tools to further their transliteracy and technocognition. In addition, we explored how pupils’ performance and attitudes regarding digital news and tools varied across four countries (France, Romania, Spain, and Sweden). Our findings showed that a two-hour intervention had a statistically significant impact on teenagers’ abilities to determine the credibility of fake images and videos. We also found that the intervention inspired pupils to use digital tools in information credibility assessments. Importantly, the intervention did not make pupils more sceptical of credible news. The impact of the intervention was greater in Romania and Spain than among pupils in Sweden and France. The greater impact in these two countries, we argue, is due to cultural context and the fact that pupils in Romania and Spain learned to focus less on ’gut feelings’, increased their use of digital tools, and had a more positive attitude toward the use of the fact-checking tool than pupils in Sweden and France. Full article
(This article belongs to the Special Issue Evaluating Methods and Decision Making)
Show Figures

Figure 1

Back to TopTop