Human-Centered Artificial Intelligence for Designing Accessible Cultural Heritage

Abstract

This paper reviews the literature concerning technology used for creating and delivering accessible museum and cultural heritage sites experiences. It highlights the importance of the delivery suited for everyone from different areas of expertise, namely interaction design, pedagogical and participatory design, and it presents how recent and future artificial intelligence (AI) developments can be used for this aim, i.e.,improving and widening online and in situ accessibility. From the literature review analysis, we articulate a conceptual framework that incorporates key elements that constitute museum and cultural heritage online experiences and how these elements are related to each other. Concrete opportunities for future directions empirical research for accessibility of cultural heritage contents are suggested and further discussed.

1. Introduction

Exhibits, activities, and instructions designed for cultural heritage sites and museums have changed a lot during the last century. The ways people enjoy museums and cultural heritage (henceforth CH) sites have significantly evolved through the last century. Both museums and cultural heritage sites used to focus on the preservation of sites, objects and exhibits and activities focused on the transmission of knowledge about the sites and objects. Activities and exhibits were centered around the sites and objects. For visitors this often implied that they were allowed to look and read but not to touch and interact. In the second half of the 20th century the museum became more user centered and invited visitors to actively think about the sites and objects, draw their own conclusions and interact with materials [1]. Established guidelines developed by UNESCO [2] and ICOMOS [3] promote the doctrine and the techniques of conservation, protection and valorization of cultural heritage sites. Cultural heritage organizations search for activities, programs and spaces that are inviting to a wide range of audiences, and have the potential to create a positive and enduring relation with their visitors.

The last technological advances allow organizations to create content and activities that are not only highly interactive, but also can be adapted to various types of visitors. Carefully designed adaptive activities that match the target of the organizations as well as the intended public have the potential to create an enduring bond with the visitors and allow organizations to create tighter connections with the community [4].

Cultural Heritage poses opportunities and challenges to the development of successful Information and communication technology (ICT) tools that allow democratic access for all. On the one hand, the digitalization of information is not enough, and besides digitizing content, one needs to work to make it accessible for users with different disabilities. On the other hand, as CH sites are extremely rich in objects and data, efficient methods for information presentation should also be thought so museums become more inclusive and create an online environment that makes online visitors feel welcome.

ICT can be a viable way for creating remote visiting experiences. Universal design advocates designing virtual as well as physical environments to be accessible and usable by as many potential users as possible. However, dominant ways of thinking about accessibility and information technology, and the resulting design and technology practices, sometimes create barriers for access and inclusion. For example, pandemic lockdown measures may close museums for long time and minorities such as e.g., the blind, may not profit from guided virtual tools, unless they are suited for blind people.

According to Cambridge dictionary, inclusiveness is: “an aura or environment of letting people in and making them feel welcome”, while accessibility is defined as: “the quality or characteristic of something that makes it possible to approach, enter, or use it”. The normative ways accessibility is implemented are mainly twofold. They can be physical or cognitive, with physical accessibility meaning, for example, wheelchair accessibility and computer access through the use of assistive technology such as alternative keyboards or screen readers, while cognitive accessibility includes accessibility for people with different cognitive disabilities, and is also associated with the creation of materials that support different users styles of navigation, providing users with multiple ways of engaging with the material and presenting the material, in short it should be regarded as simply good design practice [5].

Despite the growing concern about increasing accessibility for visitors with different disabilities, museums still do not make significant efforts to improve access for this people and for everyone [6]. Around 5.5% of the visitors are visitors with disabilities. The World health organization (WHO) estimates that 253 million people have some kind of impairment ref. [7]. Thus, increasing inclusion and accessibility is important for museums, since visitors with disabilities still experience a large amount of constraints and cannot visit the museums.

Tackling this group can have significant benefits for museums, and not only economic ones. Many museums also argue that even if they aim at creating accessible experiences, it is rather challenging to do so, especially when dealing with arts [8]. However, even today interactive technologies inside museums are built having users with all abilities in mind, and are not suited for people with different disabilities. A recent study revealed that most of these solutions do not provide adequate physical, cognitive, nor sensorial access for visitors with disabilities.

At the same time, museums must guarantee democratic participation [9]. Access to museums becomes a right that must be guaranteed to everyone, independent of age, abilities, location, religion, and preferences. Making CH accessible for all is important because it is through knowing the past that we can interpret our present times, and consciously act for a better future. Although universal design argues for creating accessible ICT solutions, the issue has often been neglected in the past literature, and there are no ‘one size fits all’ solution [10].

We study accessibility and inclusiveness in cultural heritage sites and museums, from three stand points of view: interaction design, educational and instructional design, and participatory design and co-design.

In the spirit of embracing AI, and more specifically eXplainable AI (or XAI) and Human Computer Interaction (HCI) as enabling technologies non devaluing art historians, paleontologists and any other Humanities domain experts, we present a human-centric view that enriches not only the human observer as a static one, but also the domain expert and communicator, while using XAI and HCI as enabling richer interfaces that adapt to the target audience in each case, dynamically. Personalized mechanisms based on learning progress [11], curiosity [12], and automatic curriculum learning [13] can act as ways for pedagogical agents to present content in the best order for engagement [14]. We expect this cross-disciplinary way to shape new avenues for ethical research in these areas, following the responsible AI principles [15]. We summarize our research agenda in Figure 1.

We aim to gain profound understanding on established practices from these three domains, as we believe that if we want to exploit XAI to its full potential to provide improved accessibility to museums and cultural heritage sites, we first need to understand how AI principles (for Responsible AI, as well as EU Guidelines for Trustworthy AI) need to be embedded and aligned with the current approaches, methods and processes already used in this sphere, and then implement such agenda. In our approach we always have the people at the center of the design process.

Participatory design and co-design for museums, are approaches to design strategy, that bring visitors into the heart of the design process. The core idea that underlies these approaches is that people that will use technologies and services must have the right to decide how these technologies and services will be. Therefore participatory design and co-design implement context-oriented tools and techniques, intervening in the design phases of discovery, ideation and co-creation. In these phases, the users (called also “participants”) take an active role of decision-makers in co-designing museum solutions that they will use themselves, and others after them.

Human computer interaction and interaction design have advanced significantly in this area as well, by various resources pointing to different successful or less successful approaches to bring museums closer to end users in online environments, via innovative interaction design paradigms, or solutions specifically tailored for challenged individuals. In this paper, we present a summary of these advances so far, and provide a vision for AI-empowered accessibility in the museum sphere.

A pedagogical perspective has often been taken in the CH domain to open up informal learning opportunities for the broad public, experience-based learning (constructivism), based on personal interaction with the site or the material. Design based on principles from learning sciences, but also developmental psychology, have been studied in past literature to alleviate accessibility for different groups. Instructional design can inform about the principles of effective instruction taking into account visitors with varying ages (or levels of development). Moreover, within instructional design, specific principles for the design of adaptive activities have been formulated.

This aims to be a cross disciplinary position paper to propose a human-centred contribution to XAI in the cultural heritage domain. The user centered nature, and the advances in technology, as well as the need for adaption calls for carefully designed activities and requires an interdisciplinary approach combining views of design based research, instructional design and human-centered AI, in particular, having explainable AI as part of a ubiquitous new curricula [16].

The paper is organized as follows: in Section 2 we detail the methodology we used to mining the HCI/Interaction design pedagogical design, and participatory design literature, in Section 3 we detail the results of our literature review, and in Section 4 we present a framework for future agenda for AI-powered accessibility research, and we present the different questions and challenges for such visionary research.

2. Methodology

The method for conducting this review is based on the procedure defined for conducting systematic literature reviews [17], consisting of a hybrid method including the procedure defined for conducting systematic literature reviews while driving research questions towards the future frontiers on making CH accessible for all. These are driven by the fast pace of the latest advances in AI. The procedure started by formulating basic research questions for the review overview. Subsequently, search terms were formulated and literature was selected, always bearing in mind that we want to start the research from common bases, but with specific focus. Based on a first exploration of the literature, search terms were refined and exclusions criteria were formulated. For each of the three search operations all relevant articles were fully read.

2.1. Step 1. Potentially Relevant Publications for Retrieval

As a first step we defined the shared keywords between the three fields. After several discussions among authors, as well as trying out the results produced by the queries using different key words, the decision was made to use the following four terms:

• museum*
• heritage
• access*
• inclusiv*

We decided to use Scopus database as it covers important publication venues for all of the three domains and in digital humanities and cultural heritage.

Table 1. Table detailing the queries used in the literature review.

Query	Research Field	Shared Keywords	Number of Papers
TITLE-ABS-KEY (“co-design” OR codesign OR “participatory design”) AND TITLE-ABS-KEY (museum* OR heritage) AND TITLE-ABS-KEY (access* OR inclusiv*)	Codesign/participatory design	museum*, heritage, access*, inclusiv*	32
TITLE-ABS-KEY (learning AND design AND technolog*) AND TITLE-ABS-KEY (museum* OR heritage) AND TITLE-ABS-KEY (access* OR inclusiv*)	Pedagogical design	museum*, heritage, access*, inclusiv*	93
TITLE-ABS-KEY (hci OR “Interaction design” OR “Human computer interaction”) AND TITLE-ABS-KEY (museum* OR heritage) AND TITLE-ABS-KEY (access* OR inclusiv*)	HCI/Interaction design/Human-computer interaction	museum*, heritage, access*, inclusiv*	146

was composed, it contains shared keywords that we used for all the three of the sub-domains, as well as field specific keywords that were coined by each of the author expert in the domain. Although Scopus provided a good starting point and a common base for evaluation/comparison of literature between the different disciplines, in the particular case of Human computer interaction literature, it does not index sources that cover important research in the domain, such as Virtual Archaeology Review or ISPRS Archives, Annals and journal. Thus, these were not taken into consideration in our literature review. We still do believe that the review, as presented below, is representative for the domain as a whole, although it is worth acknowledging the shortcomings in this respect. We hope that this literature review can represents a first step towards future works that can also include other indexing databases. We chose to review Scopus because it is a reference and well-known point shared by all our respective domains (interaction design, pedagogical design, participatory design). The willing to offer a multidisciplinary perspective in conjunction with the demand of a rigorous literature review methodology, led us to this choice.

2.2. Step 2. Publications Retrieved for Detailed Evaluation

In this paper, each of the authors reviewed the papers of their own subdomain that were identified in the first step and excluded papers that were false positives or less relevant for the aim of the paper. This was done individually for each of the three subdomains. A total of 271 papers remained for examination.

2.3. Step 3. Publications Included in the Review

In this phase, papers were cross-examined: in order to validate each paper to be included in the review, a specific schema for tracking all papers was created in MS Office Excel. For each paper we kept the following information: year, title, keywords, authors, methodology (empirical, using real data; empirical, using surveys; qualitative; theoretical; literature review) for further analysis.

Additionally, when writing the respective interventions for this paper, each of the authors in the domains contributed from their own previous knowledge and expertise in the domain, and in occasions, also used resources that were in their possession and knowledge from before, with the aim to make the paper even more relevant.

3. Results

3.1. Human Computer Interactions / Interaction Design

We present the results of the literature review analysis in the Human computer interaction/interaction design area from different sub-domains. Particular focus in the literature has been dedicated to interaction design in VR and 3D, interaction design on websites, multi-sensory experiences for all, and museum visiting robots for telepresence. We also reflect on papers that describe the design process and requirements followed for implementation of such types.

3.1.1. Interaction in VR and 3D

Different articles by Evett and Tan, 2020, Caputo et al., 2016, or Dulyan and Edmonds, 2012 [18,19,20] tried to investigate interaction in Virtual Reality (VR) and three dimensional (3D) environments for online museums, taking into account also accessibility for disabled and challenged individuals.

In one of the pioneering work s in this sphere, Evett and Tan, 2020 [18] tested accessibility of museum websites in VR for disabled people, where they were able to walk around the virtual museum. While talking and pronouncing commands, the disabled person can “move around” in the virtual environment. Although the focus was predominantly on the efficiency of the speech recognition system and potential improvements to it, the results from the study showed that the approach worked well and the authors suggest future improvements as well as a reflection on how this approach can be used in general for assistive technologies for different spheres as well.

A study by Caputo et al., 2016 tried to understand the gestural interactions suited for navigating in VR environments [19]. This study accounts for an experiment with 30 participants, where the findings suggest that ’Forward button’ seems best suited for moving ahead in virtual reality (among different options for such navigation), while for selection and information retrieval about an object, swipe and concrete buttons for such functions represented easiest modality to do so. The study found that picking objects in 3D was the most difficult task for participants.

A study by Dulyan and Edmonds, 2012 on the needs of a British museum in London tested and evaluated 3D speech presentations for blind people for the virtual room of the “Middle East” section. In the study [20] the exists for instance were presented with wind blowing to indicate with sound, the exit door of the virtual museum, as the blind person navigates from one room to the next one. Three out of 11 participants found disorientating the transitions, often stepping back and forth. Some even walked backwards, a behaviour that the authors did not expect. The general user experience was rated positive; 4 participants found the exits difficult to find.

3.1.2. Interaction on Cultural Heritage Websites

Many prior works have investigated the potential of interactive applications to increase access to digital contents for people with disabilities.

Tiansheng and Haiming (2016) review the design of 8 websites and how they present information for older adults [21], and tries to come up with a design of World War I museum for their town. They pay specific attention to the fact that the museum website is elderly friendly, as the visitors are mainly elderly. The key takeaways from their review analysis is that each website should: allow the user to change font size, allow the users to set favorite pages as a home page, to have a search function, allow users to bookmark options, and make fresh information easy to access. Finally, they consider relevant having an introduction video explaining the aim of the project, as well as to group information in three main categories (people, events, objects), and present this information with different background colors.

The National Library of Finland’s web portal has been analyzed in terms of accessibility and inclusiveness in a study by Schwartz and Kautonen, 2015 [22]. For this aim, email interviews and surveys with professors and students working in the USA were conducted. A total of 11 students and 4 professors participated. The participants found that the fact that the website was in Finnish and not in English was perceived negatively. In addition, the lack of information about the artistic license confused the virtual visitor regarding whether the accessed materials can be used (and reproduced) for other purposes or not. The participants suggest to use creative commons licences for presented material.

A study conducted by Sorce et al., 2018 with 3 autistic users explored usability and accessibility of avatar-based touchless gestural interfaces for autistic people. In this study, the participants could “pick up” 5 objects presented around them and thus consult information about the objects. For this the avatar is projected by a projector, and the “touch” is captured by a Microsoft Kinect sensor, both of them attached to a computer. The therapist in general noticed easy access for all of the three participants, while the third did not need any help [23].

FLIPPIN is a book like interface used in public spaces and museums for consulting contents [24]. Three different prototypes were evaluated; they allowed three different interaction techniques. One allowed only for linear consumption of contents and not giving any other choice to follow different paths, and the other two were interactive and one could choose different paths. The novelty and usability from interacting with a technology whose interaction is like “any physical book” provided a positive effect. The interactive ones arose user interest in contents.

These kind of approaches have been used for other cultural heritage domains and not only in museums. One such example is the publication by Rapp et al., 2013 on how food producers bring closer to the customers the ethnographic knowledge about food production of their region. This includes extending (e-including) people involved in the process, in order to convey how the product was produced. The evaluation was conducted at an exposition event with producers. They all found it a good way to bring new audiences, although they still expressed concern about the sustainability of the tool [25].

A system studied by Su and Liao, 2011 [26] based on RFID tagging of exhibits in the museum allows the visitors to save and share remarks about the exhibits with the non-visitors. The remarks and information saved by the visitors can later be turned into microblogs on which non-visitors can find further information about the museum and presented exhibits.

In this direction, the first system studying suited interaction paradigms for virtual visiting of museums were MINERVA [27] and MUD [28] projects. MINERVA shows how one can bring closer the museum to the people at home and it shows a first user interface, and the general architecture (based on Java) for such system. MUD includes a microphone feature that allows for communication of online visitors with the actual visitors visiting the same room at the same time.

3.1.3. Alleviating Distance by Remote Visiting with Robots

Assisting challenged individuals in visits with help of onsite robots, specifically designed to this aim, has been the focus of previous studies.

Case studies of robots developed for visiting the museum from remote have been investigated and presented by previous literature [29], successfully deployed within the EU-funded projects TOURBOT and WebFAIR. The mobile robots are positioned inside the museum, and can move inside it, in addition the robots were able to tell the number of people in the room and their positions. They could as well talk with the other visitors, and project the video streams from the spaces they were visiting. In addition, the authors simplified the system setup time and reduced it, facilitating its porting to new environments.

Experiences with an interactive museum tour-guide robot were described by Ng et al., 2015 [30]. They conclude that questions on how to provide a good user experience while users tele-operate a robot, and how to make the experience as close as possible to the real one, yet remain to be solved.

Scenarios for universally accessible cultural exhibits have been developed in the past for the needs of FORTH project [31]. These scenarios detail how old, blind, and hard of hearing people can or should access the museum exhibits, as well as which are the research challenges for providing such access, namely: novel interaction paradigms, novel forms of UIs adaptation, new accessibility interaction techniques designed for all, alternative means of information display, and personal cross-domain assistive solutions.

The ACCESSIBILITY PASS is an evaluation framework for museum structures, and measures how “welcoming” and how “disabled friendly” places are, specifically in tourism and culture heritage sites. The authors transformed the checklist into an interface/ tool to evaluate the structure [32].

3.1.4. Multi-Sensory Museum Experiences for All

In this section, we also describe the advances in terms of interaction design for museum experiences delivered on site, and how they also have been moved on to become more adapted for visitors with different disabilities.

Haptic interfaces have been used lately quite significantly by different authors such as Trotta et al., 2020, Carrozzino and Bergamasco, Coelho et al., 2020, Frid et al., 2019, Vaz et al., 2018 [33,34,35,36,37]. Some museums offer tactile and hearing multisensory experiences that help visually-impaired visitors formulate mental pictures of the objects, and provide important navigation and contextual information. Besides, they help creating new ways for disabled visitors to know about the objects presented, technology is used also to provide them better navigation information about the arrangement of the museum and how to find their ways around the museum.

One example of such novel approach was developed at the London Science museum to communicate abstract concepts in cosmology in a more inclusive and accessible manner by Trotta et al., 2020 [33]. Multistory elements that were part of the dark matter experience included: fluorescent body outline indicating where participants need to lie, Aquaruis dark matter simulation projection, haptic board for air skin stimulation, and a scent release device. The whole experience lasted 3 min, and the 46 participants found that such approach to museum visiting worked well for all visitors. They work at the moment to make it easily portable and implementable for other museums.

The Museum of Pure Form project [34,35] augmented visitors experience allowing them to touch and experience the properties of the sculptures belonging to the collection using an exoskeleton and a haptic interface for fingers.

Sound forest is an exhibition/music installation consisting of a room with light-emitting interactive strings, vibrating platforms and speakers at the Swedish Museum of performing arts. The participants in this study needed to match music, vibrations, and colors presented to them, based on the associations the museum arose in them. The musics elements were composed specifically for this study. There were 19 participants (children, teens and adults with physical and intellectual disabilities). The main contribution lies in their finding indicating that the Sound Forest can be used to provide visitors of different ages and disabilities with similar associations about music experience [36].

MM Gerdau Museum in Brasil enhanced its Pedras Sebidas exhibition with additional sensors and sound actuators to communicate the exhibition to blind and visually impaired participants [37]. Tests performed with 13 participants demonstrated very positive evaluations regarding pragmatic and hedonic qualities of the interaction.

Halbach and Tjostheim, 2019 propose a framework for more accessible museum experiences and provide measures on how museums can be assessed for their levels of “accessibility” towards adults with physical and intellectual disabilities inside of the museum. The accessibility indicators include: vision, earning, mobility, motor and cognition scores. They evaluate with users the “accessibility” of an exhibition in the Oslo Science Museum. Voice and motor-related accessibility was at the highest levels evaluated in the exhibition, while the hearing, vision and cognition scores can be improved [38].

3.2. Design Processes Explained: Requirements for Such Systems

Many studies provide requirements and design processes that future designers of CH systems should follow. In this section, we sum them up.

UniBSArt4All project [39] aims at supporting users in enjoying museum content in a tailored and accessible way. The project developers showcase the resulting design, the design method and approach to inclusive design they follow and the key dimensions of it, namely culture, age and disabilities on the human side, and time, cost and space on the technology side.

The architectural data, collection and annotation procedures adopted in Cultural Heritage Orientating Multimodal Experiences (CHROME project) were presented in [40]. The aim is to create an interactive system where remote visitors will be able to enjoy the museum.

The i-treasure project is presented with an ambitious aim to capture dancing moves, singing details in music formats, and pottery gestures for preserving crafts. Requirements were presented to make the platform easy to use although the authors did not perform an evaluation yet [41]. The paper presents sensor-specific requirements, education difficulty, and the needs for (1) adapting to the place where the person is, (2) detecting physical interactions among group members, and (3) visually highlight mistakes.

A paper by Song et al., 2014 presents the design process taken for developing an edutainment (education + entertainment) tool [42]. The authors did not perform a study but reported on the design of a system that enables visitors to trace objects visited and provides linear and non linear options for storytelling.

Interaction Design Approaches for Museums Already Using AI?

Our research identified only few use cases so far where AI was used for accessibility and inclusiveness in the cultural heritage and museums domain.

One paper describes the experience of the ArtiMuse project, which aims at passing to next generations the rare gestural knowledge and skills involved in the traditional craftsmanship, more precisely in the art of wheel-throwing pottery [43]. For the aim of the project, a human-computer interface was build which permits both the modeling and recognition of expert technical gestures, i.e., whether the online visitors mimic correctly the gestures as explained by the website.

Dal Falco and Stavros (2017) speak about the applicability of modern technologies in museums and cultural sites, and sums up different experiences on how for instance some simple natural language conversation models provide answers to online visitors questions. These are answered from a predefined set of possible answers [44]. Other similar kind of works that incorporate simple natural language conversation techniques and interactive storytelling have also been successfully tested and implemented in the past [45,46,47].

3.3. Pedagogical Design

Pedagogical design refers to a development or design process that is guided by specific learning objectives as well as pedagogical theories in order to identify and design activities, technologies and objects that are suited to reach the intended outcomes (Conole and Fill, 2005 [48]). Presently, pedagogical designers do not exclusively focus on clearly defined formal education that takes place in classroom settings, but also embrace more complex and open ended learning settings. Experiences can be obtained at physical sites, like in a museum, or at an historical building, but also through VR experiences or digital materials presented through the Internet. Designing for cultural heritage is different from designing for classroom settings in the sense that learning in the context of cultural heritage is often experiential, implying that students’ experiences and actions form the basis of their reflections and thinking processes that in the end influence their attitudes as well as the learning outcomes.

In AI, the pedagogical design counterpart to open-ended learning experiences can relate to agents deciding or inventing what problem to solve next (focusing on meta-diversity search and language [49] as a tool for creative exploration [50]),which could relate to the use of automatic tutors in CH- and generic- pedagogical design. EvoCraft [51], for instance, provides a benchmark for open-ended learning [52] and autonomous development [53] of agents. Automatic curriculum learning [13], as a methodology that tries to design the set of tasks to be learned by an agent, and the order in which they are learned, could as well be used in CH online sites.

From the perspective of pedagogical design, certain design aspects allow enactment of the experiential learning; starting with involving the visitor on the cognitive, physical as well as affective level with the presented content and acknowledging their prior knowledge and experiences that a visitor brings to the site. Following the model of Falk and Dierking, 2012 [54] we argue that the experience of a visitor is shaped by the following three interacting context factors: the personal context, the physical context and the social context. In short the personal context relates to the psychological make-up and state, the prior knowledge, attitudes, experiences, motivation and interests that visitors bring with them when they engage in cultural heritage related activities and that shape their experiences. The physical context revering to the building, the artifacts and the ambiance, all factors that influence the way visitors move through a museum and what catches their attention. The social cultural context is focusing more on the cultural background of the visitors and the type of interactions they engage in. The model nicely illustrates that learning in cultural heritage settings is complex and influenced by various factors. Taking into account the personal and the socio-cultural context it becomes clear that not all visitors enter a museum or cultural heritage site with the same amount of prior knowledge, cultural capital and resources, and also their attitudes, motivation and interest may differ. For visitors who lack background knowledge or motivation, this might negatively affect their experiences at a site or during an activity as per study from Pecore et al., 2017 [55]. The way information is presented might make certain groups feel left out (I do not understand the language, this is not for me) and therefore also has a negative effect on future visits to cultural heritage sites (Hughes and Moscardo, 2019) [56,57]. Research indicates that cultural capital can influence the way individuals and families behave during their visit at a cultural heritage site, and has a high impact on the amount of learning that takes place (Tison Povis and Crowley, 2015) [58].

Learning in the context of cultural heritage sites can be understood as a process of (co-)construction (Hohenstein and Missouri, 2018) [59]. Knowledge and understanding are acquired through interaction with the content and other visitors; it is an active process. Action, interaction, prolonged engagement and continued reflection on the content or the collection is important in facilitating learning and shaping positive outcomes (Horn et al., 2016) [60]. This process of (co-)construction can be shaped by adults that guide the experience (Land-Zandstra et al., 2020) [61]. Unfortunately there might not always be a knowledgeable person available to explain materials, and support deeper level interaction. Therefore it is important that materials and experiences are highly self-explanatory and facilitate active exploration of the material and the content. Recent technological developments can enable visitors to better understand what is being presented, and facilitate active interaction and deeper level processing of the material, without depending on active participation of parents or educators.

An example of an app that is supporting museum visitors that lack appropriate prior knowledge and has features that stimulate prolonged and focused interaction with the content is the work in (Nelson, Bowman and Bowman, 2017) [62]. Dr. Discovery or abbreviated Dr. D) was presented to the visitors as an app on a portable device. The Dr. D iPad app contains a question asking tool that allows visitors to ask questions through voice or typing. The question is matched to a database consisting of approximately 13.000 verified question answer pairs related to the museum content. When there is a close match the app provides an answer; in case of a weaker match the app asks the visitor to provide additional information, if there is no match, Dr. D will inform the user that he does not know the answer to their question yet, and offers to tell them something related to the exhibit. In this way the app stimulates the visitor to further explore the museum. To encourage visitors to focus on the content of the exhibition, gamification is introduced. Visitors are stimulated to ask questions in order to get Dr. D, who is floating in space, back to the spaceship. There is not enough fuel for Dr. D to make it to the spaceship, but the visitors can help Dr. D by earning fuel. Fuel is earned by asking content related questions to Dr. D that will be answered by the app. Visitors that were offered a version of the app in game mode, asked almost twice as many questions as visitors that worked with an app without the gamification features. Interaction with the app facilitated prolonged interaction with the material (Nelson, Bowman and Bowman, 2017) [63].

The introduction of interactive displays at cultural heritage sites also offers opportunities for active interaction with the material and promoting engagement as well as learning. Interactive displays are more and more common place in museums and sometimes as part of the exhibited content, sometimes replacing traditional labels and information stands. Lakkaraju et al., 2017 [64] investigated three different versions of an interactive display. The first version presented information in the from of menus. Visitors could select pictures and learn more about the objects they selected. This first version was compared with two more innovative redesigns of the interactive display. Both redesigns started from the idea of curiosity of the visitors; the first redesign, called Big Questions, focuses on knowledge gaps, by posing questions related to missing information that can be obtained through interaction with the exhibition. The second redesign, called Timelines, tries to create a pleasant and visually pleasing experience focusing more on the emotional and sensory-motor aspects of the visit. These interactive displays were combined with a display case that presented important innovations from the Bronze Age, and visitors conversations and interactions were carefully observed and analyzed. Results show that while the first redesign initially increased engagement with the material, this engagement washed away once the missing knowledge was retrieved. The second redesign was less directive and resulted in more exploration reflected in visitors levels of interpretative and curiosity talk. Both redesigns seem to facilitate different forms of interaction with the material and relate to different goals. The Big Questions approach is more directive and allows educators to steer the learning process of the visitors to specific contents and activities. However, the directive character might have a detrimental effect on motivation and attitude. The visually appealing redesign offers visitors more opportunities to pursue their own interest, but the attractiveness of the display might also pull their attention away from the content Lakkaraju et al., 2017 [64]. This implies that both redesigns stimulate interaction but match a different set of educational objectives (transmission of knowledge versus fostering interest). Technological and ICT-based pedagogical design in the context of cultural heritage sites should have to deal with certain constraints related to the setting. DiPoalo and Akia (2006) discussed this in relation to the design of adaptive multimedia for museum learning. It should be acknowledged that visitors have limited time, often come with a group, and that the expectations might be a mix of educational with recreational. Moreover, the organizations that implement the technology are interested in increasing visitor-ship through repeated visits. On the technical and organizational site, designed technologies should be able to deal with a constant stream of visitors at peak times; these should be engaging but intuitive to use so that no staff attendance is needed. Again, it is stressed that in order to create a positive experience the material should be adaptable to the needs and interest of visitors. The virtual beluga project (https://ivizlab.org/research/virutal-beluga-project-vancouver-aquarium/) in the Vancouver Aquarium is taking these constraints into account. The systems builds on techniques from the gaming industry in order to create a simulation where belugas interact with each other but also with the user, extending the experience of the user beyond the observation of live and captive belugas present at the aquarium. Visitors could interact with the simulated belugas by changing the environment of the simulated belugas by the placement of objects (like boats, ice, or food) that are tracked and introduced in the virtual display. The interactivity of the simulation stimulates prolonged engagement with the system but it also allowed multiple visitors to collaboratively experiment with the simulation, making it suitable in peak hours. Through interaction with the system visitors acquired knowledge about belugas, for example that belugas live in a world of sound. The system was designed in a modular way using intelligent objects and therefore allowed the exhibition designers to adapt the content and the learning experiences and stimulate repeated visits.

Learning from Digital Collections and Websites

As museums and cultural heritage sites often digitize their collections, this offers possibilities for the creation of online content, or opening up digital collections to visitors. Navigating digital collections can be hard, visitors might have difficulties understanding and interpreting the material that is displayed. To ensure prolonged and meaningful interaction with the material, visitors of a virtual collection need goals, guidance or scaffolds. The features of the typical “my collection” or “my museum” systems offered on websites that allow visitors of museum websites to create their own subcollections is often not fully exploited by the visitors Marty et al., 2011 [65,66], the more advanced options are not used. When designing experiences for learning in the context of digital collections and virtual exhibitions, engagement is central. In line with the earlier mentioned model posed by Falk et al., 2012 [67] we have to take into account the personal and socio-cultural context, and in this specific setting the physical context is replaced by the context of a digital environment that might be more or less suited and appealing for specific activities and specific groups of visitors. The study by Marty shows that the general public uses only the functions when they create their own collection, the more sophisticated functions might not be appealing for them [65,66]. To ensure learning in these online and virtual environments, personalization seems to be important. Moreover, specific activities, tools and possibilities should be intuitive and made explicit.

A simple way to facilitate prolonged interaction with the material is the creation of smaller dedicated digital collections that focus on specific groups of visitors and create activities that set specific goals for interaction with the material. Examples of such activities can be found in digital storytelling projects that are initiated in several museums. In these initiatives visitors were provided access to (sub)-collections and background materials. These materials can be used by visitors to create their own digital story. Story telling stimulates the visitors to actively explore the collection, and form personal connections to the objects and contents. This positively affects visitors appreciation of the objects or art.

Other initiatives like guided exploration scenarios can be designed for specific sub-collections. By actively asking students to compare objects from the collection, using technological tools to zoom into, or focus on specific characteristics of objects, visitors can look for similarities and differences. This allows visitors to take on the role of a researcher and actively obtain knowledge about important characteristics.

3.4. Participatory Design

3.4.1. The Roots of Participatory Design (PD)

Participatory Design (PD) is intended as an approach to design in a collaborative way technological products and services, together with the people who are their direct recipients. PD has its origins in the manufacturing and tertiary sector, as a need for workers to improve their working conditions, starting from the self-determination of the technologies they use daily to perform their tasks (Simonsen Robertson 2013), (Asaro 2000), (Ulrich et al. 1983) [68,69,70]. From its origins in Scandinavia in the 1980s to the present day, Participatory Design has evolved beyond its geographical and thematic contexts, initially in working and unionized environments, such as printing factories (Ehn, 1983), (Ehn, 1988) [71,72] and hospitals (Bjerknes-Bratteteig 1985), (Bjerknes-Bratteteig 1988) [73,74], and then in different areas such as local governments and communities, characterized by different factors, such as age (children, adolescents, elderly), origins (natives, immigrants, refugees), or interests (profession, active citizenship) (DiSalvo et al. 2012) [75]. Participatory Design has historically moved from the secondary and tertiary sectors towards the public sector, defining the new research area of Public Design that includes also museums and heritage sites (Le Dantec 2016), (DiSalvo et al. 2012), (Manzini-Rizzo 2011), (Light-Serravalli 2019) refs. [75,76,77,78].

Historically, Participatory Design approaches museum exhibitions and cultural heritage sites with the aim of preserving and transmitting the knowledge and cultural heritage of a community. Several research works have been conducted in the creation of digital artifacts to improve the visitor experience of the cultural heritage that is offered by an exhibition or collection, mainly focusing on museum environments (Avram et al. 2019), (Smith 2013), (Dindler et al. 2010), (Macchia et al. 2015) [79,80,81,82] or on the preservation and transmission of local cultures (REF. Heike Theophilous’s, Bidwell’s and Brenton’s works).

Nevertheless, the engagement of PD methodology and heritage is very much alive and in an exploratory era, as Zuljievic pointed out with a workshop at the Participatory Design Conference in 2018 [83].

3.4.2. Participatory Design and Accessibility in Cultural Heritage

As Elisa Giaccardi pointed out: “Heritage is everything we value and want to pass on to the following generations. These things are usually objects, places, and practices that we use to tell stories about ourselves, and play a central role in shaping our sense of identity as individuals and as communities” ([84], p. 17), cultural heritage is much about the intimate connection that a community has with its own past. Ciolfi et al. [85] profoundly highlight this aspect, showing how entitling local population to engage with the co-design of cultural heritage sites and exhibitions, is also a cathartic process of cultural identity re-discovery and re-shaping. Here we will talk about heritage as a macro area including also museums [84].

The papers found on Scopus library with the query [access* and inclusiv*] were 32, where 5 of them were false positives (proceedings and non-related papers), which narrowed down the batch to 27 papers. However, it is worth mentioning that other interesting papers are outside this query because Participatory Design does not always talks in terms of accessibility when it works with minorities such as children, older adults, or teenagers.

With a more extensive query: TITLE-ABS-KEY (“co-design” OR codesign OR “participatory design”) AND TITLE-ABS-KEY (museum* OR heritage), the papers were 233. In fact, the PD design approach consist on starting a project from the participants (perspective users or not), exploring, eliciting and investigating their points of view about what a museum experience or an accessible and inclusive heritage site might signify to them. The difficulty to entangle the PD research about accessibility and inclusiveness in heritage sites reflects its critical approach to include different points of view, and with them, different languages. In fact, in PD, using the word ‘user’ is not the same as using the word ‘participant’, as the former does not include the collaborative and empowering aspects that are essential in PD. In the same way, using the word ‘accessibility’ refers to an institutional language that in the normative discourse implies a disability. For this reason, the query used for the review is not exhaustive and does not catch the richness of the semantic meaning of accessibility and inclusivity in PD, which is context-based.

However, some papers found with the original query offer novel viewpoints to approach accessibility with participatory design processes. For instance, Cumbo and colleagues [86] introduce the role of least-Adult, a concept borrowed from the sociological literature on childhood developed by Nancy Mandell, as an approach to engage with children through participatory research in adult-led contexts such as museums. This approach empowers children to take action and puts accountability in the design process of new technologies and services to enjoy CH, tackling the prejudice of not being “able enough” because of the young age. Since PD recognizes that design is contextual, and the context (social, spatial, economic, political, etc.) is the first feature to understand, similarly, the least-Adult concept recognises the influence of places in shaping child participation, “pointing to spatial-temporal contextual factors as important factors for enabling and shaping participatory research” [86].

Another example that highlights the importance of understanding the context in which some cultural heritage sites are located, is illustrated by Gonzalez and colleagues [87]. Gonzalez and colleagues investigate the potential of participatory design to improve urban spaces in the slums of Caracas, Venezuela. Since the administrative and political situation in Caracas is as critical that eradication of slums becomes impossible, the researchers shift the focus from the institutional powers to the people that live in the slums. They tried to involve their inhabitants in a Participatory Development project, with the goal of improving their living conditions. The needs that the participants expressed, such as having playgrounds for the kids, green areas and community centres for all, are intimately connected with the desire to preserve and pass on their cultural identity. As the authors conclude, “shifting the methodology from slum eviction to slum upgrading fosters inclusion and respect for local heritage and culture, and creates a more sustainable and human-focused approach that aims to provide solutions to an on-going, deep-seated challenge” [87]. The in-depth understanding of CH contexts can bring tremendous benefits to the design of services that aim at making these CH sites more accessible for all, including the people that live in their surroundings, as also Ciolfi et al. highlighted [88].

Moving forward with the extensive query on Scopus above mentioned, key research of the field was highlighted. Although not explicitly referring to accessibility and inclusiveness, these research works concern making cultural heritage sites and museums more inclusive for a broader audience.

For example, Avram et al. [89] conducted a long-scale long-term (4 years) collaboration with cultural heritage professionals. Together with Avram’s team of designers and developers, they co-designed the meSch Up kit, a tool for rapid creation of smart cultural heritage environments. The kit is composed by an authoring tool, a plug-and-play hardware and software platform, designed to support museum professionals to create their own interactive installations. The purpose of this project is filling a gap in the exhibit creation, which is traditionally made by specialists, while museum curators are involved only as informants at most. The benefit of the Mesh kit is to empower museum curators and staff to independently design interactive exhibitions in their workplace where they can focus on their professional knowledge in cultural heritage, without the need of having technical knowledge on, e.g., programming languages [90].

Dindler et al. [91] entangle cultural heritage accessibility taking children’s everyday engagement as the starting point of design inquiry. They involved a primary school class in a co-creation workshop on designing future exhibitions, called Gaming the Museum. Here children are asked to contribute to the ideation of new interactive exhibitions by exploiting their own knowledge and experience about video games and online communities. In this way the perspective users (children) could shape new possible exhibitions in ways that are familiar and appealing to them.

With the project Digital Natives [92], Smith et al. investigated how to create cultural heritage innovation by combining elements of Participatory Design with issues of contemporary digital culture. Involving a team of anthropologists, interaction designers, and seven teenagers, they studied the role of digital devices in teenagers’ everyday, discovering how digital devices are more than possessed objects, they are belongings that shape and nurture teenagers identities. Giving to the teenagers the role of curators and of co-designers, they were able to express their point of view about the accessibility of museums by designing themselves an exhibition with technologies and contents considered relevant and interesting for them. The digital natives exhibition was a digital exhibition about the life of the teenagers that participated in its design. It was formed by four installations that experimented visual and aesthetic communication, allowing the audience to explore different aspects of the natives’ lives: Digital See, Portrait, Google my Head and DJ Station.

4. Accessible Cultural Heritage through Explainable Artificial Intelligence

AI as a tool is ubiquitously being applied to many domains, art being one of the latest ones being touched. As one of the many applications of AI, art can benefit from AI latest advances because the latest impressive results in AI have mainly focused on images (followed by text), as data source types. However, assessing beyond natural images can serve AI to bullet proof the latest methods on a more challenging domain as it is artistic images or paintings.

In this section, we take cultural heritage as a use case to present a conceptual framework for accessible museum experiences. In particular, we advocate for including minorities as special user and evaluator of the latest AI techniques, with an emphasis on some of the drivers to accessibility, concretely involving AI, eXplainable AI techniques and principles for accessible and responsible AI [93].

The European Commission Ethics Guidelines for Trustworthy Artificial Intelligence (AI) [94] and Responsible AI principles [15] advocate for lawful AI technology that is, among other things, more inclusive. EXplainable AI (XAI) aims at making state of the art opaque models for generation of rationale explanation, i.e., an explanation that has as target the non-technical users.

The latest AI and XAI techniques [15,95,96,97,98] could bring art closer to new audiences. By increasing the accessibility of cultural heritage to collectives not fully able to enjoy it today, missing gaps in technology could be identified. The latest advances in natural language processing (NLP), computer vision (CV) and XAI could disruptively innovate the ways in which we teach, learn, and approach art to society. For instance, people with visual impairments take and share photographs for the same reasons that sighted people do, but as they find many more difficulties, methods have been developed to assist blind photography (including audio feedback that facilitates aiming the camera) [99].

Automatically generating descriptions helps visually impaired people better browse and select photos based on human-powered photo descriptions and computer-generated photo descriptions. Could such human computation-generated visual explanations also help completely blind users, e.g., navigate a museum? Could these help any user that wants to learn from first-hand experts how a given artwork is interpreted, or what it conveys, providing the right context of its time? If the answer is positive, perhaps a model could be trained with all generated data to avoid the arduous task of labelling data so that eventually, the blind would not require human assistance.

In next subsections we put ourselves in the shoes of particular collectives such as the blind, or the deaf, and pose a set of settings we consider worth exploring in the intersection of art and science, taking into account the previous literature results in the domain coming from the three sub-domains. In particular, we propose using cultural heritage as a playground for (X)AI, and suggest a list of challenges and research questions (RQs) showing why inclusive art needs XAI, and why XAI may find on minority audiences, the right manner to evaluate where AI can have more impact. A summary figure of our research agenda is presented in Figure 1.

Given an audience, aneXplainable AI(XAI) is a suite of machine learning techniques that produces details or reasons to make its functioning clear or easy to understand [15]. XAI draws insights from Social Sciences and the psychology of explanation, and its objective is to (1) produce more explainable models maintaining high level performance, and (2) enable humans to understand, trust, and manage the emerging generation of artificially intelligent partners.

Given the inherent subjectivity of an explanation, current discussions advocate for rethinking interpretability, involving the audience expertise. When AI becomes ubiquitous across domains, it is especially important to follow the EU Ethics Guidelines for Trustworthy AI [94], Guidelines for Responsible AI and interpretable AI models [15]. Equally important is accounting for interests, demands and requirements of the different stakeholders interacting with the system to be explained. In cultural heritage contexts, accounting for the target audience is equally critical from both evaluation and personalisation points of view [100].

4.1. AI, Creativity, Storytelling and Audience Engagement

Groups of visitors inside museums have been a focus of ongoing research for a long time [101,102,103]. Some systems allowed for visitor collaboration by supporting shared listening or leaving messages between visitors [102]. To facilitate the process of engagement and collaboration between co-visitors, narratives are often introduced in museum contexts. Narratives are responsible for mental immersion through which users can be engaged and involved in the experience, increasing their sense of mediated presence as well. Visitors preferences have been studied [104,105,106], and more engaging approaches have been proposed for stimulating the visitor interests by using presentations such as film or drama [45,107]. The drama was adapting to the visitors so that different available independent drama segments were played to be group based on characteristics of the group of visitors, the specific context of the visit, and implicit input from the visitors themselves. Results showed that drama, when designed for small groups, and combined with the raw emotion of onsite visitors being in front of actual original artifacts, can emotionally engage distant visitors with mobility constraints [108].

Approaching art to different audiences should consider the target culture and background. Culture traditions can disruptively change the idea of a museum activity since early ages. For instance, opinion towards museums can be seen by kids very differently. A great example is how kids loudly enjoy and see museums as a fun place for kids when allowed to paint and talk inside (as in UK National Gallery). The idea of museum becomes that one of a ludic place, transmitting the idea that art can be a fun activity to play with. Such context makes kids at ease to approach and feel curious about heritage, leaving room for creativity. A very different idea of art is what often is formed in children when museums do not allow touching, loud speaking, nor interaction, linking the idea of museum more to a temple, or an activity that many may find boring.

Studying mechanisms to bring closer the artistic heritage to a target audience shows that, in art, the audience plays a central element, and can change the vision of society towards art dramatically. Likewise in XAI, not placing the audience in a centric role risks AI losing its deserved trust.

To renew the ways of thinking about art, we pose the following questions:

• Could AI help deliver art, personalize or write new rules on what is possible to do with cultural heritage?
• Can XAI help encode prior knowledge of domain knowledge experts [109]?

4.2. Explaining Art through Language

Unlike math, art may not always be understood, and may require extra (objective and subjective) interpretations to be able to effectively convey its message. We believe art and the story accompanying it could be made more widely understood if they would be more easily accessible. If AI models can assist the generation of content- and interpretation-wise explanations, art can be more widely understood and accessible.

One difficulty to convey the style of art eras consists of the ability to express what that era meant. E.g., Renascence’s works show people’s joy, elegance, etc. AI not only should recognize the style but also the spirit present in the era. For instance, given Venus Birth, how is to be understood the Renaissance period? How to understand the ideas and spirit of the time? What was the intention of the author? XAI may be a well-fitting candidate tool to help this objective, being a catalyst for on-demand interfaces to truly adapt to every active audience.

• Could XAI exhibit the level of detail and engagement required to effectively convey a style, or the spirit represented in the times of an artwork?

4.2.1. Image Captioning Models

Image captioning models [110] produce a text describing the scene given an input image. With the aim of producing clarifying explanations on why a particular image caption model fails or succeeds, recent strategies make sure that the objects the captions talk about are indeed detected in the images [111,112]. Textual explanations [15,113,114,115,116,117] can also contribute to make vision and language models more robust, in the sense of being more semantically grounded [118,119,120].

Since image captioning models pretrained on datasets outside the art domain fail completely at describing out of distribution inputs (e.g., pictorial compositions not found in natural images), some metrics evaluating the semantic fidelity of the model have been devised [111]. These call for models more semantically faithful to the input information, in order to reduce the bias that image caption models suffer [121], as well as object hallucination. The latter is a well-known phenomenon where image captioning models caption an image with objects not present in the image [122].

Captioning models including sentiment have also been developed [110], either using the viewer’s attitude and emotions towards the image [123], or including emotional content inherent to the artwork image [124].

• Could (X)AI explain art?
• Could (X)AI distinguish among a) content vs b) form explanations?
• Could (X)AI produce a) content and b) form explanations?
• Can XAI explain a given artwork in terms of the underlying influencing artistic styles?

The above RQs highlight the challenge of synthesizing figurative sense (interpretation) vs literary sense (content) explanations of an artwork.

4.2.2. Visual Question Answering Models

One of the advances in the NLP models brings also the possibility to produce explanations about an image though visual question answering [125], which in our view can be especially useful for the blind. To enrich the experience of a user when observing art, an advisable interactive introspection explanation could be: Pay attention to where the light is set in this painting. What is the center of focus the author is highlighting as such? Why?

• Could art explanations be generated interactively on request, i.e., using visual question answering (VQA)?
• Could advisable explanations increase the engagement and interest in artwork?
• Should only objective or also subjective information be part of an artistic explanation?

4.3. Art and Robotics

In terms of accessibility, technological advancements have brought “telepresence” or mobile remote presence (MRP) systems as another opportunity for bridging social and spatial barriers for people with mobility constrains. MRPs are designed to be teleoperated and are used to improve communication between individuals. They were found to have the potential to assist challenged individuals in instrumental activities of daily living as well as to foster social interaction between people. A number of qualitative studies where people with mobility constrains used an MRP system identified benefits for the participants such as being able to see and to be seen, reducing costs and hassles associated with traveling, and reducing social isolation [126].

Robotics can be thought of as delivery means for art explanations. For instance, a robot can sense when the group he is leading in Seville’s Alcazaba tour is getting bored, and change, e.g., the length of its explanations based on the movement of the visitors [127,128]. In this context, it is worth investigating the utility of such robots in terms of:

• Do remotely operated mobile robots increase virtual visits to a cultural site, with respect to static browser-based virtual tours?
• Do robot guides [127] improve the visitors rating when no human guide is available? Is their user experience rated better than walkytalky guides?
• Can AI provide guide explanations that reduce the boredom of the visitors?

There could be a value in having a AI-empowered robots visiting together the cultural heritage site with the humans as well. One potential application and advantage of using robots and AI in cultural heritage is with respect to language: e.g., a robot like C-3PO that speaks all languages can make the tour anytime in any language, including sign language. This has a value with respect to a human tour guide and can be seen as a next step in innovation in the field of guide systems, as the incarnation of audio guides.

5. Discussion

Most systems identified in this review were applications that promoted exploration of physical of virtual places. From our literature review it seems obvious that the landscape of cultural heritage technologies is expanding, especially the demand for technology providing inclusive and accessible museum experiences.

The overall usability was largely the primary interest of past evaluations of interactive systems for cultural heritage. This was expected by the authors, as an important goal of any interactive system and a prerequisite to pursue deeper user impact, is to first evaluate usability and UX aspects. In addition, we presented studies that have considered pedagogical and participatory dimensions, equally important in our view to provide a holistic view on what constitutes human-centered approaches for designing accessible and inclusive CH experiences in first place, and what are the prerequisites to design such future CH experiences empowered by AI, and in particular by XAI.

Interactive systems in cultural heritage fields employ diverse contemporary technologies already, and to some extend they already use AI, and this is very interesting to see. However often practitioners and researchers in the field forget the cultural heritage site context and the value for the visitors it brings, and many times they focus solemnly on the technology [129]. The virtual accessible experiences should be formed around the themes of the museums and the museum objects that the technology needs to convey, and this danger is especially important to stress in research lines such as ours. We should not forget AI and HCI as tools to help future design of AI solutions for accessibility of cultural heritage, having the human at the center.

These questions, as proposed in our paper, are not easy to answer, especially in short term evaluation approaches, as they will require cross-pollinating efforts. Thus, there is still much theoretical work to be done on the dimensions of work presented in this paper. For instance, key research questions yet to be tackled are domain-broad. Interesting examples initially answering some design questions include interaction design approaches to work with social robots in schools [130], how a child learning mindset can change with feedback delivered by robots [131], or how its critical reflections on AI can be enhanced [132].

A great example of efforts joining all HCI and AI disciplines described for a joint synergy is the core expert group set up by the United Nations Children’s Fund (UNICEF) (https://ai4children.splashthat.com/) or the World Economic Forum Generation AI: Developing Artificial Intelligence Standards for Children (https://www.weforum.org/projects/generation-ai), which are working on convening regional consultations, including children for diverse input into policy guidance. Their report on AI [133] equally advocates for the needs for policymakers to collaborate closely with technical researchers to investigate, prevent and mitigate potential malicious uses of AI. In addition, this is just a target group, children, but many other minorities need special attention too.

AI systems should be user-centric and designed in a way that allows all people to use AI products or services, regardless of their age, gender, disabilities or characteristics, as advocated by the the EU Ethics Guidelines for Trustworthy AI [134]. All the guidelines for trustworthy AI also claim, as we do in this paper, that accessibility to this technology for people with disabilities, which are present in all societal groups, is of particular importance. The assessment is based on seven key requirements: namely human agency and oversight of the system designed; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; environmental and societal well-being; and accountability. The one suggesting how to deal with users with different disabilities is the non-discrimination and fairness requirements of the framework. Future designers should follow them closely to make sure that cultural heritage solutions follow this guidelines.

Modern AI solutions try to put the user in the center by creating personalized experiences and support, to do so they collect data about users’ behavior and preferences, and combine data-sources. An example of this is for example sensors collecting psycho-physiological data measuring arousal, or the use of devices that track the route of a visitor through an exhibition, combined with personal data about preferences and the users’ background. While museum staff might also interact with visitors and try to make meaningful experiences for each visitor, this is more transparent and often is shaped in interaction with the visitor. In contrast to this, the algorithmic solution often remains a black box for the visitor and the museum staff. Therefore, it is important that the software is as transparent as possible and that the personal has sufficient understanding of the algorithms to make informed decisions. A concern is that responses that are created, for example by machine learning algorithms, might be very robust, but also are more likely to lack the human touch. Another challenge is related to AI solutions and to the use of personal data. In AI automated decisions are made based on a variety of data sources including personal data, this might lead to issues related to GDPR rules and regulations. This issue is specifically challenging when working with younger visitors in educational contexts, since they might need consent from a parent or authorised caregiver. So in these cases developers might opt for data sources that do not use personalized data and make sure that no re-identification is possible when sources are combined This all is related to privacy issues, but also sensitive for bias and discrimination (profiling).

As AI technologies can amplify existing gender and racial biases in our society, they can also exhibit disability-based discrimination [135]. A recent study found that people who were blind were over-trusting of an AI image captioning system, even when the output made little sense [136], or for instance speech recognition systems, which are used heaviliy by virtual assistants, do not work well for people with speech differences such dysarthria, deaf accent, etc [137]. Developing application of AI to inferring disability status is a crucial issue in this respect.

5.1. A Call for a Multidisciplinar Collaboration

The presented challenges aim at stimulating a call for collaborators in a joint effort to mutually learn from other domains, and form an interdisciplinary research consortium aggregating a diverse set of collective and symbiotic needs:

• Art historians: can gain visibility by making art accessible, building a portfolio, e.g., as gallery guides, art podcast content generators, etc. Humanities students could better learn by teaching their lessons outside humanities and generating AI-consumable data.
• Artists and story tellers could earn an audience willing to learn about a niche passion.
• Disabled individuals and minorities: Blind audiences could get access to art explanations through audio or text resources, the deaf through the latter’s transcriptions.
• Computer scientists would use the generated data to build robust machine learning models that (1) explain art, and (2), are explainable, (3) learn by the right reasons [138,139].

The ultimate aim would be that all content would facilitate anyone to understand any art with the right context.

5.1.1. Impact of AI on Humanities Domains

We envision a set of domain areas where the symbiosis among art and (X)AI could be further exploited. In order to guarantee Responsible AI guidelines [15], for instance, the audience could be informed of the provenance specification of AI training data. An example of guideline is always informing when a content unit is original or has been algorithmically generated.

Recommendation systems and personalization services may optimize matching art-broadcasters and art-listeners, and suggest new artworks likely to be appreciated by a given public.

Educators and developmental psychologists could find in XAI a support tool to convey humanities, social sciences and history in terms of the alignment of explanations with the cultural background mental model of the learner. Music understanding and interpretation could also benefit of tool development to equally be explained to who cannot hear. Likewise, musicians could express what their work represents, to make anyone (including the deaf) hear what they cannot.

After all these technologies are put into place, and human in the loop machine learning systems have gathered enough data, a new wave of creative AI algorithms will emerge. All byside data generated through Human-Machine collaborations involving the stakeholders above could train AI models to capture the underlying generating factors that make humans interpret art the way they do.

One may argue that any song, painting or sculpture could transmit a story, perhaps without the need of having an author expressing it with words, because art communicates what words cannot express. However, since making cultural heritage accessible demands the possibility of making different modalities available on demand, we argue that language is the universal exchange vehicle to translate art from one sensorial medium to another.

Despite the fact that using language to express art poses a challenge in terms of constraining its expression, we believe language to be the closest mean for universal communication. We expect art expression through AI models to be one more form of intermediate language (e.g., see [140]) for accessibility, allowing singing a painting, or drawing a song.

5.1.2. Tackling the Lack of Personal Touch in Technology

During quarantine/crises, diverse cultural agendas are made available for free (operas, museums, virtual tours, circus, libraries, etc.). At-home vs on-site experiences can degrade the experience of culture, perhaps due to lacking the social touch involved in the original experience. Human computation, art-history and humanities expertise on the approach to such cultural offer could not only serve the purpose of bringing art home, but also set the basis for future ML models that could generate personalized explanations about a given artwork. A hypothesis is that museum experiences require of a personalized, social or physical involvement experience in order to maximize the inherent pleasure of enjoying cultural heritage sites, with everything that it conveys.

ML algorithms generate sketches [141], steerable playlists [142], music [143], and incite creativity through editing tools [144].

Auditing AI systems and informing the end-user of the nature of the content (natural or artificially generated) is part of the guidelines for responsible AI [15]. Since machine discriminators outperform humans in detecting generated text [145],

• Could AI recognize XAI generated explanations better than humans?
• Can human testimony personalized art explanations stimulate engagement and discovery of art by society?
• Could artist voice note explanations uplift the lack of social touch in traditional virtual/ audio guides?

We hypothesize that the digitized artwork personal (human) reviews can enrich access to cultural heritage based on artists audio/transcriptions, making it available to any art consumer, including the deaf and the blind.

5.1.3. Evaluating AI-Generated Art Explanations

• Is AI being evaluated in the right tasks and with the right audience?
• Can we evaluate AI generated text explanations’ quality in a quantitative manner that is both user questionnaire-free and audience-specific?

In order to assess the quality of a story, word embeddings can be used to estimate cognitive interest [146,147,148]. Fashion styles and its social media tags can be used to predict subjective influence and novelty [149]. Could such influence and novelty metrics correlate with actionable or useful explanations?

• Can social media metrics such as likes correlate with an actionable or useful explanation?
• Could AI explain what makes an artwork appreciated or liked?
• Could we quantify the amount of surprise or originality it conveys?

It is as much as important to also define explanation standards. General XAI techniques usually evaluate XAI techniques on their ability to generate visual or textual explanations [15]. However, the requirements to evaluate an explanation positively by a blind or deaf person are likely to require very different criteria.

• Can we define standards for XAI explanations, including those subject to subjectivity?
• Can we always provide automatic satisfying answers when the observer is unable to see/ visually impaired?

Explaining concepts that are hard to visually grasp is another dimension on which future research will need to focus on.

A single format may not fit to convey all art modalities. At times, some modalities, e.g., sound, may be a better format to translate into. However, visual-textual semantic embedding [150] and retrieval [151] is possible; in the latter case, without labelled cultural heritage data, thanks to transfer learning.

If what is essential is invisible to the eyes, as Antoine de Saint-Exupéry said, symbols such as words or knowledge graphs could act as intermediate proxy representation to verbalize complex abstract concepts [152].

5.1.4. AI as Vehicle to Human Creativity

Historians can argue that humanities education can currently abuse the use of images to teach. This is demonstrated by the success of an influencing teacher’s podcast that prepares for History university entrance exams. While the use of words stimulates the imagination and keeps the mind working, providing an image to explain the same concept keeps the mind static and inactive. This is why teacher Pleguezuelos points to the images corresponding to the podcast explanations only after students had to imagine the described period, era, or artwork, exclusively with words. Could a machine learn the same way? Could it reinforce the knowledge through later confirmation with a different learning modality?

• What is the key role that AI can play in bringing heritage closer to the viewer?

An artwork can inspire our mind if we are taught in what epoch it was represented, and in what context it was created. If AI models could ever be powerful enough to make us re-live that era, the inspiration they transmitted, and even imagine the spirit of the age,

• Could AI destroy the creativity of the viewer, that part that inspires the audience?

We argue that since AI can learn from a multimode of inputs, it can provide interesting analogies or links to other artworks that a human could not do. XAI techniques should explore ways in which AI could be not a threat to the development of creativity that the artwork itself implies, but rather a facilitation medium that suggests questions, allows exploring unknowns, and further stimulates scientific curiosity and hunger for knowledge. In this context, artificial models of computational curiosity [12] could align with those of humans, teach humans [153], or guide the latter to improve its mental model, trust, and curiosity [154].

5.2. Challenges for AI in Cultural Heritage and Input for Future Research

Debates discussing the abilities of computational creativity involving scientists and humanities can results in fiery discussions. An example can be seen in the debate Computational Creativity: Art through the Eyes of Computation (panel arranged by N Díaz-Rodríguez & S Tomkins, Data Science Santa Cruz initiative, including art historians, computer scientists, musicians and humanists http://ihr.ucsc.edu/event/quantifying-creativity-art-through-the-eyes-of-computation/, Video: http://travellingscholar.com/qcreativity/). In this section, we summarize hypothesis and research questions into challenges, discipline dimensions affected and concerns to address such challenges in

Table 2. Challenges to tackle to leverage (X)AI for Cultural Heritage Accessibility.

Challenge	Dimensions	Concerns
Augmenting accessibility to minorities or users	Interface and content personalisation,	Inclusion, AI Ethics
with physical and cognitive disabilities	Generative and Multimodal AI
Explaining art with AI	Human computation, Human in the loop,	Trust, Responsible AI
	[Multimodal, Generative] AI
Each museum has a particular story: it is a challenge for AI to convey it to different audiences and minorities	Human computation, Multimodal respresentation learning	Shifting focus of attention into Technology rather than specific CH content
Technology that is comforting and motivating	Human computation, Engagement, Artificial Curiosity and Intrinsic Motivation	Do not compromise privacy, security and wellness
Cross language barriers between AI and non-AI experts involved	Participatory design, eXplainable AI, HCI	Adapting model outputs to the right kind of audience
Guides and educators	Pedagogical challenges	Guides and educators may stay without jobs or may not now how to work with AI tools
Gaining AI trust and adoption	EXplainable, Ethical and FATE (Fairness, Accountability, Transparency, and Ethics in AI.)	Interactively adapt a model output + interpretation to the right audience. Assure Responsible AI.
Creativity as research engine, AI for content synthesis	Engagement, Curiosity, Computational creativity, XAI	AI Storytelling and Trust, Subjectivity, Fair and objective XAI Metrics.
Job market shift needs for more specialized technical expertise	Interdisciplinary research and communication, XAI	Hand in hand work of curators/educators + technologists (e.g., back up adaptive curriculum learning with specialist educators).

. We presented some disruptive art settings as motivating examples where AI and XAI could have novel research playgrounds to validate models. Since concerns involve fairness, accountability, transparency (FAT) in ML, we gave a first step listing questions that need to be addressed to obtain insights on how AI can best help accessibility to audiovisuals.

Despite having presented challenges and opportunities focused on how AI (and robotics) can help access cultural heritage and the digital humanities, from the point of view of AI, this is just an application domain where the limits of current AI models can be stress-tested.

It is of utmost importance that future work looks beyond the horizon to establish a common framework on which diverse disciplines within AI and HCI can meet; Pedagogical Design, Interaction Design, Participatory Design, eXplainable AI and Robotics need to make sure a common language and lingo interface are established to allow different communities understand each other, and produce versatile and feasible models and interfaces. Assuring the end-to-end deployment will require users, developers and decision makers as human in the loop to facilitate the continual learning and evaluation of both systems and humans.

6. Conclusions

The user centered nature of accessible CH, the advances in technology, as well as the needs for user adaption call for carefully designed activities to attain accessible CH. A user-centered accessible CH requires an interdisciplinary approach combining views of design-based research, instructional design and human-centered AI, in particular, having explainable AI as part of a ubiquitous new curricula [16]. The contribution of this paper consists of an ecosystem of user-centred design approaches to the XAI discourse, with cultural heritage as case study. If we were to implement all these technologies in a wide-accessibility fashion, we might need to touch issues such as potential ethics concerns in AI, and how a human-centred approach can contribute without becoming its watchdog.

We summarized hypothesis and RQs into challenges, discipline dimensions affected and concerns to address such challenges in Table 2. We presented some disruptive art settings as motivating examples where AI and XAI could help accessibility for cultural heritage sites, and provided concrete examples of such future possibilities as well as challenges that will raise from such use of AI and XAI. Since concerns involve fairness, accountability, transparency in ML, we gave a first step listing questions that need to be addressed to obtain insights on how AI can best help accessibility to audiovisuals.

Despite having presented here challenges and opportunities focused on how HCI, AI (and robotics) can help access cultural heritage and the digital humanities, CH is just a perfect application domain where the limits of current AI models can be stress-tested. The existing challenges to attain explainable and responsible AI in any real-life problem are equally relevant and should be explored far beyond, especially in practical applications of AI safety and AI for social good (from elderly telepresence robots [155] to epidemic and hospital crisis management [156]).

At last, cross-pollinating examples lead by national and global organizations are finally raising concerns regarding the design, development and use of AI in applications. For instance, the fundamental human rights perspective to designing and developing better AI by UNICEF [157] and robots for children [158,159] should be extended and adopted by other minorities to regard inclusion and accessibility as key target objectives to attain. A human-centered approach to develop AI systems that protect and empower minorities should be implemented according to legal and policy frameworks regardless of the minority’s (i.e., children in this example [157]) understanding of the system, and AI-related regulatory frameworks to integrate (child) rights should be reviewed, updated and developed. This way, we encourage regional and local administrations to replicate these model initiatives and extend it to different services in society.