Museums and Digital Culture: From Reality to Digitality in the Age of COVID-19

Abstract

Museums increasingly recognize the need to address advances in digital culture which impact the expectations and needs of their audiences. Museum collections of real objects need to be presented both on their own premises and digitally online, especially as digital and social media becomes more and more influential in people’s everyday lives. From interdisciplinary perspectives across digital culture, art, and technology, we investigate these challenges magnified by advances in digital and computational media and culture, looking particularly at recent and relevant reports on changes in the ways museums interact with the public. We focus on human digital behavior, experience, and interaction in museums in the context of art, artists, and human engagement with art, using the observational perspectives of the authors as a basis for discussion. Our research shows that the COVID-19 pandemic has accelerated many of the changes driving museum transformation, about which this paper presents a landscape view of its characteristics and challenges. Our evidence shows that museums will need to be more prepared than ever to adapt to unabated technological advances set in the midst of cultural and social revolution, now intrinsic to the digital landscape in which museums are inevitably connected and participating across the global digital ecosystem where they inevitably find themselves entrenched, underscoring the central importance of an inclusive integrative museum model between physical and digital reality.

1. Background

When COVID-19 forced itself into our lives in March 2020, it sent shockwaves across the globe; suddenly we faced “lockdown”. We said goodbye to the way it was but did not understand what this brave new world of isolation and separation would mean and how it would impact life as we knew it, including our identity, relationships, and the freedoms that we enjoyed. We wondered what daily routine post-COVID-19 would look like, while the experiences defining life itself were for the taking. With social distancing, masks, and work from home mandates, the arts and performing arts, from theaters, museums, galleries, and the public square, were shuttered. Their very existence was challenged and spiraling out of control as staff were laid off, exhibitions cancelled, while concurrently creating an urgency to go online to dwell in cyberspace, the new daily destination.

The crisis of the pandemic now converged with critical sociocultural and technological advances in AI, robotics, and computing, questioning human identity and purpose, trends that had already intensified by summer 2020. As the world sought to navigate through this unprecedented and life-changing disruption, the rising digitality of our states of being pushed us evermore toward virtual life. With remote work, we witnessed the emptying out of the public square, leaving a lacuna of human activity and social interaction. The social forces, already at play, of diversity, equity, and inclusion, entered this deserted space and filled it with protest fueled by social movements, most prominently Black Lives Matter. This paper explores these issues through the lens of museums and cultural life, especially in New York and London, where art pervades the fabric of life, to be heard and seen on city streets, in galleries and museums. We consider how arts and cultural life have evolved during this pandemic period and how museums might survive during a time of change, uncertainty, and the social forces influencing human identity and relationships.

Well before the pandemic, museums had already made a paradigm shift from collection-centric to user-centric in the context of human participation and interaction. Significantly, when COVID-19 struck, there was little of the digital to be seen in museums, and even digital art was mostly excluded. Suddenly faced with the COVID-19 crisis, museum doors shuttered, their visitors gone—and so was their main source of revenue.

Libraries also closed, but the stakes were not the same, as their main menu, the book, is well-served by the digital book, a great surrogate of growing popularity for the physical book, surpassing it in many ways. In contrast, the material collections of museums, their main raison d’être, is defined by their physicality, for example, size, shape, color, texture, materials, and the spaces they occupy in galleries, all of which are central to how we experience an artwork. The longer COVID-19 continues, the more human behavior adapts to altered states of being digital, while the void of the public square, a place where people gather and socialize, shrinks. We find ourselves in retreat from the life we knew, and when the pandemic ends, that life that once was will end, too—somewhat of a Rip Van Winkle effect, as big tech and big government occupy the void. What will happen to the public places we abandoned during COVID-19, and what will motivate people to come back, and will museums attract new audiences?

Enter digital artists, designers, and museum curators—those who know how to use the digital to make art and engage audiences, to tell stories and be contemporary as well as relevant. Digital artists, whose presence in museums seems sorely lacking, must now assume a more central role in the digital integration and visualization of human digitality. While information and technology services can be outsourced or remote, curators remain at the heart of defining collection content and context but need the collaboration of artists on new narratives and ways of thinking, which in turn enable museums to reimagine collections in ways that encourage museums to engage with more diverse communities, artists, and audiences.

2. Academic Context

This paper builds on joint work by the authors, particularly in the context of the annual EVA London Conference on Electronic Visualisation and the Arts, together with a 2019 book on Museums and Digital Culture [1]. In particular, the authors have organized symposia at EVA London conferences [2], annually since 2016, with panelists from a diverse set of backgrounds, including many with academic and curatorial experience. Themes have included the move from analogue to digital in literature and art, digital senses and aesthetics merging art and life, art and identity in digital space and time, digital art, culture and heritage, computational culture and AI challenging human identity and curatorial practice, and arts and identity between reality and virtuality in a post-COVID-19 world [3]. This symposium series initially started in association with the Pratt Institute London Summer School, with an emphasis on digital culture in museums [4,5]. Aspects of digital culture have continued as a theme, most recently considering identity in the digital arts, crossing between reality and virtuality, and the effects of the COVID-19 pandemic on developments in this area, including a museum curator and exhibition organizer [3]. The authors of this paper come from different but complementary viewpoints of information science and computer science, but both with a broad interdisciplinary interest in digital culture and heritage.

Digital transformation with respect to the cultural and heritage sectors is an active area of interest [6,7]. It is known that digital technologies can be used as tools to improve museum visitor engagement [8]. The boundary between the physical and digital world can become blurred and this can affect the operation of museums [9]. The whole field of museology is being affected by developments in digital technology [10]. Specifically for museums, immersive exhibitions are increasingly popular [11,12].

COVID-19 has dramatically accelerated the pace of change across our digital ecosystem while ushering in major advances in computing, concurrently within the expanding virtuality of art and identity [13]. These converging forces shine new light on digital cultural arts, set in the context of today’s social and cultural revolutions on a global scale, where diverse cultures converge and collide, demanding that cultural institutions reimagine professional practice and replace legacy systems, as art and human identity shifts to a beyond-human state augmented by computing and technology [3] and life is lived on platforms and networks, as our human identity morphs into a new state of being human, where the arts and human identity become one with computational culture.

Museums have necessarily adapted the way they operate and handle visits during the COVID-19 pandemic (e.g., see Figure 1). At a worldwide level, the International Council of Museums (ICOM) and the United Nations Educational, Scientific and Cultural Organization (UNESCO) have responded with surveys of museums and associated reports.

In May 2020, ICOM published a report on a survey with some 1600 responses from 107 countries, covering the period from 7 April to 7 May 2020, early in the pandemic [14]. This revealed that around 95% of museums were closed due to the pandemic, with many museums enhancing their digital activities. Over half were engaged in greater social media activity. In 84% of museums, most staff were working from home and about 16% of freelance staff had been laid off. A follow-up ICOM report in November 2020 was based on 900 responses from five continents, covering the period from 7 September to 18 October 2020 [15]. A total of 14% had some staff furloughed or laid off (around 16% during the period February to September 2020). A total of 15% had enhanced digital activities, rising to 50% with social media activities included, while 11% of freelance staff were laid off, a slight improvement on the previous report, and 31% were planning to downsize permanent staff, rising to about 46% for freelance staff. ICOM provided some COVID-19-related recommendations for museums on their website [16]. These included areas such as the conservation of museum collections, ensuring cultural heritage security during lockdown, how to reach the public remotely, supporting community resilience, and handling the end of lockdown, including the safety of the public and staff.

UNESCO has also provided two COVID-19-related reports relevant to museums internationally. In May 2020, a UNESCO report [17] revealed that 90% of museums were closed, including 10% that may never reopen. The sector reacted rapidly in developing its online presence. However, only 5% of museums in Africa and small islands had online content, representing a significant digital divide. The report estimated that there are 95,000 museums in the world, 60% more than in 2012, which is a rapid increase. In April 2021, UNESCO published a follow-up report [18]. Its findings show that 80% of museums’ revenues had dropped compared to 2019. A total of 43% reported closures in the first quarter of 2021. On the positive side, strengthened links with communities and increased intermuseum cooperation were reported. In 50% of states, public subsidies decreased for national institutions. This report estimated the presence of some 104,000 museums in the world, 9% more than the previous report the year before, although such estimates are difficult on an international scale, and yet the global aspects of the pandemic have sparked intense public debate on the future of museums, increasingly challenged by activism and protest in the arts seeking to break down cultural silos to find common ground.

The Museum Innovation Barometer 2021 [19] provides a large-scale survey of museums. This is an annual survey by Museum Booster focusing on technology and innovation in museums since 2015, with a sample size of 150 institutions, representing 200 museums spread globally. The survey period in this report covers December 2020 to May 2021. This extensive survey of museums brings evidence to bear to show that COVID-19 was the key driver of digital transformation in museums and for introducing new technologies such as AR (augmented reality) [20,21,22], VR (virtual reality), and hybrid approaches like MR (mixed reality) [23,24]. The survey confirms that these technologies are here to stay, with their use clearly on the rise, confirming our previous investigations in this area [1] and those in this current paper. However, significant challenges remain in terms of sustainability and financial feasibility.

In the area of education, there has been some success. For example, the Barnes Museum has tried online art classes that are bringing in revenue [19]. Interestingly, the report does not mention digital art or digital artists, despite the rise of the importance of digital art to art museums [25] and the financial success of non-fungible tokens (NFTs) [26]. More generally, artists are mentioned several times in the survey, but all in the context of collections, mostly from the 19th century. The value of the arts sector to governments and the public in the light of the COVID-19 pandemic has also been studied [27].

The COVID-19 pandemic has affected all museums, from the size of the Metropolitan Museum in New York City downwards [28]. A survey in the United Kingdom has studied the effect of the pandemic on the online content for 21 museums [29]. The pandemic has certainly caused changes and presented significant challenges for museums [30]. Virtual tours have become more popular in association with museums [31,32]. Digital initiatives in museums, and in particular the use of social media by museums, have increased during the pandemic [33,34]. However, it is difficult to assess the precise effects for the future [35].

A study on the impact of COVID-19 on digital data practices in the UK and USA has focused on digital preparedness for employees with digital skills [36]. This concludes that not only has COVID-19 had a short-term impact, but there have been longer-term effects as well. Although some museums may never reopen, most institutions have demonstrated significant resilience, adjusting their strategies and operational modes within the new paradigm in which they find themselves, although this is not greatly elaborated.

The area of museums and digital culture is an emerging field. To date, there has been little research from the perspective of the heritage/museum experience and human digital behavior. There is also little research on the culture of “digital culture”, and the divide in museums between art and technology remains an issue. The 2019 book Museums and Digital Culture [1] breaks new ground in the realm of “digital culture”, as opposed to considering digital technology being the driving force. It is only very recently with the advent of COVID-19 that the schism between real and digital aspects at museums is being addressed. Museums are only beginning to take an integrative approach, considering the uncertainty and dynamic nature of the pandemic, which requires heritage institutions to be more agile and seek to incorporate VR, AR, and MR, both onsite and online.

Increasingly, the aim is to form a dynamic system between these various aspects and to embrace this transformational shift from the staider nonparticipatory model, so long in place, to one that deeply engages the audience. This trend was already in motion before the pandemic, with exhibitions such as that on the rock group Pink Floyd at the V&A (Victoria and Albert) Museum in London, which ended in a large gallery with floor-to-ceiling projections while the audience reclined on the floor. This was a precursor of Van Gogh: The Immersive Experience (see later). A major trend for museums will be working in partnership with digital artists and designers, including experience design tied to human digital behavior research, as museums embrace the integrative and inclusive museum model.

3. Research Method

The main part of this paper represents an emerging research methodology reflective of computational culture, in that it integrates digital research, evidence-based and primary source digital documentation, humanities research, contemporary and relevant experience, observation, and digital capture, curation, and analysis. We examine cross-disciplinary practice, where art and technology converge, and how this will continue to transform under the increasing pressure of computational culture and attitudes in a post-COVID-19 world. Undertaking research set in this dynamic model, we prioritize first-hand experience, digital capture/curation, and diverse perspectives.

Importantly, the Internet offers a rich array of publicly available digital sources (blogs, online newspaper websites, etc.) that cross disciplines and institutional boundaries, creating new documentary juxtapositions. From these, one can construct new narratives, interpretations, and perspectives, with the potential to become part of the fabric and creativity of cultural heritage studies in a computational world. Rather than taking a retrospective view, this article seeks to look from present to future through a wide-angle and interdisciplinary view, exemplified here in the work of leaders and influencers in the museum field. The paper covers issues that have been accelerated by the COVID-19 pandemic, especially for museums and related heritage institutions. Due to the nature of the research, much of the material used for the research is in the form of online information, rather than existing journal publications. Key web references are cited where appropriate.

For this paper, the authors observed various in-person and digital online experiences relating to digital technology, culture, and heritage. These observations were sourced by visiting museums and art galleries, walking the streets of New York City, and reading blogs and online newspaper websites during 2021. The researchers discovered opportunities for observations through keyword searches on Google that featured appropriate combinations of keyword phrases such “COVID-19”, “digital culture”, and “museums”. These keyword searches enabled the researchers to identify opportunities that were trending online at the time, mainly in New York City, but also London and elsewhere. The researchers captured their observations via web page searches, physical observations, and the production of associated notes. They then reflected on the trends between digital technology, culture, and heritage, especially in the light of the COVID-19 pandemic, through their existing knowledge of the area [1].

The researchers’ perception of how each observation relates to digital technology and culture and heritage limits the discussion. It firstly limits it due to their subjective understanding and existing knowledge of the area. A second limitation is the individual perspectives in the research that limits the observations available for reflection within the survey period. This is an area of online rapid development, accelerated by the COVID-19 pandemic. These two aspects limit the research, as it is necessarily directed by the authors’ complementary expertise.

4. Observational Perspectives

4.1. A Transformational Shift to Digital Integration and Inclusion from Onsite to Online

In this section, we present a number of different aspects of digital culture, especially with respect to museums, and how they have impacted real life, particularly with respect to the COVID-19 pandemic. These provide snapshots of the increasingly digital environment in which lives are now being led. Each of the following subsections brings new consecutive insights to recent directions of the emerging integrative museum landscape from various perspectives, leading from one view to another as the research developed. These place digital culture at the forefront of transforming a museum’s relationship with its audience and community, as well as digital art and artists, marshaling the creative and innovative use of new technology such as AR, VR, and MR, all of which can support diversity and inclusion.

4.2. COVID-19—Remanded to Digital Life

Remote work and online education confines people to their homes. People retreat as cities seem lifeless and begin to die and have themselves become diseased. People begin to leave city centers to seek their own space, saying goodbye to those tall buildings with hundreds of apartments that somehow seem like prisons. Cars, like private spaces on wheels, soon to be driverless, are the refuge between here and there, as we avoid public space at all costs while public transportation looks deserted. Being locked down in our homes and locked out of real-life causes feelings of disorientation, displacement, and dissolution of life as we knew it. Seeing art, seeing life and human interaction moves to digital screens, as our sense of physicality fades, replaced by virtual experience, as when the size of an artwork becomes the size of the screen—the digital simulation of physical attributes replace the real and are lost to digitality.

4.3. Seeing Digital in Art Museums—Immersive Reality and Virtuality

Progressing through the COVID-19 pandemic, seeing the digital in art museums and in exhibitions has still not been broadly conceptualized and accepted, lacking regular space, place, and purpose, as the digital mainly serves the online presence of museums and seemingly disappears when entering the physical museum, where digitality is generally achieved through visitors’ smartphones or tablet devices. Extending physical reality in museums using VR, AR, MR, and 3D awaits broader use and regularization. Where VR has been cultivated to a high degree to enhance the visitor experience, for example, the 2018 Modigliani exhibition at the Tate Modern [37] and the Louvre’s Mona Lisa exhibition as part of a 2019 da Vinci exhibition celebrating 500 years since his death [38], it has received worthwhile results and wide acclaim. For both exhibitions, the museums partnered with Vive Arts. In 2021, a new foray into VR and integrated digital arrived with the V&A (Victoria and Albert) Museum’s exhibition, Alice: Curiouser and Curiouser, on view during 2021, featuring an AR Madhatter’s Tea Party exhibit with an animated table/background (see Figure 2a) and a VR experience (see Figure 2b) created by Vive Arts that can also be viewed in an online version [39,40].

Charles Dodgson (1832–1898), the Oxford mathematician writing under the pen name Lewis Carroll, brought together the arts, mathematics, and science to conceive The Adventures of Alice in Wonderland, one of the most enduring stories, so relevant today as it challenges the human sense of reality, time, and space, and uses storylines in Alice suggestive of his mathematical writing such as The Alphabet Cipher (1868), The Game of Logic (1887), and Curiosa Mathematica I (1888). Carroll’s Through the Looking Glass (1871), a sequel to Alice, evokes through the digital screen as if we could enter virtual life. At the heart of the V&A’s exhibition’s conjuring of Alice are curators, digital artists, and designers of the creative industries working collaboratively. The overwhelming critical success and broad public enthusiasm for the VR components of these exhibitions, as well as other digital enhancements, points to the need for museums to cultivate in-house digital artists/curators to be part of their team for exhibitions who work onsite and online in ways that heighten visitor experience while accounting for their digital behavior.

Post-COVID-19, the playing field of exhibitions and visitor experience has forever broadened its scope to include an ever-expanding range of virtual technologies that, in the first instance, expand the artistic palette for creating exhibitions and other visitor experiences serving to inspire public learning and emotional engagement, as museums tell new stories in relevant ways. The Alice exhibition does just that. While bringing new and thought-provoking interpretations, it takes the visitor on a journey spanning 156 years, beginning in 1865, the year Alice was first published, arriving at 2021, a time of global pandemic when we all, to some extent, have fallen down the rabbit hole of virtuality, a journey punctuated by a 1969 edition of Alice illustrated by Salvador Dali (1904–1989), the surrealist artist. His work highlights fantastical surrealist connections and insights into states of being beyond reality—so pertinent to the COVID-19-driven crisis of identity, time, and space, and we note Dali’s 1931 painting, Persistence of Memory, as a reminder that our perceptions of virtual time alter our memories of past and states of being in the present. The topsy-turvy world of Alice finds deep concordances with life in the virtual world. Alice leaves her real world and enters a simulated virtual world of “wonderland”, as contemporary life has fallen down the “rabbit hole” of COVID-19.

4.4. New York City Streets through a Digital Lens, Museums and Stores

Capturing the cityscape with a smartphone camera hones one’s ability to observe and capture street art, and to curate a New York experience. The visual richness of city streets tells new stories that have relevance to the new realities of what might be a post-COVID-19 world. Suddenly being juxtaposed with Greek classical beauty and mythology, but set in a 21st century urban environment, forcefully expresses the artist Jim Dine’s fresh interpretation that relates to the dynamic states of the human form and notions of aesthetics (see Figure 3). Standing on Manhattan’s 6th Avenue in the presence of the artist’s work, its large scale, buttressed by the skyscraper setting, causes one to stop and contemplate the power of art to transform public space, a museum-like experience beyond the walls.

4.5. Museums—Integrating Real and Virtual—”Mind the Gap”

When we re-emerge from the void of COVID-19 life, how will we put reality back together? The lacuna in the public square left by COVID-19 veered from occupied to empty, stranded between lockdowns and protests (see Figure 4 and Figure 5). Filling that void in a physical sense, and in our hearts and minds, presents an existential challenge to human existence—life as we knew it and what it might become. Museums are now asking, what will be a sustainable model of the future under the impact of the newly-formed principles of digitality? The book Museums and Digital Culture [1] takes up the notion of the museum beyond its walls, connecting it to its community, both local and global, merging the museum’s online identity with that of the physical museum, rethinking how museums can generate income, and what opportunities there are for partnerships, new roles for government, and philanthropy. Some initial consideration might be found in those that build on existing resources and strengths. The book devotes a chapter to museum education through the lens of the MS Museums and Digital Culture master’s program introduced by Giannini at the Pratt Institute in 2016, which illuminates current issues and looks beyond 2021 to new ways of thinking and teaching.

As AI and machine learning prevail in this moment, for the first time in human history, artificial intelligence challenges human intelligence, as humans have bowed to the machine power of the fourth industrial age. Humans and machines working together have engaged millions, if not billions, of people—but as AI invades more of the things of the mind, human thinking, imagination, creativity, naming things, arranging, ordering, organizing, and defining—what will be left to humans?—will we need to be more than human? The first to go is repetitive, make-a-day work, which is broadly embedded in our institutional legacy systems, including those for archives and libraries. At the very moment that university students turn to technology as the answer to “getting a job” will those jobs go to AI and human personification—robots? Learning to operate and write programs soon will be dominated by big tech and communication corporations, most humans will simply follow program instructions, buy the product, play the games, and watch the shows.

Art and artists are the future in a future where creativity, critical thinking, and free expression merge with science and technology. Digital art now moves to center stage—using digital tools to expand our mind and vision, interpreting through computational culture a new sense of aesthetics that connects with all cultures, as societies worldwide see themselves and others through a digital lens. How much will humans morph to live side by side with robots and see the world through screens?

4.6. COVID-19 Conflicts and Identity

Information and communication corporations are becoming bigger, offering each person their place in the digital ecosystem. Governments are being cowed by big tech (Google, Facebook, Twitter, etc.), the social order is disintegrating, and the big news media corporations represent the reality of social and cultural revolution through a digital lens—we were not prepared. During COVID-19, most American cities have become centers of protest, witnessing a new movement of migration from big cities, and life at the precipice of a radical shift where each human is at the center of creating their identity—choose your gender, choose your religion, choose your age—but you cannot choose to live outside of the digital ecosystem—where there is still lots of room at very little cost—life in the cloud, on a platform—where being unplugged means death—who are you, anyway?—a place where big tech corporations are all about serving individual customers and consumers. What are they serving: information, products, experiences, entertainment, services, customization, etc.? Yet, we wonder, are humans the customer or the product?

4.7. Are We at the End of Reality? Simulating Life, Art, and Identity—Chaos and Confusion

Like many other periods of transformation, revolution, and reinvention, in the early stages there are attempts at simulating what was, and then using a new media. For example, in 1968, Harriet Avram (1919–2006) changed library catalog cards to machine-readable cataloging—in essence, the MARC record, which is still being used, is a digital simulation of sorts. However, living in a computational culture, we are trying to simulate the real experience of education on location at a school to an online experience as a sort of virtual classroom, while we increasingly see that online courses have little in common with the totality of real experience, one of high complexity existing in time and space, where physical and digital converge, and where the digital is used to heighten the human experience in the physical world. However, during COVID-19, the new goal is to create a digital simulation of what was real, in an attempt to convince ourselves that it can and will surpass it, as if sensing that reality is a thing of the past and is old school.

The key symbol of COVID-19, the mask covering the nose and mouth, produces a sense of fear, isolation, and potential punishment for not following rules, like “go to your room—don’t come out until we tell you the coast is clear—it’s safe to step out.” In the past, we had “masked balls” where the mask did not cover the mouth—it concealed identity and communicated intrigue and invitation to become acquainted. Familiarity has not been acquitted, but rather seems omitted from daily life, reinforcing feelings of isolation. Museums depend on academe to prepare students for careers in museums, at the heart of which is curatorship and cultural leadership. Looking back over centuries, we see that apprenticeship has yielded the most significant and tangible results. This long-established mode of learning has been significantly curtailed and will impact students’ ability to understand the curatorial side of museum work.

4.8. Digital Capture and Curation

Digitization remains foundational for museums and the humanities generally. Recent advances in computing and AI for computer recognition across media—image, text, sound, video, 3D, VR, AR, and MR—offers new opportunities for innovative and integrated processes and applications on a large scale, moving from the legacy labor-intensive models and systems that take item-by-item approaches tied to human selection, tagging, cataloging, organizing, etc., to digital tools using AI and machine learning that can automate this process. Importantly, this would greatly increase capacity, access, and use. Museums, archives, and libraries need to collaborate and share across institutions to find ways to automate the process, to be more plug-and-play. Digitization is a repetitive process that can be improved using AI and machine learning to accommodate intelligent aspects of the process normally assigned to humans.

COVID-19 has raised the stakes on implementing and investing in new intelligent systems, as it reduces labor costs, increases productivity, avoids duplication across GLAMs (galleries, libraries, archives, and museums), addresses hidden collections, and broadens the scope of selection while reducing prejudices that are inherent in individual selection and in authoritative knowledge systems. Soon, repetitive work will be superseded by automated systems, and this will be the case in the museum, archive, and library world, and will enable their professionals to focus on important tasks requiring higher levels of human intelligence, creativity, critical and abstract thinking, innovation, communication, interpretation, and storytelling. With this approach, collections come alive and are open to discovery from the perspectives of curators, visitors, and users. This results in a more inclusive democratized system and more sharing for the cost-reduction of routine activities while shifting resources to working teams of artists, curators, and creative computing designers for main-menu immersive and educational visitor experiences, interactive exhibitions, and the like.

4.9. Between Two Worlds—Real and Virtual

We have entered a new period of experimentation and research at the crossroads, before us with an unprecedented number of choices, all leading to degrees of transformation of our ways of being and knowing ourselves. As we enter the “fourth industrial revolution” of AI and machine learning under the forces of COVID-19, with the associated social and cultural revolutions, it seems that we have little choice but to accelerate change and leap into the future. However, lockdown has limited our choices, leading to new solutions that have pushed us to a more artificial and virtual life, dependent on new digital services from the realm of education, to shopping, eating, entertainment, and, in the first instance, communication. Some sectors of society are raking in more money than ever—digital entertainment, communication, and online commerce—while others, such as libraries, archives, museums, and education, are in search of a new order. The most likely appears to be a huge leap towards digital dominance, shifting to digital frameworks beyond information management systems—but rather, to human digital behavior and interaction systems—not big data per se, but increasing levels of the digital embedded in day-to-day life—the “Internet of Behavior” (IoB) that guides, monitors, and controls human behavior. For example, growing out of the COVID-19 crisis, systems have been developed for “contact tracing”, a huge experiment in imposing specific behaviors on the global population that seems unprecedented.

Museums are being lured by large corporations such as Disney, Netflix, and Amazon—not just for sponsorship but also for collaboration with content tied to social justice movements receiving priority in art and education. Digital globalism creating the new phenomena of digitality that is dominating human behavior and interaction, greatly increased by COVID-19, propels us into unknown territory, challenging human identity. The so-called “fourth industrial revolution” challenges traditional notions of living, communication, and working in the world, as AI and ML become embedded systems for day-to-day human functioning, simulating, reinventing, and assuming roles once reserved for humans. This developing collaboration between humans and computing machines using AI and robotics means that human abilities for creativity, innovation, and critical thinking take on new importance, as does human judgement, values, and ethics—stepping out of the box, going beyond the walls—the irrational, unexpected, exceptional, artistic, creative, unrelated, and outside the data set. Rapidly increasing levels of digitality in day-to-day life imperceptibly transform the nature of human experience and what it means to be human. Between elation and fear, digital engagement and isolation, and the old and new order, the unstoppable force of the digital revolution is mirrored by a new social revolution of human identity and equity driving very public social unrest (for example, the Black Lives Matter movement, see Figure 4). This has been in parallel with a widespread shutdown of society due to the COVID-19 pandemic (see Figure 5).

4.10. Museums—A Prelude to COVID-19

In the crisis-leaden environment of COVID-19, to quote Bob Dylan, the “old road is [indeed] rapidly aging” and being replaced by a new Internet highway constructed around AI and neural networks for virtual life in cyberspace and physical place, as we reach out through Zoom in a world where legacy systems are disappearing and being replaced by digital infrastructures simulating human experience and senses, from the IoT to the IoB (“Internet of Behavior”). Looking now to the future from the vantage point of the present, a period of extreme change during which Internet communication, computing, and AI are the key drivers of change in the social order, human behavior, and experience under the weight of COVID-19, causing a surge of digitality in everyday life.

For example, we see education moving en masse to online, shopping online reaching unprecedented levels of participation, and eBooks, together with other forms of digital information, holding sway among users. During the same time, museums, libraries, and archives were forced to close and move online, leaving behind their traditional modes of operating in the physical museum for a virtual presence on the Internet. Thus, these institutions, especially museums, are scrambling to find alternative paths to financial sustainability [41]. Ultimately, although the answer will be in an integrated model of onsite and online, during the coming years the Internet will play a key role. One approach might be partnerships with corporations such as Netflix and Disney, as well as seeking commercial sponsors of exhibitions who might also contribute to the show itself. Museums might also consider fee-based programs such as expanding their education programs to include credit degrees, such as a master’s program in museums drawing on existing resources, collections, and curators, particularly since over the past several months museums have deepened their online presence and connection to their audience and are becoming adept at using 3D, AR, VR, MR, and AI. Over time, such degree programs could become hybrid, adding focus to experiential learning onsite. Further, there is a need for museum programs that prepare professionals to engage the social and curatorial challenges of the field as they strive to connect with a more diverse audience, and are especially effective in visual modes of communication, increasingly the norm, as the old text-dominated world fades.

Although the advance of digitality is inevitable [42,43], there are critical choices in how and where it is applied, why and for what purpose and outcomes, how it serves and enhances human experience, education, quality of life, and importantly, the roles and responsibilities reserved for human work and participation that will more and more be a measure of human relationships with AI systems and robots.

Brian McKenna has posed the question [44]: “At a time when museums and art galleries have closed their doors to art lovers, has the time come for digital art? Art that has been digitized, but also Art that is born digitally, especially video art?” The Israeli startup company, Niio, is developing a platform for digital art that draws on the Netflix and Spotify platform model to publish and view digital art that bonds artists and artifacts using blockchain technology, especially non-fungible tokens (NFTs) [26] and AI. The founder and CEO of the company, Rob Anders, “believes COVID-19 has sped up the growth of digital art, and that technology holds the key to the future of the art sector” [44]. He wants to provide a digital platform and repository for digital art.

For cities like New York City, new economic models are tested as skyscraper apartments, townhouses, and the brownstone houses of the wealthy stand in stark contrast to the tenement and project housing where millions of poor workers reside. If the growing disparity between rich and poor residents grows, where will culture go? How will it be sustained? Will new generations of Americans continue to visit traditional venues that were already struggling before COVID-19, or will a new cultural milieu take center stage? Already, the top educational institutions are turning in favor of ethnic and racially diverse culture while the cultural and social symbols of the past under the glare of COVID-19 are attacked as remnants of a colonialist past. Many immigrants are keen to move to the United States at this moment of cultural shift. How will these trends influence human life and identity, and will human altered states of being appear almost unrecognizable when looking back to where we were in the light of new directions? For the future, it seems that civilization itself will be tested. For the United States, it appears that the “melting pot” is empty. Figure 6a,b brings attention to how museums are becoming more porous with their external environment, championing social popular movements while becoming active members of the broader cultural community.

The murals seen in Figure 7, created by graduating seniors of the Fashion Institute of Technology (FIT), demonstrate their keen awareness of how advances in human computing such as AI, VR, and AI will impact their future, while Figure 8 shows the close ties between art and creative industries influencing human identity and artificial life.

4.11. Immersive Experience: Digital Art and Mixed Reality

The integration of virtual and physical reality in time and space is at the heart of mixed reality (MR). Placing MR in the context of digital art, a new type of exhibition centered around immersive experience has seized the imagination of the public, who seem enthralled with the notion of “being inside art” [46]. Set in a 3D spatial and architectural digital art environment, this new digital art genre offers unique experiences on a grand scale, escaping the monotony and excessive use of flat screens. This new exhibition phenomenon dates from 2018, when the l’Atelier des Lumières in Paris, under the auspices of Culturespaces digital art centers, became the first grand digital art space, and now has eight such art venues, including Hall des Lumières in Tribeca, Manhattan to open in 2022. What appeared at first to be a one-off experiment has rapidly expanded around the world, including Dubai, Amsterdam, and Seoul, attracting many new entrants into the immersive experience space, a trend accelerated by COVID-19 and museum closings.

MR exhibitions fill a noticeable gap in traditional museum exhibitions, where digital art, spatial design, participation, and visitor interaction are often lacking. Seeing that visitors buy tickets averaging from 30 to 50 USD, and more with added virtual reality experiences, immersive exhibitions could provide exciting ways for museums to increase revenue, audience and public awareness, and appreciation of digital art. As a new exhibition genre, the immersive experience creates a model that can be reproduced without having to ship artworks or need high security, since a digital projection cannot be stolen, while the exhibition images can be stored in the cloud, a feature that, for example, means that Van Gogh: The Immersive Experience [47] can be showing at several digital art spaces simultaneously.

A key element of Van Gogh: The Immersive Experience is the way it brings together the real and virtual to create a mixed reality experience, where the art is digital and the gallery space is physical, conceived to produce an artistic and aesthetic presentation using computational methods for imaging, music, and design, modeling aspects of museum exhibitions. Van Gogh: The Immersive Experience in New York City, among other places across the United States and the world as of 2021 [48], takes full advantage of 19th and early 20th-century art no longer in copyright, making the original painting fully usable through open-source access collections. Van Gogh’s Starry Night, recreated through digital art (see Figure 9), synthesizes VR, AR, and MR technology to create immersive experiences integrated into the physical environment. There are even exhibits in shopping malls, such as the Dubai Mall in the UAE [49].

• “Inside Art” by Tula Giannini
• I’m inside art
• My eyes filled with light
• of Starry Night
• Experiencing Van Gogh’s world
• where images unfurl
• My mind expanding
• Safe landing
• into virtual reality
• Surprised
• Mesmerized
• So close to Van Gogh
• don’t want to let go
• Sharing his life and emotions
• Mixed reality through digital notions
• of nature and art
• don’t want to depart
• Van Gogh –
• you’ve stolen my heart ♡

4.12. Precursors of Digital Immersive Exhibitions

Newly launched immersive experiences have followed those associated with Van Gogh, particularly those based on the work of the impressionist painter Claude Monet [50]. The Beyond Monet immersive experience, as with “beyond human” [3], implies mixed reality and mixed states of being between real and digital. The curved walls of the l’Orangerie in Paris featuring Monet’s canvases created specifically for that space in 1922 foreshadow the digital art immersive experience (see Figure 10). Monet was ahead of his time with his wall-to-wall art, finding connections with the experience of Michelangelo’s Sistine Chapel.

This Monet experience, as viewed in the galleries of the l’Orangerie, where their curved walls are filled with large Monet paintings, finds deep consonance with digital light, human digital behavior, spatial dimensionality, and the emotions evoked by the Beyond Monet immersive show. Monet’s Water Lilies paintings (Figure 10) were a donation to the l’Orangerie finalized in 1922. Monet helped with the architectural design along with the architect Camille Lefèvre (1876–1946). Eight panels, each two meters high and spanning 91 m in length, are arranged in two oval rooms which form the infinity symbol. Monet also required skylights for observing his paintings in natural light.

The London Barbican Centre’s 2019 exhibition More than Human featured a large interactive digital installation in a separate gallery, What a Loving, and Beautiful World, by the art collective teamLab [51], which was, in essence, an immersive exhibition invoking a natural environment filled with colorful flower-like shapes and Chinese characters that changed in response to visitors’ movements.

The Van Gogh and Monet digital art immersive exhibitions find inspiration from the l’Atelier des Lumières in Paris, which introduced the key element of sculptural digital art. The space is architectural and in 3D surround space and sound, creating a new genre of the gallery as a place that is design-through, revealing the interrelatedness of the digital art and exhibition space.

Significantly, major changes in museum collecting and exhibitions, education, and service have emanated from outside forces of change, especially those at the convergence of human digital behavior, COVID-19, and the global digital ecosystem in which we all reside. As we begin to see the effects of the resulting digital states of being, we see that revolutionary changes in museums already are making them more accessible, participatory, and connected to communities in ways that enable museums to better serve their public and engage audiences.

4.13. Global Cultural Patterns: Mixed Reality and Tipping Points

Immersive digital experiences are rapidly coming into the forefront of public awareness. Algorithms, deep learning, and neural networks are changing the way we see and make art, as our existence becomes increasingly inseparable from the digitality of life, as we shift to a computational culture encompassing both the natural and built environment connected across platforms and networks while influencing climate change.

What is exceptional about the new immersive exhibition genre discussed here is its global reach and popularity, attracting millions of people, which draws comparison to Alan Turing’s work on patterns and tipping points in nature [52]. The long-established norms of museum exhibitions have endured without significant change. Now, with the convergence of MR, AR, and VR tied to rapid shifts in human digital behavior under the impact of COVID-19, new technologies and tools allow us to observe natural environmental changes and shifts in the human landscape, changes that seem to suggest a tipping point in arts and culture—not just in what we see, but how we see and experience it. As the sophistication of spatial analysis and mapping is applied across the global ecosystem, we can apply these methods to bring greater understanding and awareness, using these digital tools to map cultural change while considering the relevant patterns and trends.

The merging landscape of global culture, now with deep learning algorithms, just as we can map climate change and potential tipping points in the natural environment based on Turing’s ideas, opens the possibility of applying similar techniques to cultural patterns to predict the impact of cultural change in museums, education, and cultural life, including human digital behavior (HDB).

In 1952, Alan Turing described how patterns in nature, such as the stripes on animals’ coats, can develop from a homogeneous starting position, which has been influential in both science [53] and art [54]. Arjen Doelman of Leiden University has stated [52]: “In ecological science, the Turing patterns are often explained as early-warning signals, because they indicate disturbance … Turing’s mechanism of pattern formation is still undisputed. But the fact that a pattern is forming somewhere does not necessarily mean that an equilibrium is disrupted beyond a tipping point.” As an example of such a situation, Max Rietkerk of Utrecht University refers to the transition from savanna to desert [52]: “There you can observe all sorts of complex spatial forms. It’s a spatial reorganization, but not necessarily a tipping point. On the contrary: those Turing patterns are actually a sign of resilience.”

Is this a one-off moment of more than 40 immersive exhibitions just in the US, with many more across the globe, or the start of MR immersive art shows and possible spinoffs changing the tide of the museum landscape? Is this a global pattern, a tipping point of sorts, and how might this impact museum priorities? The New Fields Museum, “a place for nature and the arts”, is the first museum to test this model. Already we have seen how the first foray into immersive experience art at l’Atelier des Lumières in Paris launched the immersive experience movement during the COVID-19 pandemic with its Van Gogh exhibition, which was magnified by the Netflix television series Emily in Paris in October 2020. These two media formats coming together catapulted the immersive exhibition into global view, with visitor numbers trending exponentially. Looking from the perspective of Turing’s theory of patterns in nature as a force for disruption and potential tipping points, in a similar sense we can map digital culture across the Internet, which could provide important insights, especially comparing immersive and museum exhibition patterns considering visitor experiences in their own words, the number of visitors, and the patterns they create across the web. Already, MR, VR, and AR are challenging museums to rethink what they prioritize in their interactions with visitors. One example is how the new director of the Musée d’Orsay has prioritized research and education at the heart of museum development, stating that the museum “can no longer be a tourist factory” [55], while it is planned that the fourth floor of the museum will become a 7000 sq. ft education center, while an international research center will be set up in a nearby building at 29 Quai Voltaire on the river Seine. Combining these innovations with the impact of the digital, social, and cultural revolutions bodes that the tipping point approaches, advancing through MR, a force that is integrating the digital and physical, which for too long has been understood as a dichotomy.

Nineteenth century Romanticism, through to the first half of the twentieth century, now sets the stage for a twenty-first century renaissance—the creative human spirit captured in the work of artists across the globe can not only be accessed and viewed but used as the basis for new work and fresh interpretations through new media. In multiple ways, with the vision of the French “museum”, l’Atelier des Lumières has inspired this burst of mixed reality experiences that have relevance to contemporary identity and life.

Nature is not art but inspires it—being inside art. This is a new phenomenon that is founded in digital art, by which Van Gogh’s paintings are translated through digital and spatial expression, rendering artworks that introduce a new type of immersive digital art in physical space in ways relating to digital human behavior, experience, and inspiration. While VR extends human vision through a small lens that enables access to virtual space, and AR superimposes digital images using QR codes embedded in physical space, digital art defines physical space and goes beyond flat screens which dominate the human mind and vision, turning digital spatial art into a super-reality. Validating the MR digital art form is the fact that millions of people are willing to pay more than the cost of the normal museum entrance fee and are flocking to immersive exhibitions. Museum terminology counts too; these are exhibitions in gallery spaces, not shows on theater stages, where the audience is separated from the “performance”. Rather, the audience is embedded inside the work. Is this a new Disney-type experience that lasts about an hour or two? Museums could learn from exploring the relationship between the immersive Van Gogh to Van Gogh’s original paintings, and its impact on audience reception, while experimenting with museum gallery immersive art exhibitions scaled to their physical space while testing income potential.

Van Gogh and other immersive art exhibitions that reimagine works out of copyright and in an open-source framework are having a huge impact on the art world. Digitization and open source increasingly become a powerful force for creativity. As creative technologies develop, what was considered reuse has evolved through digital art to produce original works worthy in their own terms as a new art form.

5. Conclusions and Future

In the previous section, different aspects of the impact of the increasingly digital world have been presented in the context of art and museums in particular and culture in general. These vignettes indicate that society is on a journey to more and more digitality, digital integration, and inclusion in the future.

What remains to be addressed if museums are to embrace the digital even further—what would that look like? At this critical juncture, what seems most needed are fresh insights for new conceptual models that bring creative ideas to identify structural change, and a move away from old-school notions, legacy systems, and that identify digital tasks that can be outsourced while others are redefined. Still, the question remains, how will museums accommodate rapidly evolving human digital behavior and expectations? What museum activities sit at the core of museum transformation? How will museums marshal their precious resources so that, when the postpandemic visitors return, things will not look the same but, rather, more exciting, more visually engaging in ways that integrate digital life, and that define a spirit of creativity and innovation—not dusty, not old, not unconnected to the current environment, but presenting fresh museum stories, narratives, and relationships past to present that shows awareness of social issues of diversity, inclusion, and equity.

Much emphasis has been afforded social media, in particular Twitter and Instagram, to understand museum audiences, yet social media sits in the realm of visitors while social media analysis goes to Google—and mapping and searching collections can go to Google and Microsoft. So, what can museums do that plays to their strengths and values? The answer is somewhat before us—look at what visitors see, experience, and learn, and ask how museums can enhance and expand that visual, educational, and social experience.

Perhaps the greatest challenges for museums will be finding new models for sustainability and being able to respond to the complex challenges posed by the contemporary cultural landscape. Designing models that encompass onsite and online, both inside and outside, and their integration, while focusing on visitors and community relationships, understanding, and inclusiveness. Museums need to play a greater role in education and consider its income possibilities; for example, offering master’s degrees to prepare museum professionals to meet the critical challenges of digital and cultural transformation. While libraries and universities are hugely subsidized by public funding, museums have relied on financial models that seem more akin to amusement parks and entertainment.

Museums need to realign their missions in ways that increase social engagement, teaching, and learning, and reach out to more diverse audiences while not being so dependent on tourism, especially since COVID-19 has greatly reduced travel. Considering again Salvador Dali’s painting, Persistence of Memory, and an idea to which museums have been closely aligned (heritage institutions, and all that implies), museums must now broaden their mission to include contemporary contexts of art, education, human identity, and life. By so doing, they move to become an essential part of the social fabric of urban life, a recognition that, without museums, we are surely less human and more disconnected from our past, less engaged with the present, and less informed about what our future might be. Yes, we are at the crossroads of an unprecedented digital and cultural transformation rapidly evolving from the COVID-19 crisis, and we see that museums are responding with a new sense of purpose and connection to diverse audiences. Buoyed by the global digital ecosystem, a museum renaissance is surely in the offing.

Recent interviews with leading museum directors point to the need for greater engagement with community, immersive experiences, and connecting with popular art such as Disney classics as possible paths forward. Katrina Sedgwick, director of the Australian Centre for the Moving Image notes [56]:

“It is important not to get hung up on the technology itself, but to focus on how you can support artists to experiment with the technology as a canvas for their ideas. The technology is going to constantly change. Depending on whom you ask, VR is over, and AR will be the dominant mixed-reality platform.”

Although these directors raise insightful points, ultimately new models that tie to financial sustainability and integrate museums onsite and online will be crucial. No doubt, the Internet will play a key role as museums expand their state of digitality across communication, user experience, interaction, outreach, and diversity, adding emerging technologies such as 3D, AR, VR, MR, AI, and introducing experiential learning online post-COVID-19. The digital future for museums will certainly be exciting and, no doubt, sometimes unexpected [57]. In summary, the COVID-19 pandemic has accelerated digital developments in the cultural and heritage sectors that were already underway.