Journal of Pervasive Media - Current Issue

Computational creativity is a growing component of new artificial intelligence (AI) technologies that allow a machine to render creative constructs such as music, text and images. A rapidly growing area of computational creativity is AI text-to-image engines capable of producing realistic imagery that can now meet the standard of human quality outputs. DALL-E 2, built by OpenAI, is a leader in the field and offers commercial access to AI-produced images. To understand the impact of engines such as DALL-E 2 on advertising agencies and their creative workflows, we conducted a series of focus groups with Aotearoa, New Zealand-based advertising agencies exploring creative practitioners’ considerations on the capability of the DALL-E 2 text-to-image technology. An existing Volkswagen advertising campaign called ‘Small but Ferocious’ that used ‘blended’ animals as a visual metaphor for their economical yet powerful ‘TSI’ engines was expanded in a ‘faux’ continuation of the campaign. Four new images produced by DALL-E 2 were presented to creatives attending the focus groups. Participants were then asked about these new creative-AI assets concerning image quality, creative production and collaborative models. A thematic analysis of the comments from the focus groups was conducted and elicited three themes: aesthetics, creative practice and human vs. machine. Participants’ responses revealed that they were both excited and concerned about DALL-E 2’s capability in image production, its effect on creative workflows and the role of the human vs. machine in generating creative outputs. The result was a clear sense of inevitability for how creative roles will change as computational creativity systems, such as DALL-E 2, advance and are adopted into agency workflows.

The intersection of artificial intelligence (AI) and art has been a topic of great interest in recent times. Driven by greater visibility of accessible AI applications within mainstream media, artists have increased their uptake of such tools as means of exploring and expanding their creative expressions. With the music industry also displaying similar levels of curiosity for AI tools, practitioners and audiences voice diverging opinions on the topics of artistic authenticity, creative labour and the threats posed by thinking machines on the future of musicians’ careers. This article aims to explore these topics through an ethnographic study conducted through interviews with five composers active in the areas of electroacoustic music, contemporary composition and experimental electronic music. The discussions reveal some of the software and methodologies currently popular among composers, the challenges faced and avenues presented when adopting AI tools, as well as the attitudes and discourse that permeate the niche circles of AI-generated music. The findings point towards the swift uptake of new technologies by curious artists and the slow development of trust in AI applications by traditionalist makers and listeners, suggesting a continuation of the patterns of behaviour evident since the emergence of music technology.

This article presents the NEFFIE (which stands for ‘Neuroaesthetic Photography’) project, a contemporary reinterpretation of Franco Vaccari’s iconic artwork, presented 50 years ago at the 1972 International Venice Art Biennale and entitled Exhibition in Real Time No. 4: Leave on the Walls a Photographic Trace of Your Fleeting Visit. Thanks to today’s technological innovation, NEFFIE develops Vaccari’s idea from a more contemporary perspective. Vaccari’s original photo booth becomes a technologically revisited booth that uses an algorithm and wearable sensors to reshape the subjective emotional-cognitive responses derived from viewing particular images, thus generating the ‘cognitive photograph’ or ‘COFFIE metapicture’. These metapictures are subsequently minted as NFTs (non-fungible tokens) objects and then displayed on a virtual wall, the ‘COFFIE wall’, generating what can be defined as a Virtual Exhibition in Real Time, a hypothetical contemporary re-actualization of Vaccari’s artistic proposition. The NEFFIE project could be considered a pervasive media art proposal capable of promoting social cohesion, by encouraging creative participation and co-creation. The project is being developed in Milan, thanks to a fruitful interdisciplinary collaboration between the Research Center in Advanced Technology in Health and Well-Being of San Raffaele Hospital, and ICONE, the European Research Center in History and Theory of the Image of Vita-Salute San Raffaele University. This multidisciplinary approach combines art-historical theoretical thinking with the world of technological innovation. In particular, the NEFFIE project integrates the visual and conceptual tools of photography with the methodologies and knowledge of biomedical engineering and computer science.

The exploration of light as a transformative and narrative tool in urban spaces has been traced from its historical roots in the 1800s to its contemporary application in creating ‘spatial augmented realities’. This evolution has been examined through the lens of advanced light projection technologies, which have been used to craft compelling narratives and transform public and private spaces. The role of audience engagement in constructing these narratives has been investigated, highlighting the importance of participation and interaction in creating meaningful stories through light technologies. This exploration has led to developing a theoretical model for ‘accordant’ and ‘discordant’ audience responses, providing a framework to understand the spectrum of audience engagement and its impact on the success of light-based narratives. The concept of light projection mapping has been further elucidated, revealing its potential to expand and transform existing spaces through the superimposition of digital representations. This process has been defined, and its technical aspects have been discussed, providing a comprehensive understanding of its application in various settings. Light art has been categorized based on its narrative function and application, with examples drawn from events and festivals that utilize light projection to celebrate heritage or culture. These examples demonstrate light art’s versatility and multifaceted nature in creating memorable and meaningful experiences. The social implications of light projection mapping on monuments have been discussed, positioning it as a medium for audiences to reflect on historical events and their impact on the present. This exploration contributes to the understanding of how light can be used to enhance spaces, tell stories, and engage audiences in urban environments and provides a theoretical basis for future research and practice in the field of light art.

The use of augmented reality (AR) in literature has predominantly focused on providing instructional tools for educators and learners, emphasizing its capacity of enhancing engagement and supporting immersive learning. Aside from its central literacy purpose, AR in children’s literature appears in the form of digital pop-up books or as add-on games, which are usually located outside of the main narrative. In this sense, AR risks disrupting the flow of a story instead of enriching the continuous imaginative space of the narrative. The Dragon Defenders series, by James Russell, uses AR as a narrative tool in ways previously unseen in children’s literature. Russell puts the reader into the shoes of his main characters, Paddy and Flynn, using AR to show the reader what the boys themselves see. There are moments in these books where the AR system and traditional narrative system fully overlap, integrate and enhance the narrative dimension of the story. While readers navigate through the book by turning physical pages, the addition of AR at specific moments in the story not only provides a more immersive reading experience but, in this case, advances the narrative to great effect. This case study asks how AR can be used to enhance the narrative structure and flow in a text-based novel, using formal gameplay analysis to examine how the AR and narrative systems interact and identifying which examples of this interaction work to the best effect. It analyses the AR moments in the context of the book, deconstructs the unusual first-person perspective from the protagonist’s point of view and explores how these AR experiences help drive and truly augment the core narrative. The larger context of this study seeks to emancipate AR from its predominantly technological ontology and contribute to the development of AR as a genuine narrative device in fiction storytelling.

In this article, we present the creative strategies implemented on the practice-based research project Threaded Edition 21, an international design magazine published in Aotearoa New Zealand. The editorial model draws from a cultural and collaborative framework situated in Tikanga (customary practices of the Indigenous people of New Zealand), Mātauranga Māori (Māori knowledge) and kawa (protocols). In this Special Edition, the structural format was inspired by Māori Maramataka (lunar cycles) and whakapapa (genealogy), and incorporated queues to communicate the multi-layered Māori creation narrative of Te Kore (nothingness), Te Pō (night realms and darkness), Te Whē Ao (coming in to being) and Te Ao Mārama (light realms). These narratives resided primarily in the virtual realm of augmented reality (AR) (as a spiritual encounter), while the printed matter functioned as an enigmatic vessel, creating an ontological reversal where the ‘virtual’ (AR) answered to the ambiguities of ‘real’ (publication). AR-animated vignettes and sonic artistry were activated through printed motifs to incorporate hau (breath of life) into the pages. The AR-animated sonic experience expanded the connection with the audience that when synthesized, provided a sense of immersion into the domain of the unseen which is felt aurally and visually. Structurally, the AR pages open with a chanted karakia (a welcoming prayer) and end with a mihi (closing speech), creating a balance that acknowledges aspects of Māori cultural significance and dualities. When scanned, the AR-animated sonic experiences direct the audience to the accessibility of the Instagram platform, expanding the materiality of printed matter. This approach afforded distinct modes of cognitive processing that combined the visual and the linguistic, the spatial and the temporal, sound and virtual reality to graphically symbolize the many layers and dimensions of things unseen: the realm between being and non-being.