The True Film

Virtual Reality Aesthetics

Digital Archival Histories

Research Environment

The True Film

Content A – Here comes some Content

Content B – Here as well

Relationality in VR – Interdisciplinary Workshop on Aesthetic Research in Virtual Reality

March 22, 2023, Lucerne University of Applied Sciences and Arts – Nylsuisseplatz 1, 6020 Emmenbrücke, Building 745

How can we describe co-presence in Virtual Reality? Which modes of relationship are presupposed or encouraged by VR technology? Does a symptomatic reading of VR content and user behavior help us understand recent cultural, social and aesthetic dynamics? This interdisciplinary workshop pushes the boundaries of plausibility to explore the possible impact of corporeal experiences in digital encounters on our perceptual, social, and psychological capabilities.

aIMG_6639
IMG_6591
IMG_6599

Virtual Reality (VR) has the potential for creating new forms of relationship between people and other actors. We invite interested researchers and artists to participate with lectures and project presentations on the topic of relationality in a one-day research incubator event.

The concept of relationality comes from sociology. It focuses on the properties and functioning of relationships in a community, rather than describing the community through the properties of its individual members (Donati, Archer 2015). Relationality models are used in various scientific contexts today, such as in ethnography when describing culturally determined ways of thinking and acting in situations where members‘ identification with their community outweighs egocentrism (Hofstede 2001). Scholarly research on swarm intelligence and models of networked societies (Latour 2005) follow this line of thought, as do more holistic, performative approaches in posthuman philosophy (Barad 2015, 2012, 2007). A particular focus on the interrelationships between beings of different species has been developed by Donna Haraway (2018, 2007), and technological and aesthetic innovations, e.g. in the field of digital media, are also evaluated on the basis of relational models in media philosophy (Hansen 2015, 2005 / Shaviro 2010).

With the availability of VR technology to create experiences of digital co-presence, our custom concepts of social and temporal embedding have been challenged in unexpected ways. Although we do not yet have an idea of its long-term impact, VR technology is already being used successfully in many fields such as health, tourism, planning, and so on. At the same time, this technology offers us a unique experimental setup to explore the bodily experience of one‘s own relationality and its conditions: by putting on the VR goggles, the virtual reality guest temporarily shields her own sensual relational embedding. However, the immersion that is achieved through the direct and exclusionary address of the optical and acoustic sense is accompanied by a categorical shift in perception: Although we know about the artificiality of the digital world, we attribute liveness and realism to it (Tavinor 2022).

Naming the digital world „virtual reality“ pointed to a striking conceptual gap as early as the 1980s, when the term was first coined (Lanier 1990, 46): By calling the artificial audiovisual environment a „reality“ of its own, we make it an alien sphere that plunges/entices all participants into an aesthetic adventure. Adventures, in turn, trigger culturally and personally conditioned physical and mental responses. Recently, it has even been noted that VR technology is used in different ways by users from different cultural backgrounds: While in America, simulation of the real world and a rhetoric of immediacy are central to the use and discourse surrounding VR technology, in the Japanese VR industry and community, the exclusion of reality through the headset is seen as a central feature of the technology (Roquet 2022, 7, 138). Although escapist inclinations can hardly be attributed to nationality, Roquet relates the strong gender divergence in media use to a deeper crisis of masculinity that extends far beyond the Japanese cultural sphere.

This workshop aims to bring together expertise and knowledge from disciplines such as film studies and philosophy, game studies, psychology, semiotics, phenomenology, media theory, cultural anthropology, artistic research, narratology, etc. to better understand the context and implications of the perceptual and symbiosensational processes in interactive VR experiences. Aesthetic experiences of body transfer illusions (Slater et. al. 2010), including gender swapping, have been popular in a range of artistic VR experiences since their very beginnings, and strikingly stereotypical gender representations incessantly populate consumer VR environments. Circling around questions of relationality at different levels, i.e., in both perception and social interaction, the interdisciplinary workshop could address topics including:

  • the qualitative description and quantitative analysis of virtual co-presence, encounters, and interactions in VR,
  • the influence of psychological processes of social cognition in VR experiences,
  • the entanglement of corporeal and audiovisual perception
  • the changing situatedness and influence of context in linear and emerging narrative structures
  • possible effects of gender stereotypes and their transformation in character design and movement
  • …

Scholars and artists are invited to illuminate central aesthetic aspects from their own research perspectives in approximately 20-30 min lectures or presentations. The gathering intends to foster an interdisciplinary dialogue and aims at further developing research questions on the aesthetic impact of VR, to ideally plant and establish future research collaborations. Accordingly, two follow-up events (online) are planned for autumn/winter 2023.

Abstracts for papers (2000-3000 characters incl. spaces) or work presentations (short abstract plus 1-3 images or video link) as well as a short biography (max. 500 characters incl. spaces) should reach us by February 15, 2023 at this address: christina.zimmermann [at] hslu.ch.

Submissions can be made in English or German, all presentations at the workshop will be in English. The workshop programme will be announced by February 28, 2023.

 

References:

  • Barad, Karen (2007): Meeting the Universe Halfway. Quantum Physics and the Entanglement of Matter and Meaning. Durham, NC, Duke University Press.
  • Barad, Karen (2012): Agentieller Realismus. Über die Bedeutung materiell-diskursiver Praktiken. Berlin, Suhrkamp.
  • Barad, Karen (2015): Verschränkungen. Berlin, Merve.
  • Donati, Pierpaolo und Archer, Margaret S. (2015): The Relational Subject. Cambridge University Press.
  • Hansen, Mark B.N. (2005): Bodies in Code: Interfaces with Digital Media. London, Routledge.
  • Hansen, Mark B.N. (2015): Feed Forward. On the Future of Twenty-First Century Media. University of Chicago Press.
  • Haraway, Donna (2007): When Species Meet. Minneapolis: University of Minnesota Press.
  • Haraway, Donna (2016): Staying with the Trouble. Making Kin in the Chthulucene. Duke University Press.
  • Hofstede, Geert (2001): Culture’s Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations, 2nd. Ed., Sage, Thousand Oaks, CA.
  • Lanier, Jaron (1990): Life in the DataCloud. Scratching Your Eyes Back In, John Perry Barlow interviews Jaron Lanier. In: Mondo 2000, no. 2 (Summer 1990).
  • Latour, Bruno: (2005): Reassembling the social: an introduction to actor-network-theory. Oxford New York: Oxford University Press.
  • Roquet, Paul (2022): The immersive enclosure. Virtual reality in Japan. New York: Columbia University Press.
  • Shaviro, Steven (2010): Post-cinematic affect. Winchester: O-Books.
  • Slater, Mel; Spanlang, Bernhard; Sanchez-Vives, Maria V.; Blanke, Olaf (2010): First Person Experience of Body Transfer in Virtual Reality. PLoS ONE 5(5): e10564.
  • Tavinor, Grant (2022): The Aesthetics of Virtual Reality, New York, Routledge.

8.30 Welcome, Registration, Coffee

9.00 Introduction (Cinema REX)

Dr. Christina Zimmermann, CC Visual Narrative, Art & Design, Lucerne University of Applied Sciences and Arts

 

ME AND BEYOND (Cinema REX)

9.30 Immersion beyond Belief – Objective, Subjective, and Social Presence

Paul Labelle, DFG Research Training Group “Literature/Presence”, University of Bonn

10.00 Enactive Co-Presence in Narrative Virtual Reality

Prof. Dr. Pia Tikka, Baltic Film, Media, and Arts School, Tallinn University

10.30 How am I? Voids within References to Self and Others in Artistic VR Experiences

Manischa Eichwalder and Dr. des. Manuel van der Veen, Collaborative Research Centre “Virtual Lifeworlds”, Ruhr-University Bochum

 

11.00 Coffee break and Visit to the Exhibition

 

11.10 – 14.00 Filmscreening (Cinema REX)

Splendour in the Grass, 17 min (loop)

Hana Yoo, Artist, Berlin

 

PERFORMING INTERACTION (Exhibition Space)

11.30 Freud’s last hypnosis – validating emotion-driven enactions in cinematic VR

Dr. Marie-Laure Cazin, École Supérieure d’Art et de Design ESAD-TALM, Tours/Angers/Le Mans

11.50 The Performative Player: Scales of Embodied Agency in Virtual Reality Games through Gestures

Oliver Sahli, Immersive Arts Space, Zürich University of the Arts

12.10 Facing the glass wall: User-AI conversation in VR films

Katharina Fuchs, Université Vincennes-Saint-Denis, Paris

 

12.30 Lunch Break

 

MIXED REALITIES (Exhibition Space)

13.30 OpenSoundLab – A Virtual Sound Laboratory for the Arts

Ludwig Zeller, Institute Digital Communication Environments, Academy of Art and Design FHNW

13.50 Virtual Real Rooms: Game Mechanics of Co-Presence MR in a Full Home Environment

Chris Elvis Leisi, Immersive Arts Space, Zürich University of the Arts

14.10 XR Futures: Co-Presence, Co-Extensive Space and Bodily Experience (Cinema REX)

Prof. Dr. Chris Salter, Immersive Arts Space, Zürich University of the Arts

 

14.40 Coffee Break

 

15.00 Roundtable-Discussion

16.00 Summary and Farewell

 

 

16.30 -18.00 Closed Workshop Session (Colabor, 4th floor)

“Netted Letters in Immersive Environments”

 

19:00 Dinner

VR research in the fields of cognitive science and presence research has pointed towards a new way of understanding the underlying mechanisms of presence experience. The research and theorizing in these fields together with recent work in the analytical philosophy of perception makes a new and fascinating way of thinking about presence possible – a way of thinking that carries significant implications for the study of art as a whole.

This paper will present and defend a naturalist theory of presence both in- and outside VR environments. Presence is understood as a set of distinct perceptual processing operations phenomenally accessible as ‘metacognitive feelings’ (Dokic and Martin, 2017). The theory is grounded in an eliminativist ontology and a corresponding simulation theory of mind. From this position, no recourse can be made to the idealist’s ‘beliefs’ of presence-experiencing subjects, nor to the naïve realist’s ‘brute reality’ of the present-appearing objects of perception. I argue that the resulting theory of presence not only conforms to contemporary cognitive theories (e.g. Perceptual Processing), but also provides a far more fruitful way of discussing the aesthetic implication of presence effects in art.

My talk describes our most recent project “The State of Darkness” (SOD 2.0; work-in-progress). The SOD 2.0 is an artistic dissemination of the research project “Enactive Co-presence in Narrative Virtual Reality: A Triadic Interaction Model”.

Read more of the project 

SOD 2.0 is a virtual reality installation in which human and non-human lives coexist. The first is lived by the participant while the latter is lived by the non-human Other. The narrative VR system is enactive, this is, all elements of the narrative space are in a reciprocally dependent state with the other elements. – The concept of non-human narrative allows the State of Darkness 2.0 to reflect the human-centric perspective against that of a non-human perspective. The intriguing question is whether narratives and the narrative faculty should be considered as exclusively characteristic to humans, or if the idea of narrative can be extended to other domains of life, or even to the domain of artificially humanlike beings.

In virtual reality, the technological perception and that of the viewers coincide – both are characterized by a common void, namely that of their own position. How consistent is this initial position when, technically speaking, all relations are calculated from a point which is itself the target of this calculation? And what are the consequences for the visitors in a virtual world when instead of their torso they see nothing, while their hands levitate, the proportions are out of sync, and the field of vision is narrowed? This ultimately leads to the fact that the visitors are necessarily thrown back on themselves.

With this observation in mind, we would like to examine the complex relational structure in VR from an initially counter-intuitive relationality to the self. Artistic VR experiences are particularly revealing in this regard, as they make this void productive as a contact zone. We, therefore, want to focus on strategies in which the viewer’s body appears as the intersection of a relational structure with objects and artificial augmentations. Against this background of self-referential correlationality, how are connections between self, world and other reconfigured? How does this affect a socio-critical reflection on the promises of VR’s immediated presence?

I propose to share with you the first results from the qualitative and the quantitative analysis of user experiences in our 360° VR film “Freud’s Last Hypnosis”, in which the public has the possibility to experience the point of view of the patient and the one of Freud, in the same sequence. The hypothesis is that we find empathy-related presence indications in the psycho-physiological data and subjective report data for the person observed in the point of view (be that Freud or his patient), and for the person embodied in the subjective viewpoint (be this of the patient, or Freud), respectively. The presence in cinematic VR is understood in terms of reported emotions, character identification, and empathy.

We have explored the emotional feedback of 40 participants using psycho-physiological measures and eye-tracking data while watching the film “Freud’s last Hypnosis” with Head-Mounted Display (HMD). In the experiments, we collected behavioral data of eye-gaze in CVR, measured heart’s rhythm (electrocardiogram), galvanic skin response (electrodermal activity) and subjective ratings on the participants’ emotional feelings. Participants have replied to an Empathy Quotient questionnaire before arriving at the experiment, then ITC-SOPI Presence Inventory and AttrakDiff questionnaires after the experience. The recorded data are inferred against the annotated film events and interviews of the participants just after the experience.

In the context of video games, the act of playing has always had a performative aspect. The players’ performative acts consist of the micro-actions enabled by way of input devices that act as Human-Computer-Interfaces [HCI]. Micro-actions are amplified in the game as actions that constitute agency – that is, meaningful interaction possibilities in the game (Murray 1997). Contextualized within wearable Virtual Reality devices, however, such actions become part of a larger embodied act, gaining a stronger performative quality than simply being an extension of the device (Viseau and Suchman 2010).

For embodiment (Kilteni et al., 2014) and the sensation of spatial presence (Slater 2018; Nilsohn et al., 2016), the coupling of action and gesture is problematic. Gestures cannot be suggested but they must be acted out or performed explicitely, otherwise erosion or a rupture of the sense of presence might occur (Weibel et al., 2011). At the same time, such performative acts are essential for fulfilling the totality of the plausibility illusion of VR.

I will examine several works from the Immersive Arts Space at the Zurich University of the Arts, commercial VR games and current research projects which operate in this field of tension between the capability-enhancing quality of extensions and the embodiment enhancing qualities of gestures and focus on enacting through the players’ bodies a novel example of “Kinesthetic interface” (Sutherland 1968).

Voice interaction in VR has developed considerably these last fifteen years, especially in video games, where the player’s voice is used to interact with the game’s menu, with characters in the game, or to trigger an action, supposedly making the game more fluent, immersive and engaging. Narrative VR productions (VR films) are starting to discover human-AI voice interaction, too. Like the voice-over, massively employed by VR films, it is used to create a higher psychological, perceptual and emotional engagement of the user by creating a conversational situation between an author’s or protagonist’s voice-over, and the speaking user. But is the use of the voice really such a well adapted tool for breaking the “glass wall barrier” between the user and the virtual environment (VE)? Does it really allow conversational situations in the VR film, and an improved engagement?

Reflecting on the ontological difference and recognizing the impossibility to (really) interact with the voice-over and the virtual contents might cause a distancing from virtual contents, rather than an increased engagement. Some VR films use this distancing in their narrative about impossible interaction or the incapacity to influence the virtual environment: In “The Passengers” (Ziad Touma, 2021) and in “Darkening VR” (Ondřej Moravec, 2021) I would like to show how the desire—and the impossibility—of connecting with a virtual other, of breaking the glass wall in AI-human communication, and the spectatorial position in a VR film are reflected.

How can a virtual sound laboratory allow for new and exciting ways of sonic interaction in the context of the arts? The project, that I conceived and realized together with Hannes Barfuss, addresses this question by developing the virtual sound lab ‘OpenSoundLab’ (as an open-source fork of ‘SoundStage VR’ by Logan Olson) that introduces to the artistic and musical production of sonic media with the help of the VR goggles ‘Meta Quest 2’.

The aim is to combine the physical experience of working on spatial experimental systems, which is often perceived as positive and productive, with the advantages of digital tools and thus to enable independent learning and experimentation. The virtual lab allows to become familiar with the basics of creative sound generation and processing. Specially produced video tutorials play a central role here, which can be viewed at any time within the virtual environment and thus make it possible to study in individual lab environments independently of time and place. Furthermore, ‘OpenSoundLab’ may serve as an open-source tool for the professional and academic community of musicians, performers, and artists alike. In our reflection, we develop the notion of ‘cooking’ sound while ‘flowing’ in a mixed environment and apply this to experimental work in a virtual sound laboratory.

Recently, there has been a shift in the traditional experience of virtual reality (VR) gaming environments towards incorporating more of the exterior physical world. What kinds of new social experiences might emerge in such a mixed reality (MR) context in which we can overlay both the real world and the computationally generated? How can game mechanics and story elements support these layered worlds while using the limitations of the real world? Could new forms of immersion arise that incorporate elements of the real world (the haptic quality of objects or other forms of presence) with those of the virtual?

To address these questions, a framework was developed that enables new ways of conceiving of co-presence in MR-based game spaces by employing a multiroom setting. Through an iterative process with multiple testplayers (N=120) at several locations, a game entitled “Spacecraft – A New Way Home” was further developed to explore how to structure a variety of spatial game mechanics. In this game the players progressively rediscover their own apartment together, whereupon an increasingly complex, yet always individually shaped spaceship emerges. Players move around their home discovering more of the game in and around the apartment. After the spaceship has been brought under control, the team is ready: unknown worlds lie ahead. The obtained mechanics, developed in this process, can be used as design principles for future mixed reality games.

The recent and dramatic acceleration of technical research, technological promises, and corporate and public imaginaries in VR, AR (XR) has led computer scientists, Silicon Valley executives and the media to claim that the emerging “metaverse” will change human interaction as we know it” (Nardella 2019). At the same time, the emergence of a new set of wearable and half transparent AR (e.g., Microsoft Holo-Lens or Magic Leap) devices together with VR headsets which use a video technology called “passthrough,” are radically configuring not only the experience of user’s presence “between the digitality of VR and the concrete reality of their surroundings” (Saker and Frith 2020) but also embodied understandings of co-presence (Goffman 1959): “the conditions in which human individuals interact with one another face to face from body to body” (Zhao 2003).

Yet, the phenomenological experience of co-present, co-located (in the same physical space) sensory interaction in relation with others in new VR/AR environments remains understudied. Indeed, “presence” in XR contexts has been long accepted as the sense of “being there” (Riva et al., 2003), “telepresence” (Minsky 1980) or what has been called the “place illusion:” the experience of being in a place (even though the real place one is in is usually irrelevant to the virtual experience) “in spite of the sure knowledge that you are not there” (Slater 2009). While there is a vast literature on such virtual presence (Biocca and Levy 1995; Biocca et al., 2003; Lombard and Ditton 1997; Slater and Wilbur 1997), its focus is mainly on “the degree to which a virtual environment submerges the perceptual system of the user” away from the “real world” (Biocca and Delaney 1995). At the same time, little of this work addresses concepts of presence that focus on sensing, materiality and “thingness” from a philosophical or aesthetic perspective (Gumbrecht 2006; Noe 2012). Furthermore, studies of co-present interaction in VR/AR have overwhelmingly focused on purely virtual environments (Bulu 2011; Lankes et al., 2017; Schroeder 2002) with avatars and virtual humans (Wang 2011; Freiwald et al. 2021; Shin and Dongsik 2019), virtual agents (Strojny et al., 2020) and video conferencing (Kim et al., 2014). In other words, the vast majority of research into presence and immersion in VR “has only a limited involvement in concrete space” (Saker and Frith 2020).

Using a recent XR-based theatre project called “Animate” focused on climate transformation as a case study, this presentation aims to present concepts and methods for grappling with what Ronald Azuma claims is the fundamental challenge as we move into increasingly mixed reality-based experiences: “how to enable virtual content that is integrated with the surrounding real world, while users remain engaged with and aware of that ‘real world’” (2016).

Polyphonic Authorship

Here comes content

Research Environment „Netted Letters"

Project Head, Senior Research Associate

Christina is a researcher, lecturer, author-director, artist, and film curator. She is passionate about story development in all kinds of audiovisual formats. Her research interests focus on experimental film narration, dramaturgy, theories of perception, and film philosophy. Christinas work has been screened internationally and she received various grants and awards (among them Japanese Society for the Promotion of Science, Research Association Marbach, Weimar, Wolfenbüttel “Digital Humanities”, Swiss National Science Foundation (SNSF). She has been teaching at Bauhaus-University Weimar (2005-2010), at FHNW Academy of Art and Design Basel (2017), at Zürich University of the Arts (2022), and since 2017 at Lucerne University of Applied Sciences and Arts (MA Film).

Research Associate
Simon is an artist and researcher, trained at ECAL (Ecole Cantonale d’Art de Lausanne) and Lucerne University of Applied Sciences and Arts. He persues theoretical and practical research on new technological and aesthetic approaches in audiovisual and interactive media, especially in VR. His work is exhibited in Switzerland and abroad. Simon de Diesbach is currently working on the visual development and the user experience for the Research Platform “Netted Letters”. Besides, he regularly gives workshops in animation and design in various Swiss art schools and universities.

Scientific Research Associate

Developer

Research Assistant
Christoph works as a research assistant for the Visual Narrative research group. He completed his studies in Post-Industrial Design at the FHNW Academy of Art and Design in Basel. He is actively involved in several research projects that explore the potential of augmented reality in urban planning and how it can facilitate citizen participation in the planning process.

Nach oben scrollen