Harmonising interaction in media augmented spaces: a semio-narrative investigation of rhythm and tension

This thesis argues that semiotics can contribute to the development of media augmented interactive settings. Focusing on the context of exhibition spaces, the thesis compares and intersects the semiotic approach with communication studies, critical theory and interaction design. Contemporary media augmented spaces, variously inspired to notions of ubiquitous computing, are increasingly characterised by a concern with aesthetic design and bodily engagement. This thesis argues that these defining features require not only distinctive hardware and software solutions, but also recognition of crucial aspects of interactive experiences, such as how users’ attention is shaped and how the narrative and rhythm of their engagement with a media environment is modulated affectively. Against the backdrop of a critical comparison between the fields of semiotics and Human-Computer Interaction (HCI), and primarily drawing on French semiotician Algirdas Julien Greimas’ generative theory of signification, this thesis develops an analysis of media augmented spaces attentive to the increasing centrality of emotive communication and meaningful practices. To this extent, the theorisation of rhythm and tension by the Italian semiotician Barbieri (2004) is developed into an analytical framework exploring users’ involvement with the three interrelated layers of architectural spaces, digital interfaces and physical artefacts. This framework is employed for the investigation of the “Built on Gold” exhibition, based at the City Museum of Melbourne. The case study shows how specific semiotic analytical tools can be brought to bear on interactivity, generating new concepts and avenues of research and suggesting a revision of some of the theoretical assumptions made by Barbieri and others. The thesis shows that our increasingly multifaceted and affective relationship with digital media environments entails a balanced design of interaction, fostering the harmonisation of rhythmic identities and expressive media styles.