Understanding experience in interactive systems

Understanding experience is a critical issue for a variety of professions, especially design. To understand experience and the user experience that results from interacting with products, designers conduct situated research activities focused on the interactions between people and products, and the experience that results. This paper attempts to clarify experience in interactive systems. We characterize current approaches to experience from a number of disciplines, and present a framework for designing experience for interactive system. We show how the framework can be applied by members of a multidisciplinary team to understand and generate the kinds of interactions and experiences new product and system designs might offer.


INTRODUCTION
Understanding experience is a critical issue for a variety of professions, especially design.Design has had a fairly long history of attempting to support specific experiences when interacting with products.The growing popularity of ethnographic methods applied to design in both academic and business practice has allowed today's designers to mediate their implicit knowledge with that of the people they are designing for.In addition to ethnographic methods adapted from anthropology, methods from social and behavioral science have been adapted to explore the form, function, and content of the products made by designers.One by-product of this activity is the creation of new roles within multidisciplinary design teams (e.g., User Experience Designer, User Experience Researcher, Experience Modeler).Another by-product of this activity is the "hybridization" of research activities between members of a multidisciplinary design and development team.
What is unique to design research relative to understanding experience is that it is focused on the interactions between people and products, and the experience that results.This includes all aspects of experiencing a product -physical, sensual, cognitive, emotional, and aesthetic.The results of this investigation, when used to inform product design, greatly extend simple usability techniques in differentiating particular products in the marketplace [9,12].
The term "user experience" is associated with a wide range of meanings, and no cohesive theory of experience exists for the design community.However, there is great interest in the subject, and there have been both initial efforts to create theories of user experience [3,22,35,40] as well as more recent efforts to exemplify and categorize specific types of experiences as they relate to designed products [1,2,17,48,50].Rather than increase the diversity, we need to better understand how the different approaches relate to each other.In practice, these theories must be made actionable through relevant tools, methods, and processes.
Understanding experience is complex.Designing the user experience for interactive systems is even more complex, particularly when conducted by a team of multidisciplinary experts.Many approaches exist, and many are informed by the insights of different disciplines.Others are informed by implicit knowledge and are made explicit when they are tried out first hand.Integrating all these perspectives is a challenge.What is needed is a framework that articulates experience in a way that does not rely on the point of view of any single discipline, but provides a common design-oriented frame of reference for all the relevant actors involved in design.
In this paper, we discuss experience and its role in multidisciplinary research and practice.We characterize current approaches to experience from a number of disciplines.Some of the approaches take the perspective of the user, others attempt to understand experience as it relates to the product, and a third group attempts to understand user experience through the interaction between user and product.We argue that an interaction-centered view is the most valuable for understanding how a user experiences a designed product.Next, we introduce a framework for understanding the experience of interactive systems.The framework explains how product interactions unfold and how emotion and experience is evoked.We show how the framework can be applied by members of a multidisciplinary team to understand and generate the kinds of interactions and experiences new product and system designs might offer.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.DIS2004, August 1-4, 2004, Cambridge, Massachusetts, USA.Copyright 2004 ACM 1-58113-787-7/04/0008...$5.00.

MODELS AND THEORIES OF EXPERIENCE
A number of models and theoretical approaches have been developed to help understand experience.These include contributions from design, business, philosophy, anthropology, cognitive science, social science, and other disciplines.These approaches examine experience from a number of perspectives.We have grouped these approaches as product-centered, user-centered, and interaction-centered. Product-centered models provide straightforward applications for design practice.In general, they provide information to assist both designers and non-designers in the process of creating products that evoke compelling experiences.They describe the kinds of experiences and issues that must be considered in the design and evaluation of an artifact, service, environment, or system.These models most often take the form of lists of topics or criteria to use as a checklist when designing.For example, Alben provides a set of criteria for assessing the quality of experience of a designed product during conception, planning, and execution [3].Jääskö and Mattelmäki provide a set of design guidelines for understanding experiences and applying them in usercentered product concept development [33].User-centered models help designers and developers to understand the people who will use their products.They integrate a number of disciplinary approaches to offer ways to understand people's actions, and aspects of experience that people will find relevant when interacting with a product.For example, Hassenzahl provides a theoretical model to describe people's goals and actions when interacting with products.It broadens traditional goal-and task-based thinking to include fun and action-oriented modes of behavior [29].Hudspith provides three dimensions grounded in philosophy to derive information about how people relate to products through utility, ceremony, and appeal [31].Sonic Rim, a well-known US-based user research firm, defines the categories of "say, do, make" in research tools to learn of people's experiences with products and their expectations [52].Cain, formerly of E-Lab and Sapient, developed similar user-based categories of "think, do, use" [13].Mäkelä and Fulton-Suri use design to target people's motivations and actions, unfolding within particular contexts, as important in understanding user experience [40].Interaction-centered models explore the role that products serve in bridging the gap between designer and user.Here, too, we see approaches from a number of disciplines.For example, the philosopher John Dewey has been instrumental in helping designers understand the qualitative and definitive aspects of experience [17,18].Essentially experience is a totality, engaging self in relationship with object in a situation.Researchers and practitioners in a variety of disciplines have built on the foundations of Dewey's theory to create knowledge about how people engage with products and the world.Wright et al. [55] discuss experience from a design perspective as consisting of four threads: compositional, sensory, emotional and spatio-temporal.The threads contribute to actions (such as anticipating and recounting) that create meaning.Margolin, a design historian, provides four dimensions that clarify how people interact with designed products -categorizing operational, inventive, aesthetic, and social uses [41].Pine and Gilmore differentiate between passive and active experiences, and experiences that are immersive as opposed to those that are absorbing [48].Overbeeke and Wensveen focus on the aesthetics of interaction and the ways in which form and behavior support feedforward and feedback.Information in interfaces and action are coupled in six ways: time, location, direction, modality, dynamics and expression [47].

AN INTERACTION-CENTERED FRAMEWORK OF EXPERIENCE
Our framework takes an interaction-centered perspective, situated within a social context.It builds on the interactioncentered model presented in [22] as well as studies on collaborative aspects of user experience [4].The framework focuses on interactions between individuals and products and the experiences that result.Additionally, it stresses the importance of these experiences in the context of social interaction, in which people interpret particular events and create meaning.The framework describes user-product interactions (fluent, cognitive, and expressive), and dimensions of experience (experience, an experience, and coexperience) (Table 1).

User-product interactions: fluent, cognitive, expressive
There are three ways that we describe user-product interactions.Fluent user-product interactions are the most automatic and well-learned ones (Figure 1).These types of interactions do not compete for our attention; instead, they allow us to focus on the consequences of our activities or other matters.For example, one's morning coffee-making ritual or the ability to effortlessly ride a bicycle are examples of fluent user-product interactions.
Cognitive user-product interactions focus on the product at hand (Figure 2).These types of interactions can result in knowledge, or confusion and error if a product does not match anything in our past history of product use.Such experiences are often encountered while abroad and encountering foreign toilets, taps and kitchen tools.Cognitive experiences cause a change in the user (such as a skill or a solution) and often the context of use as a result.
Expressive user-product interactions are interactions that help the user form a relationship to a product, or some aspect of it (Figure 3).In expressive interaction users may change, modify, or personalize, investing effort in creating a better fit between person and product.These interactions may be expressed also as stories about product relationships.For example, restoring an old piece of furniture, customizing cars or creating workarounds using a word processor are expressive user-product interactions.

Types of experience: experience, an experience, co-experience
These user-product interactions unfold in a particular context, yielding what we characterize as three types of experience (Figure 1).The first, experience, is the constant stream of "self-talk" that happens while we are conscious.Experience is how we constantly assess our goals relative to the people, products, and environments that surround us at any given time.For example, walking in a park or doing light housekeeping are typical experiences.
An experience is more coalesced, something that could be articulated or named.This type of experience may be characterized by a number of product interactions and emotions, but is schematized with a particular character in one's memory and a sense of completion.An experience has a beginning and an end, and often inspires emotional and behavioral changes in the experiencer.For example, a dinner party or a news hour seen on television could be classified as an experience.
Co-experience is a third way to talk about experience.Coexperience is about user experience in social contexts.Coexperience takes place as experiences are created together, or shared with others.People find certain experiences worth sharing and "lift them up" to shared attention.Shared experiences allow a range of interpretations by others, from the expected and agreeable to the unusual or even deviant.For example, one may reciprocate, reject or ignore an experience.[6] Therefore, expressing meaning is invited by, and the meanings are elaborated in, co-experience through social interaction [4].
Social situations greatly influence co-experience.For example, whether running out of gas when driving to the countryside with friends is viewed as an adventure or a disaster depends on how the friends decide to interpret the situation.One person might be upset, another might point out the humorous potential, and a third might agree.Other examples of social interaction influencing the user experience might include watching others interact with a museum exhibit before using it oneself, looking at a new digital camera in a store with a friend, and adopting abbreviated spellings from friends in mobile messaging.
Co-experience reveals how the experiences an individual has and the interpretations that are made of them are influenced by the physical or virtual presence of others.Other researchers have examined how mutual understanding and context for action shaped the flow and construction of experience, particularly when interacting with technology [19].Interactive technology systems can play a large role in supporting co-experience, through providing mediated communication channels and the possibility to create, edit, share and view content with others.These systems enable coexperience by providing new channels for social interaction, but can also constrain it by disallowing particular actions or making them cumbersome.When an individual interacts with a product (fig.4), his or her experiences dynamically flow between fluent, cognitive and expressive interaction as they happen.Co-experience is the process of lifting up experiences to shared attention, where they become part of a social interpretation process that can influence what the experience comes to mean to the individuals and others.

Emotion and experience
Emotion is at the heart of any human experience and an essential component of user-product interactions and user experience.Standard theories of emotion generally explain how we are disposed to act, the positive or negative valence of that disposition, and the bodily changes associated with emotional arousal.From a psychological standpoint, emotion has three basic functions: to shape our plans and intentions, to organize the procedures related to the plans, and to evaluate outcomes [14].From a design standpoint, emotion shapes the gap that exists between people and products in the world.
Emotion affects how we plan to interact with products, how we actually interact with products, and the perceptions and outcomes that surround those interactions.Emotion serves as a resource for understanding and communicating about what we experience.
For example, emotion can shape our plans and intentions for how we will interact with products.These plans, or mental representations of the actions we plan to take, provide a link between our physical and mental beings and the goal we intend to achieve.Plans can be short or long-ranging.Suchman noted that plans often change in the face of experienced emotion and the constant re-evaluation of a particular situation [53].For example, we may have a friend visiting and suddenly find ourselves very hungry at six p.m.In such a case, emotion guides us to look for ways to feed the guest and ourselves -in an appropriate way.For some guests leftover pizza or a quick omelet does not seem proper.Our goal is to deal with the intensifying hunger in the proper way, and a plan is constructed (going to the supermarket for supplies or going to the Chinese restaurant instead).
Next, to achieve one's goal, emotion coordinates our activity with products and interfaces in the environment.The affordances of products give us clues about how to support our activity.The psychologist J.J. Gibson originated these ideas in his theory of affordances [27].Some have associated the concept of affordances with product usability, but affordances can also be seen as the way people undertake cognition and action in the world to make meaning.If products make suitable activities available and easy at a given time, pleasure and positive product interactions result.If our plans and resulting activities are interrupted, negative emotion results, often startling us to devise a new plan.
Finally, emotion helps us to evaluate our outcomes and experiences in interacting with products.If the outcome is satisfactory, a sense of accomplishment results, and effort is reduced or a new goal is created.This type of outcome supports fluent experience; it has also been described as the flow state [15].
The concept of pleasure as the emotional outcome of a product interaction is one that has been discussed in design literature [34].Pleasure that results from interacting with products may be any benefit that is perceived in the product.However, these theories fail to explain how negative emotions can turn into positive, often shared experiencessuch as how hardships during a hiking trip make for a good story, or how parents may put up with a young musician's terrible trumpet playing with affection, while hoping that it will improve soon.
Emotion serves other roles in social interaction as well: exceeded social regard is pleasant, failed interaction expectations can be disappointing, embarrassing or even enraging.Clearly, social contexts play a role in how we feel, express, and modify our emotions, as well as the resulting meaning that is made.Emotional experiences change, often quickly, in the presence of other people, activities, artifacts, and environments.For example, a call from a friend may cheer a boring day, or a sad incident may bring the entire party down.Emotions mark the experiences that might be suitable or worth sharing with others [6].We control what emotions we choose to show and communicate, in a way that tends to portray events and experiences as more positive or just more intense than they actually were [36].We also aim to create emotional responses in others with our actions [37].
For example, we may comfort someone by providing intimacy, be it just being together, sharing values, or expressing commitment.This is a type of social interaction that is inherently emotional [5,54].

Scalability of experience
Experiences and emotions are not singular events that unfold without a relationship to other experiences and emotions.To address this in design terms we define Scalability of experience as the infinite amount of smaller user-product interactions and emotional responses (relating to contexts, people, goals and actions at a particular moment) that build up to yield larger and larger experiences over time.Typically, as time passes the smallest experiences are forgotten, and only larger experiences, extremely emotional ones and experiences that connect to others are remembered (Figure 2).Scalability of experience is important to consider when using the framework.A small experience will be interpreted in a number of ways, and contribute to an evolving set of larger experiences.
For example, when doing research to inform the design of a wearable computer with web-based services to help users maintain a balanced healthy lifestyle [23], models of emotion and experience were used to help the research team think about how a person's relationship with the product might change over time.Users need to attain fluency with the product early on, to ensure that they will continue to use the product and not abandon it in frustration.This means that minimal time can be invested in learning the basic controls, and that use should be rewarding from the start.Over time, the product should enable cognitive experiences as users begin to learn about their diet, exercise, sleep, and wellness habits, and make the necessary changes in behavior.Perhaps these experiences are associated with positive, longer-term emotional responses, as the user begins to foster a long-term relationship with the product.Finally, the product should foster co-experiences through the creation of a support group, and communication about the product through the assistance it has brought the individual who is using it.A similar study followed how people's interactions and experiences with small objects such as bags and keyrings changed over time Pand context of use [39].people construct meaning with products by following product use through a number of realworld contexts, as well as witnessing the responses of others.Scalability of experience can help to build an overall picture ranging from details of product interactions to the stories and meanings that people use to articulate their experiences.Continuing with the above health and wellness example, smaller experiences around the product could include trying it on the arm for the first time, installing the software, the sound and feel of a button, excitement upon seeing the first results, viewing the data with a friend, or feeling anxiety about the privacy of the data.In interactive systems the challenge is to understand the influence small experiences and emotional responses have on others, as well as the overall view.Each product interaction in an experience can be characterized by a particular fleeting emotional response, may coalesce into a particular emotional expression or mood, and is ultimately stored in memory as a particular aspect of an experience.Scalability of experience also relates to how people's experiences change in relationship to products over time.These changes are best understood with time-based investigations of products in a real or realistic context of use.
Mapping smaller experiences inside bigger ones can be done afterwards by designers and researchers, or be the focus of an activity with participants, in order to understand relationships between small and large.Associated emotional responses are hard to understand, let alone quantify.New research methods are needed to better articulate the relationship between what we feel and what we do.A current trend, for example, is to use biometric data collected from wearable sensors to attempt to capture the most fleeting emotional experiences.

THE ROLE OF THE DESIGNER IN A MULTIDISCIPLINARY TEAM
Armed with the framework as a tool, how can designers make an informed contribution to a multidisciplinary team tasked with designing an interactive system?Designers can offer a unique perspective on what kinds of user-product interactions and experiences a system might offer, and how these experiences might change over time.To do so, designers along with other researchers need a deep understanding of those they are designing for.Designers also need to become inspired by and apply information and knowledge about the aspects of how people use and make meaning with products to the design team.
Conversation analysis methods have been used to understand how people arrange their activities with, through and around products [30].It focuses on common understanding as an achievement by the participants, how their actions proceed as turns, and how this makes both talking and listening active roles in a conversation.
Ethnographic methods have also been used to understand the needs of groups of people, and to generalize theories and ideas from detailed investigations.Cultural anthropology has defined a cultural ecology as the study of symbiotic relationship between people and their environment, in order to understand how people understand, use, and modify the environment in which they live [46].Cultural ecologies provide a basis for understanding a particular culture.They take a material approach, focusing on products, services, and interactions in an environment to describe the behavior for a given group of people [28].
Nardi and O'Day use the term "information ecology" to describe an interrelated system of people, practices, values, and technologies within a particular local environment [45].
An information ecology is used to situate new technologies ethically and responsibly, and to understand technology as a catalyst for change.Bell uses the term "ecology" to qualitatively describe relationships between people and their environment, choosing to push the definition of environment beyond physical and biological limitations to include all the aspects of a specific experience [7].According to Bell, cultural ecologies and the ethnographic research behind them help to "convey an experience, a sense, a glimpse, or a window into another world… a way of talking about deep cultural patterns that implicate everything we do.Knowing these stories, interests, and patterns makes it possible to design and develop products and services that fit (intuitively) into people's lives."[8].
Bell's approach seems highly relevant for understanding the quality of experience, because it offers a mechanism for examining all of the aspects of a particular experience that may be relevant to designing products.However, any approach that brings detailed knowledge of users must be considered along with the designer's understanding of products and contexts.The designer's view should be one of several perspectives within a multidisciplinary team.This concept has also been described as design empathy, 25]; one of a range of holistic perspectives for solving a design problem [16,49,51]; and one of several perspectives a designer can assume, from being a creative, to a scientist, to a "bricoleur" [21].

USING THE FRAMEWORK
Multidisciplinary design teams can use the framework to understand and generate kinds of interactions and experiences that new product and system designs might offer.Once a set of user-product interactions and experiences has been generated, research can be conducted to better understand the people, contexts, and activities to provide solutions for the design problem.
Table 2 shows examples of research activities that can be used to learn about different kinds of experience within the framework.To illustrate we have selected similar case studies of design research and generative design, where similar research activities were carried out.For the more fluent aspects of experience, it is important to capture much of the user's interactions in context without disrupting them.For cognitive and expressive experience, it is important not only to capture interactions in context as they unfold, but also the articulation of experience after the fact.
Design teams seeking to understand experience must learn about the most basic interactions and experiences that the product will offer.What are the current issues in the context where the product will be placed?How can a new product improve the user's current experience?Will the new product be easily adaptable, learnable, and usable?Answers to these questions are best found in taking an objective perspective to the user's experience and interaction.
Design teams must also follow how an experience unfolds, and how it is coalesced, and articulated.What product stories are mentioned as memorable or important?What critical incidents come to light?What language is used to discuss changes in users and contexts of use?What emotional responses accompany these changes?Answers to these questions are best found by taking an objective perspective to the user's experience and interaction.Additionally, understanding the scalability of product interactions, experience, and emotions is particularly useful.The concepts in the framework offer ways to seek meaning in people's interactions, be they alone or together with others.
To understand co-experience design teams must look at all of the potential conditions for collaborating around, communicating about, or sharing a product.How do users collaborate physically and virtually to create shared emotions and experiences?What are the potential outcomes of collaborative product experience?Answers to these questions are best discovered by taking both a subjective and objective view to collaborative experience, emotion, and interaction.
When designing interactive systems, it is critical to understand the social and collaborative aspects of interaction and experience.We have found that several of the more fruitful research methods include introducing concepts, products, and prototypes into the user's world through studies and participatory design activities.Prototyping includes any and all of the design representations of a potential solution for the purposes of learning subjectively and objectively about those who will use the product.While traditional knowledge gained from prototypes has focused on the product function and interface, we have found prototypes to be very useful for learning about what social interactions and co-experiences can potentially unfold.

CONCLUSION
Understanding experience is an exceedingly critical issue for those tasked with designing interactive systems.Understanding user experience -how people interact with products, other people and the resulting emotions and experience that unfold -will result in products and systems that improve the lives of those who use them.Interactive systems for work use can benefit from a more experienceoriented approach, but for new technologies with no immediate perfect use, the experience oriented approach is the only real way that user centered design can impact the technology push.By understanding experience, meaningful and experiential applications can be found for technology as well.
Our research has led to a common way to understand experience, and to understand how social interaction and collaborative product use influence the individual's product experiences and the meanings those experiences come to have.In this paper, we offer an understanding of the experiences of the individual and co-experience as a sensitizing concept to help in interpreting meaning from a social interaction perspective.This process needs to be visual, empathic, and emotionally driven to be ultimately successful in supporting inspiration and gaining insights into user experience.

Figure 1 .
Figure 1.The dynamics of experience in interaction for individuals and in social interaction.

Figure 2 .
Figure 2. Products for camping trips may include phones, maps, bike tools and cooking equipment.The trip is also an experience: preparations are ready, the trip begins.The experience of the trip accumulates from many smaller ones.In the process, experiences with products may change: a tent may start leaking, or a phone may save the day.