Articulating Media Arts Activities in Art-Science Contexts

ABSTRACT This paper discusses the conflicting expectations for media artists taking part in art-science collaborations. Despite the increasing opportunity to participate in these interdisciplinary projects, it can be unclear how media arts activities are best articulated, or even if they need to be defined at all. Additionally, this paper examines a methodological framework widely used in the visualization community for identifying different visualization tasks within research activities. Inspired by its success, this paper proposes a new methodological framework for media arts activities in art-science contexts. This framework splits media arts activities into overlapping areas: generation, augmentation, provocation and mediation, providing a useful way to articulate the broader importance of media arts in interdisciplinary collaboration.


Introduction
Works by media artists tend to be evaluated in terms of either cultural or pragmatic utility. The function of the media arts is often described by highlighting the societal contribution of creating products of cultural enrichment, introducing tools for promoting innovation or providing the means by which to think critically about the ethical ramifications of technology. The media arts also are seen as having the potential ability to aid in the solving of specific scientific and engineering problems, especially those having to do with creative representations of, interacting with, and reasoning about data. Many media artists also characterize their own work as critical reflection on technology, embracing technology while questioning the implications of its use. Articulating these multifaceted tensions between artistic outlooks and technical engagement in interdisciplinary art-science projects can be complex-What is the role of the artist in research collaborations? Many artists have wrestled with this question, but there is no clear methodological approach to conducting media arts research in these contexts. This paper explores contexts in which a media arts methodology is appropriate and necessary, such as when media arts practitioners work in close collaboration with scientists, introducing or participating in the development of pragmatic contributions to research. Roger Malina, in a series of editorials, examines a range of contemporary needs that require creativity and innovation and explores how the media arts (or creative aspects of computer engineering) could be effective in addressing these needs. He outlines increasingly problematic issues in contemporary scientific practice and hopes that art-science collaboration may lead to new techniques and methodologies for approaching them. He explains that a primary motivation for the "art-science movement" is the "epistemological inversion" that has occurred due to the exponential increase in data that is available, whereby the sciences have become "data rich and meaning poor" [1]. Moreover, the type of data has changed-instead of having "snapshots" of data, we have temporal "streams" or complex "data systems." This leads to a "crisis in representation." Science is largely conducted through data analysis and not through empirically sensing the world, and, because there are no obvious ways to represent probabilistic, fleeting or otherwise unintuitive data, this necessitates the attention of the artistic sensibility that is skilled at thinking about issues of representation, or what he calls "re-sensing": "use of visualization and sonification technologies from computer science is a rich terrain of art-science practice and should be viewed as projects in translation." Moreover, this "deluge" of data creates potential "blind spots"-areas and interpretations that are ignored because the amount of data exceeds our ability to analyze it. Finally, he advocates for the importance of art-science collaboration because the solutions to complex, global issues require diverse skill sets and the involvement of experts from multiple disciplines. Interestingly, Malina disagrees with the idea of a transdisciplinary merging of art and science, but instead imagines a "shared language" that "entails shared ontologies and eventually connected epistemologies" that "contribute to creativity and innovation." However, the process of building this shared language involves a perpetual negotiation between different perspectives, a process of "trade and barter and not assimilation" [2].
Media artists also continue to develop tools and platforms that foster creative ways of representing and interacting with data. In a 2007 essay, Ben Shneiderman describes the need for creativity support tools that enable researchers to come to new understanding or to make discoveries during different stages of research, from the "early stages of gathering information, hypothesis generations, and initial production through the later stages of refinement, validation, and dissemination" [3]. Specific examples of tasks that creativity support tools facilitate include: the rapid generation of multiple alternatives, the freedom to explore implications of alternatives and the option to revert to earlier stages. Linda Candy sees the media artist as especially well situated to develop new tools. She explains that media arts projects present a way to "harness" the various "complex social, organizational, and cultural factors" that are required for innovation. Candy thinks of the artist as a "power user" that pushes the boundary of existing frameworks and tools in order to discover new forms of expression [4]. Zafer Bilda also examines media arts installations as proving grounds for techniques that are helpful for understanding how different forms of interaction promote increased engagement and creativity [5]. Pamela Jennings further extends the idea of the creativity support tool as a framework for pervasive computing. She explores ways in which media artists more expansively consider creativity support tools to be "distributed structures that mutually reinforce both individual and social creativity." She recognizes that any tool is part of "a socio-technical architecture deeply interwoven with the physical environment and social fabric of local communities" and calls for a more "pervasive" outlook on creativity when "investigating and promoting situated and distributed aspects of creativity, particularly in relation to temporal, spatial and conceptual distribution across multiple interaction spaces" [6]. Edward Shanken, in "Artists in Industry and the Academy: Collaborative Research, Interdisciplinary Scholarship and the Creation and Interpretation of Hybrid Forms," explores the gap between the perceived importance of media arts' engagement with science and technology and lack of clear methodologies for conducting interdisciplinary projects: "Despite the general recognition that there are substantial challenges to collaboration across disciplines, there is scant metacritical research that studies best practices, working methods, and contextual supports and hindrances" [7]. Moreover, there is confusion about the end goals of interdisciplinary projects, especially when they do not lead to a typical artistic or scientific outcome: "One must wonder about the epistemological and ontological status of these hybrid forms. What exactly are they? What new knowledge do they produce or enable? What is their function in the world?" Shanken is concerned that if there are no clear practices or results to evaluate, then artists and researchers will not have the incentive to "facilitate the creation of new forms of invention, knowledge and meaning." On the other hand, there are exemplary projects that might serve as case studies, and certain individual artists are successful at functioning within interdisciplinary contexts. But it is less clear exactly how and why they are able to function as "catalysts." That is, it is not always clear what specific methods were used in these projects and what specific roles media artists take on in order to help make them successful.

The Nested Model for Information Visualization Tasks
An example of a successful approach to generating effective research strategies can be found in the field of information visualization. The field of information visualization, or infovis, is concerned with issues regarding visual representations that enable effective reasoning about abstract data [8,9]. Infovis is generally situated within the larger discipline of computer science, focusing on advancing scientific knowledge through empirical methodologies and the use of precise evaluation metrics. Infovis is inherently interdisciplinary and involves research in human-computer interaction, cognitive science, computational methods and graphic design. Tamara Munzner's widely cited "Nested Model for Visualization Design and Validation" arose out of a recognition that existing methodologies were not sufficiently accurate in characterizing visualization research and thus were less effective at "guiding" researchers in defining the scope of their investigations and in making decisions [10].
According to the tenets of the Nested Model, an effective visualization research project should: a) accurately describe the specific domain that is targeted and correctly characterize the problems and data associated with that domain; b) map the problems and data into appropriate tasks and data types (what she calls operations); c) design visual encodings and interaction techniques to support those operations; and d) create algorithms to make those encodings and techniques more efficient.
A main concern of Munzner's is that visualization researchers neglect some of these "levels" or make incorrect assumptions about how they can be used to strengthen research practices. Specifically, she identifies threats associated with each level, and further presents guidelines for handling these threats and for providing validations for research activities. At the domain level, a potential threat is that the researcher mistakenly identifies a minor problem that is not necessary to solve and ignores the major issues that actually are important to that community. Here, a possible validation is to observe and interview domain experts who can give insight into the problems in the domain. At the task level, one threat is translating the problems into operations that might not effectively represent or address the domain problems. In this case, a potential validation is to test on target users and to conduct field studies to confirm that the tasks actually are useful for the particular problems. At the encoding and interaction level, a threat is that the specific choices for visual representation or choice of interaction techniques may be ineffective, even if the appropriate operations are identified correctly. In order to validate these choices, the researcher needs to justify their design decisions and also to conduct quantitative and qualitative user studies. Finally, at the algorithms level, a threat might be a slow or otherwise ineffective implementation of even a successful interaction technique or visual encoding. A validation could be to investigate the computational complexity of the algorithm or to quantitatively measure the method to evaluate its effectiveness.
Munzner recognizes that any visualization problem spans multiple levels. In her model, each of these levels is interconnected to the others by "upstream" dependencies and "downstream" influences. For instance, it may not be possible to validate the appropriateness of a task until a visual encoding is chosen and evaluated, and then that visual encoding might not be able to be evaluated until an algorithm is designed that is fast enough to generate the visual encoding at interactive rates. Because all of these levels are interconnected, Munzner believes that it is important to iterate over prototypes that test assumptions at each level before committing to a decision about any one of the levels. Through analyzing the methodological requirements of a successful visualization project, Munzner provides effective guidelines for how to think about a problem and how to approach the creation of new projects. It may in fact be less important whether or not the levels accurately carve out the levels of abstraction related to visualization research with complete precision.

Interdisciplinary Threats and Validations
A previous survey of ethnographies of artist-in-lab experiences and postmortems of art-science collaborations found that media artists working in art-science contexts tend to see themselves in one of four main roles: as a leader, a communicator, a visionary or a challenger [11]. However, overemphasizing any one of these roles may attenuate the impact of the artist in a collaborative project. For instance, activity in one role may not be useful in a pragmatic sense to the more engineering-minded participants, but activities in others may sacrifice the artistic sensibility that motivated the inclusion of the media artist in the first place. That is, the media artist practitioner may need to clearly define his or her roles in an art-science project or be able to switch adroitly between multiple roles.
The space of interdisciplinary research is in many ways at once broader and less well defined than infovis research. Media artists have different agendas than visualization researchers; different practitioners prioritize their goals in different ways. It is therefore not straightforward to define clear steps for media arts practitioners that enable pragmatic outcomes, maintain creative agendas and allow all participants to be directly integrated with the primary goals of interdisciplinary research collaborations. However, it can still be useful to think in terms of threats to and validations of media arts activities and to identify the interconnected dependencies and implications of these activities. Introduced below are four methodological themes, synthesized from an investigation of art-science collaborations that help to articulate and guide media arts activities in art-science contexts [11]. Unlike the Nested Model, which is explicitly, if loosely, hierarchical, these methodological themes are more modular in nature, and perhaps could be described as rhizomatic [12].

Augmentation
Media artists function as augmenters of research, enhancing the representation, interaction, tool-building or narrative components of the research in order to make it more effective for domain-experts and more accessible to outsiders. A fundamental activity of art is thinking about representation; research into visualization and sonification are obvious areas in which media artists can make contributions. Similarly, it makes sense for media artists to explore ways to augment research through investigations into immersion and interactivity. The creation of artworks, whether static or interactive, involves issues of engagement and how to use the "language of art" to tell stories and provide context and narrative; media artists could apply this know-how to make research both more accessible and more meaningful. Additionally, the tool-building interests of media artists can also serve to augment research, as media artists apply their creativity to novel ways of working with data representation and interaction.
Media arts projects, even if they have personal or cultural outputs, inherently involve using and pushing the bounds of existing tools. Media artists can work in art-science contexts simply by recognizing that their work can function as augmentation. Some potential activities include: iterating over different approaches to representing a dataset; using media arts projects to explore research narratives to help users turn raw data into a story; or analyzing the existing representation techniques (if any) to see what can be augmented or improved. Threats to the successful integration into an interdisciplinary collaboration include inadvertently working on tasks that are interesting rather than practical and focusing on techniques that are original rather than effective (i.e. the "not invented here syndrome" [13]). Downstream implications arise from resulting new understandings of data that may necessitate the need to generate new research.

Generation
Media artists function as generators of research agendas in which art exploration leads to and helps define the contours of interesting research problems or applications of research. Simply by following an artistic inclination, a media artist will quickly run up against personal and disciplinary limitations. Many times what the artist wants to do is not possible or is inaccessible. Through a bottom-up approach, media arts projects can generate valid, interesting research questions. Moreover, the exploration of a particular area of interest can serve as a way to explicitly trace the contours and boundaries of the research, of precisely what is not known or is not easily accessible. In addition to surveying the research topic, media artists, who are often experts at particular engineering tasks or generating creativity support tools, can switch hats and turn from idea generation to research, whether in the area of augmentation or other research areas. Of course, it is impossible to be an expert in everything, and this may be a point where the media artist now has a precise enough understanding of what they don't know in order to seek out appropriate collaborators.
Media arts projects will naturally come across interesting research topics and run into technical, representational and even fundamental issues that can be explored. Some potential activities that can be leveraged in art-science contexts include: defining a project or series of projects purposefully to explore a particular area; keeping track of where creative exploration becomes research and being able to switch from exploratory to methodical approach; and actively looking for these boundaries instead of skirting around them. The goal of generation is to discover new lines of research and to clarify tasks needed to reach research goals. Its operations include conceiving of and developing artworks that investigate interesting or incompletely or inconsistently defined topics in order to discover and promote novel research. A main threat is that this process can be seen as risky, that it ends up being counter-productive or that it takes too long and expends too many resources. Another threat to interdisciplinary research projects is that the team (or some of its members) becomes too focused on artistic output and neglects primary research goals or, similarly, that too much focus is placed on traditional scientific practices, neglecting other important goals. One downstream implication is that the generation of new tasks and research ideas may necessitate critical discussion and further provocation about the research.

Provocation
Media artists function as provokers, questioning assumptions and introducing alternative perspectives and interpretations. The theme of provocation involves the idea of challenges to the status quo, of introducing alternative perspectives and interpretations. New thinking can arise by questioning assumptions from different points of view: Why are the stated goals of a project important? Are there other more interesting or useful goals? Are the methods being used to achieve these goals ideal? The conceptualist artist and interdisciplinary researcher Roland Jones writes, "If artists and designers want to participate in reshaping the political, social, economic and cultural agendas, they will have to begin to think beyond the exhausted forms of critical belligerence and mere consciousness-raising" [14]. To Jones, provocation is related to questioning and experimenting with goals and methods. Through provocation, media artists can introduce an interpretive element to research and can provide new formulations through exploring cultural implications embedded in research questions. Some potential activities include: pointing out areas that are underrepresented in research and exploring ways of including those areas; exploring meaning and implications, not just functionality; and explicitly discovering guiding metaphors and aesthetic modalities in order to try to frame research using other metaphors and modalities. In other words, provocation leads initially to new complications, but ultimately opens up the dialogue for more interesting and meaningful research.
The goal of provocation is to expose blind spots and to challenge assumptions in order to create new possibilities. Its operations include asking probing questions and thinking about "broader impacts" and cultural implications seriously. Working with scientists to challenge and refine roles can be difficult; the impact of provocation could be mitigated without a framework to enable this type of dialogue [15]. Media artists wanting to play this role in art-science contexts may even need to study deliberative methods to be more effective at achieving consensus [16]. Threats to interdisciplinary collaboration might include asking questions that are antagonistic rather than helpful, that expand the scope of the project too far, that are too broad to be addressed meaningfully or within a suitable timeframe, or that are simply not applicable to a particular research agenda. A downstream implication of provocation is that it introduces new ideas and people that require new forms of mediation.

Mediation
Media artists function as mediators, creating systems that bridge perspectives and languages, promoting knowledge-sharing between different disciplines and communities, and helping these communities come to consensus about prioritizing research goals. Media arts projects often consider collisions between different realms, bringing different worlds together. This includes starting or forcing a conversation between the everyday and the rarefied, the natural and the unreal, the public and the private. Interesting projects, almost by definition, make new knowledge through combining different areas of experience and expertise, by presenting new perspectives and interpretations that are informed by multiple disciplines or ways of thinking. Thus it is important to create processes and strategies that support effective knowledge-sharing. What if this were an explicit goal of a project, rather than a side effect? In order to function in collaborative contexts, Victoria Vesna writes, media artists "must learn the etiquette and language of various disciplines … Negotiating the gap between the canon of rationality and the fluid poetic is ultimately the goal of artists who work with communication technologies … Bridging and synthesizing becomes the art" [17]. Media arts projects already engage in this kind of negotiating, bridging and synthesizing-an important yet often unarticulated focus of media arts-but this is a valuable way for media artists to explicitly position themselves as builders of mediation support tools that can help mediate between different research communities in art-science collaborations. Some potential media arts activities include: creating projects to represent or externalize thinking and understanding; exploring ways to translate concepts from one domain to another; and creating projects that facilitate communication.
Mediation operations include finding strategies to enable knowledge-sharing and exposing underlying models of users, systems and disciplines. Threats include the unnecessary complication of communication and being unaware of one's own biases and promoting specific models or agendas. Downstream implications include the need to generate new research in order to mediate effectively and to augment data and models relevant to the knowledge of different users.
These four methodological themes, synthesized from an investigation of ethnographies and postmortems [18], can be used to articulate and guide media arts activities. There are many forms of "authentic" collaboration; these themes may comprise one of them. They are presented as a means to reason about pragmatic collaboration and to clarify the "downstream" implications that activities in one area may introduce in others.

Discussion and Future Work
These themes address a particular type of art-science collaboration in which artists operate in scientific contexts and aim to directly impact research goals. They may not sufficiently address many outstanding challenges inherent in art-science collaborations. For one, it needs to be remembered that the roles of artists, engineers and scientists are in flux and that many individuals do not necessarily see an easily separable distinction between the different types of work they engage in. Thus, for instance, the mediation operations described above may be applicable not only between institutions and people, but also within institutions and within individuals. Additionally, this paper is largely focused on exploring how individual practitioners (both scientists and artists) might benefit from using these methodological themes to develop new approaches that are more broadly meaningful to aesthetic, cultural and scientific domains. However, this practitioner-centric approach requires extra theorizing in order to address the specifics of how deeply engrained social and epistemic differences within cultural and scientific institutions could be overcome to promote more extensive and equitable collaborations. To be clear, even within the context of art-science collaborations, the integrity and validity of an artist's practice cannot be criticized on the basis that it differs from what scientists expect to be done with their data and ideas. The effects of artistic practice extend beyond the lab, and can be invaluable for the wider social and cultural understanding of both general and specific scientific inquiry, regardless of the explicit goals of a research agenda. This paper explores one common set of reasons why artists get involved in art-science collaborations, and raises questions about the lack of methodological clarity in such collaborations. I look closely at a framework that successfully provides methodological guidance in the field of information visualization and investigate how and why it is an effective tool to produce multi-layered research. I introduce a novel methodological framework, similar in spirit, that could be useful for media artists working in art-science collaborations. The methodological themes introduced here take into consideration the very different and often contrasting goals of media arts projects, but provide a starting point from which to articulate these goals to individual practitioners, to communities of artists and to their collaborators in science or engineering. More work is needed especially to understand how the role of mediation can work in practice within interdisciplinary projects. Future work will examine how my collaborators and I have applied this framework successfully in a number of art-science projects, including public art installations, scientific research projects funded by federal agencies and artworks that interrogate scientific datasets [19][20][21][22][23].