Cafe Delphi: Strategies for successful remote academic collaboration

,


Introduction
Developing collaborative and cooperative research across academic disciplines and university administrative boundaries can be a major challenge, as evidenced by numerous initiatives established by Higher Education Institutions to encourage greater social interaction, shared research cultures, and shared corporate identities.This paper uses a methodologically innovative combination of methods to generate and evaluate ideas and strategies for developing a cross-faculty research culture at the University of Wolverhampton and beyond.The paper draws on the findings from two participatory 'Research Café' seminars hosted by the Institute for Community Research and Development (ICRD) at the University of Wolverhampton.The first seminar used a 'Café Delphi' approach which drew on a hybrid combination of the World Café approach and the Delphi technique to discuss challenges and solutions to building a cross-faculty research culture.This session made use of an online tool called Mentimeter.In the second follow-up seminar, the approach was written up in real-time using a single shared Google Doc.The final draft was completed using Microsoft Teams group chat and video meeting software to discuss changes made, drawing on insights from the use of massively authored papers (also known as 'massively open online papers', MOOPs).This methodology could be applied to a number of purposes in academic life, such as developing bids, drafting departmental strategies, and most significantly, writing academic papers.In the current public health environment of increased social distancing and remote working across the world, the method offers a flexible means of using technology to elicit information, develop consensus and collaborate with colleagues.

Literature review
This section will introduce the two methodological tools used in this paper -the Delphi technique, and the World Café method -before discussing the concept of massively authored online papers, and concludes with a brief explanation of how cross-faculty working is currently utilised in higher education institutions.

Real-time Delphi
The Delphi technique is a research design which involves quantitative method(s) designed for seeking solutions to issues in cases when data is incomplete or does not exist.The purpose of using this methodology is to come up with a consensus of opinion of a group of individuals.In doing so the method allows us to find the most suitable solution for a problem under study.Although the Delphi technique originated in the 1950s as a project conducted by the United States Air Force, today it is successfully used by researchers that seek a consensus of opinions among research participants.In practice, the Delphi technique employs the use of a series of questionnaires.After the first round of questions, a moderator provides participants with anonymous feedback on the answers offered by all research participants.Each subsequent questionnaire is adjusted taking into account information and opinions found in the previous questionnaire.Participants reflect on this feedback and are able to include these reflections as well as their opinions during subsequent questionnaires.This process is repeated until a desired level of consensus is reached among all of the participants.
There are four key elements to Delphi surveys: 1) anonymity; 2) iteration; 3) controlled feedback; and 4) statistical aggregation of group responses (Rowe and Wright, 1999).The Delphi technique is flexible and can be adjusted by the researchers who can include various methods of selecting participants, types of questions employed, and tools used for the analysis of responses.There are various advantages and disadvantages of the Delphi technique.Disadvantages of the Delphi technique include the lack of opportunity to elaborate on and discuss responses with other individuals; the need for a commitment from individuals to continue to revisit similar questions on multiple occasions and that the answer reached may not be 'correct' but rather an indication of an important point for consideration for the group of individuals involved.

World Café Method
If we acknowledge that successfully addressing challenges requires insight from multiple perspectives, appropriate methods must be developed to allow these perspectives to be shared and heard.Where a project aims to include the voices of a range of partners, the World Café approach has been used with some success.
The World Café approach is a participatory method for engagement.This method is more conversational in approach than other methods, allowing participants to engage in 'constructive dialogue around critical questions' (Fouche and Light, 2010).The World Café is a relatively simple method, but powerful in 'fostering constructive dialogue, accessing collective intelligence and creating innovative possibilities for action, particularly in groups that are larger than most traditional dialogue approaches are designed to accommodate' (Brown, 2005: p3).A number of broad principles underpin the World Café approach, although the principles can be adopted and adapted to be responsive to the aims of the project and needs of the participant group.The basic model incorporates the following five principles: 1) Setting -an informal, café style environment 2) Welcome and Introduction -the host puts participants at ease through a warm welcome and overview 3) Small-Group Rounds -three or more 20 minute rounds of conversation between small groups of four or five participants 4) Questions -a key question underpins each round, which may build upon one another in subsequent rounds 5) Harvest -each group is invited to share the results of their conversations with the wider group, and these are reflected visually (eg.through graphic recording in front of the room) (The World Café, 2020) Figure 1, below, outlines the principles for hosting World Cafés.All participants are viewed as experts by their own experience.Diverse views and opinions are sought and there is no pressure to reach a consensus (MacFarlane et al., 2017).The World Café method has been applied to a number of disciplines and applied settings, with several authors reflecting on their experiences.Fouche and Light (2010), for example, report on the design and implementation of the method in social work research; while Macarlane et al. (2017) provide an analysis of the method in primary care settings in Ireland and the USA; and Page and Temple-Malt (2018) reflect on its application in criminological research.
As MacFarlane et al. (2017: p.278) conclude, the World Café method is generally viewed as 'a valuable, participatory, flexible method' and thus suitable for gatherings of groups with different backgrounds, experiences, and viewpoints.

Massively Authored Papers
Changes in technology have enabled new ways of working within academic writing and research, to allow more people to easily collaborate on research and writing both in person and remotely.Tennant et al. (2019) recognise the opportunity for 'massively open online papers' (MOOPs).Significantly, this not only highlights new practical possibilities within academic writing, but, importantly, begins to enhance the breadth of ideas, experiences, and contexts contained in pieces of academic writing and research that were previously out of the reach of many academics.Whilst this may pose challenges in terms of continuity, it can also bring significant benefits regarding the comprehensive and collaborative nature of research.
Recognising the threat of organised chaos and a loss of focus, Tennant et al. focus on the need for there to be a clear focus and structure underpinning a project that is shared among all participants.For Tennant et al. this is achieved by agreeing a set of clear 'rules' for the process, such as recognising editorial control, guidelines for contributors, setting the structure of the paper and the ways by which contributors are 'invited' by those involved.As such, despite concerns that the research and writing process could descend into being amorphous and fragmented, abiding to the rules allows for coherence and relevance.To this end, it is important to invite appropriate people and ensure that the opportunity for exponential growth in authorship is balanced with clear focus, direction and control over membership.In effect, far from being a fluid or organic process, the management of MOOPs should feature rigorous controls and management.The use of online writing platforms, such as Google Docs or other comparable platforms, can allow for collective writing, reviewing and recognising a broad context for one's work.It also allows people to be allocated to focus on specific sections of a writing project, in addition to academics working remotely.
Recognising the benefits of massively authored papers, Tennant et al. (2019) see the process as being able to open up academic debate and collaboration beyond the academic confines within which we can often find ourselves operating.This might include working with potential partners from other time zones, other institutions or, importantly, those from outside of our own disciplines.Not only does this push the boundaries of our knowledge and expand analyses, it can also lead to appealing to wider audiences.With this in mind, it might be that the authors reflect different traditions within a particular subject area, such as Social Sciences or Humanities, but may also come from academic disciplines not previously associated with these topics but who are in a position to make critical and important contributions.Similarly, it might also be that authors are invited to take part because of their different experiences and knowledge, which may include professional and/or lived experiences.Therefore, editorial control of the process means that not everyone involved has to take on comparable roles within the paper, produce the same amount of work, or come from the same starting point.Tennant et al. (2019) set out a series of steps and guidelines to enable a smooth process and facilitate productive outcomes.Importantly, they also recognise that research and writing projects never turn out entirely as planned and that flexibility needs to be integral to the process.Further to this, Tomlinson et al. (2012) have explored the practicalities of the process, and usefully examined the feelings of those involved in the process.Recognising the challenge that authors could feel abstracted from the process, feeling that their engagement was of a 'helicopter' nature and that they would have liked greater engagement.The risk of author isolation and sporadic engagement can limit feelings of ownership.This also highlights the importance of inclusivity, raised by Tennant et al., that can be achieved by identifying structures, roles and ensuring that these are maintained.
For Vaish et al. (2017), the process of crowd research allows for a democratisation of research and reporting.Inherent in this is the opportunity to create a more diverse research team and reporting mechanisms.On an instrumental level, such an approach is useful for introducing new and critical voices that can add a degree of validity and insight to research and reporting.However, at a time when there is greater scrutiny of diversity within research and higher education in general, then the democratisation of research can be seen as beneficial.

Cross-Faculty Research
Research that crosses faculty boundaries has been supported in a number of higher educational institutions both in the UK and in other European countries.The University of Warwick in the UK, for example, has a number of cross-faculty degree programmes which are administered by a School for Cross-Faculty Studies, and nine Global Research Priorities (GPR) that respond to 'multi-faceted global problems' and cut across faculties and disciplines (University of Warwick, 2020).Similarly, University of Wales: Trinity St Davids (UK) has identified three themed areas for cross-faculty collaboration (UETSD, 2020).Imperial College London (UK) has a framework for Cross-Faculty Research initiatives and research which is cross faculty can apply for formal endorsement and institutional recognition.Our own Institution -The University of Wolverhampton (UK)has a number of annual funding calls for multidisciplinary research targeted at all stages of the researcher career.Our research Institute (ICRD) is multidisciplinary and exists as a result of significant investment by the University.
Others have used cross-faculty partnerships as a heuristic tool, for instance, Coventry University (UK) (2020) has developed a cross faculty project where students from different disciplines collaborate on a project to design a toy, and Portsmouth University (UK) (2019) has held a number of workshops to develop ideas for cross faculty student projects based on five cross cutting research themes.Outside the UK, Delft University of Technology (Netherlands) has developed university-wide partnerships for societal challenges where: "The research questions emerging from these problems are often so complex that they can only be answered by joining forces" (TU Delft, 2020).Finally, Hanover University (Germany) has a university supported network of research initiatives, centres and schools which are interdisciplinary and cross faculty in nature, to work on "complex, cross-disciplinary questions" and which have their own "cross-faculty organisational structure" (Hannover, 2020).The significance of such trends is recognition that in terms of epistemology, many heads are better than one and that the diversification of knowledge can be beneficial in the development of ideas.

Method
The core approach underpinning this project was a hybrid 'Café Delphi' method, following the guidelines outlined in more detail in the literature review above.The project took place across three two-hour sessions over three days several weeks apart.In the first, information was elicited using the process outlined in figure 2 below.In the second, Google Docs was used to write the article collaboratively in real time.In the final session, Microsoft Teams was used to review and redraft the paper.
In this case the method was adopted as a novel approach (participants had not used it together before as a single group) in the hope that it would maximise the generation of ideas and dialogue that could be taken forward and then formally written-up via the 'massively distributed authorship' approach illustrated by this paper.In order to conduct the initial piece of 'research' and prepare the 'findings' reported in this paper, a small and coherent group of researchers, already known to each other in a workplace setting (n=7), went through a threestage process, during which the clear, shared intention was to learn and practice the approach in order to apply it in other similar projects, and potentially in other settings.It was therefore embarked on in a shared experimental and 'open-minded' spirit.
The first step was to conduct an exercise that in effect was a hybrid combination of a 'realtime' Delphi panel process and a World Café, which lasted for approximately one hour in total, and was carried out in the room where the researchers were gathered (all researchers were physically present).In keeping with the World Café approach the setting was informal, and food and drink was provided.In its most basic sense, the approach consisted of a group 'brainstorming' discussion in order to surface ideas, the outcomes of which were shared on a large wall mounted video screen using Mentimeter; followed by a second stage in which the first round ideas were ranked in perceived order of importance.Mentimeter allows users to 'vote' on the options in order to establish the final ranking.The schedule employed was as follows: The group-sourced answers to each question were displayed on the screen (in 'bubbles') so that the group could see the emerging ideas, but these were anonymised and unattributable to individuals (unless they chose to highlight that they have supplied a particular answer, which in this case no-one did).At this point there was no ranking or any other method of highlighting or distinguishing the 'quality' or popularity of the ideas -they are treated as equal at this stage.The next step of the exercise however is to rank the ideas, an implicit recognition or judgement of the relative merit of each idea.These 'findings' were then taken forward into the writing process carried out through massively distributed authorship using Google Docs.In the transition from the 'Café Delphi' phase, decisions were made as to which answers to prioritise and discuss in more detail.The quantitative ranking suggested that the top four challenges were worth further exploration; and the top five ranked solutions came out well above the following options (see Figure 3 and 4. In the findings we focus on these top four challenges and top five solutions.
For this process, the authors were face to face in the same room and able to communicate directly with one another, although this could be adapted to operate remotely using software such as Microsoft Teams.

Findings Challenges
The group raised 12 potential challenges to building a cross-faculty research culture.Participants raised the following challenges: research leadership and reporting within faculty structures.Also, that in some instances leadership can become either too dominant or too weak, thus closing down opportunities for a collaborative culture.A lack of opportunities for researchers to spend time together, whether this be ad hoc opportunities or cross-faculty events, was noted as a possible challenge.There is an additional challenge of people attending and prioritising structured sessions and events that are run as and purposely set up to be cross-faculty.Another perceived challenge was the time to prioritise research generally.
Disciplinary boundaries were considered another challenge, due to different methodological traditions and disciplinary styles of working (one might even say dogmas and pedagogic and research ideologies); the challenge of knowing how to identify who might be open to and 'good at' collaborating in other faculties, and the varying incentive structures/motivations to engaging with research.Finally, challenges were raised in relation to how to develop shared aims and how to measure success.Similarly, another challenge raised was that the REF (Research Excellence Framework) process and the method by which QR funding (Quality Related -distributed to Universities based on REF performance) was distributed (and may be in future) could push research to be conducted in silos, thus not promoting cross-faculty research.
When asked to rank the biggest challenges to building a cross-faculty research culture, participants clearly felt that the most important challenge was a lack of opportunities to meet face to face in daily University life (Figure 3).This almost certainly reflects a feeling that the physical space of the University precluded ad hoc interactions or possibly also that there was a dearth of organised cross-faculty sessions (such as seminars, public events, and social gatherings).The balance between these would be worthy of further qualitative exploration.Second most important (though only chosen half as many times) was 'getting people to attend organised sessions' (reflecting perhaps frustration that even when provided, events are not well attended).Solutions relating to face to face interaction included: (1) face to face events to address the culture and encourage informal conversations; (2) core research staff attending subject meetings to ensure research remains on the agenda; and (3) a physical shared space (i.e., bar, cafe, pub or Union).Virtual communication (4) was also discussed whereby researchers could share news of successes and research related activity via virtual spaces (i.e.newsletters).
Three solutions proposed related to funding.These included: (5) the availability of internal funding for cross-disciplinary projects; (6) incentivising collaboration through joint calls/bids/seedcorn proposals to kickstart discussions and projects; and (7) cross-faculty research themes proactively identifying funding opportunities to bring people together.
The final two solutions related to senior leadership and support, including (8) Associate Dean's facilitating thematic collaboration and (9) clear leadership engagement with support for research.
The top ranked solution was joint calls, bids and seedcorn funding promoting discussions and projects, followed by cross-faculty research themes and internal funding for crossdisciplinary projects as shown in Figure 4.

Reflections on the process
This section discusses, firstly, the authors' reflections of their involvement in the 'Café Delphi' approach and secondly, their experiences of writing a multi-authored paper using Google Docs.

'Café Delphi'
The authors were in agreement that the combination of Delphi and World Café research methods could be considered complementary methods, whereby expert consensus can work well with group consultation.It was felt that this hybrid method allowed for both defining the discussion and moving forward to propose and rank challenges/solutions in one session.It was good to have participants in the same room for the purpose of sharing opinions and a combined sense of brainstorming, however, the authors believe the method is flexible enough to involve wider participants in discussion and ranking activities from a distance.Bearing in mind the claim in the literature review that clear editorial guidance, membership, and direction are essential to avoid a process sometimes called 'mission drift' or the production of an unfocused piece of work, the communication was useful.Had we been in different institutions or even different time zones, this element of everyone recognising a shared vision and process may have been slightly more challenging.Geographic or even temporal variation can be recognised as a reason why some people may feel helicoptered into the process.As such, clear communication and guidance from the start is important if the process is to be effective and focused.Significantly, the potential fluidity and variation of multi-authored work should not be confused with being directionless or without leadership.The method was also suitable for bringing together disparate perspectives, given that all suggestions are considered equal during the discussion phase, before anonymous expert ranking took place.
The use of technology (in this instance Mentimeter) was rated highly by the authors, who stated this tool was intuitive; inclusive, allowing ideas and thoughts to be suggested simultaneously in real time; and unfiltered by the facilitator, encouraging equal voice.On a more practical level, the authors felt that future implementation would be easier having now been through the full process.In particular, authors reflected on the need for a clear briefing email to all participants in advance of the first session so that they were fully aware of the process and need to bring a laptop in order to participate fully in the workshop and the subsequent multi-authored paper session.

Multi-authored papers
The positive, supportive peer environment, which led to a focussed effort and rapid formation of a draft paper, was considered the main advantage to writing a multi-authored paper.All authors felt a great sense of achievement at the end of a one hour session.The authors reflected on the positive social pressure of seeing others contributing to the content in realtime and how this helped them maintain their own focus.Having identified a dedicated time for all group members to participate in this shared activity, it was found to be a good corrective to the procrastination and 'writer's block' which can bedevil lone working (and particularly lone home working).In particular it was found that writing sections of the paper in small blocks of time, and agreeing a deadline, contributed to the timely completion of writing tasks.Furthermore, the agreed approach was for individuals to draft sections separately, so as to avoid duplication and 'over-writing', then to 'swap' in the next block of time and to edit or add to other participants' previously drafted sections.This led to rapid refinement of text and the overall progress of the paper.Overall it was felt that this method of multiple authorship was easily applied to report writing.
Despite having a clear report structure in place, one challenge of multi-authored papers is the various writing styles of different authors which can result in a poor flow to the paper and/or discrete sections of the report, though this can clearly be an issue with traditional multi-authored papers.It is really important to outline the style and focus of the report and to have identified an editor who will take responsibility to pull the draft paper together more coherently at the end.
Later in the process of completing the paper the Coronavirus crisis intervened and we were forced to distance ourselves physically.We therefore found it invaluable to move to Microsoft Teams in order to discuss changes to the paper in real time and particularly in order to agree wording and final edits.We used Google Docs throughout and recommend this way of working as a way to prepare and complete MOOPs at distance.

Conclusion
This paper, we hope, does a number of things.Firstly, it is an attempt to put into practice the collaborative writing of a 'massive open online paper' (MOOP), in this case among a small group of academics who ordinarily work together quite closely; but in so doing we hope to

Figure
Figure 2: World Café process

Figure 3 :
Figure 3: Ranking of challenges identified

Figure 4 .
Figure 4. Ranking of solutions identified