Designing a learning analytics dashboard for twitter-facilitated teaching

Social media sites are increasingly being adopted to support teaching practice in higher education. Learning Analytics (LA) dashboards can be used to reveal how students engage with course material and others in the class. However, research on the best practices of designing, developing, and evaluating such dashboards to support teaching and learning with social media has been limited. Considering the increasing use of Twitter for both formal and informal learning processes, this paper presents our design process and a LA prototype dashboard developed based on a comprehensive literature review and an online survey among 54 higher education instructors who have used Twitter in their teaching.


INTRODUCTION
There are many potential benefits to using social media for teaching and learning. Previous research in this area suggests students' class participation, engagement, and even performance, are likely to increase once social media platforms are introduced [1]- [3]. However, there is a lack of empirical investigation about whether use of social media is beneficial to students' teaching and/or learning needs. This study addresses this research gap by investigating the affordances and potential roles of Twitter in teaching and learning, while exploring the design space of information dashboards to support learning analytics.
Information dashboards have been commonly employed in the area of Learning Analytics (LA). To date, however, research in learning analytics has primarily focused on providing insights in the context of more traditional, closedlearning environments such as online classrooms with student-only access. We contribute to the field of LA by investigating how Twitter data can be incorporated in the evaluation of learning processes. From this investigation, we propose a design of a Learning Analytics dashboard to support Twitter use in teaching, with the subsequent goal of evaluating its usability in teaching activities. To inform our design process, we first conducted an online survey among instructors to assess the experiences of college and university instructors using Twitter in teaching activities. The outcome of this survey informed the development of a prototype dashboard. The paper presents the findings of the survey, addressing such aspects as previous experience and objectives of using Twitter in the class, as well as what information instructors might find valuable in helping them understand students' Twitter activities. We then present the resulting design of our LA dashboard prototype and the description of its interface and data analysis components.

METHODS
Our survey aims to gather information about instructors' use of Twitter in teaching. In general, our goal is to gather as many case studies as possible and thus answer the following research questions: RQ1: How is Twitter being used to support teaching and learning by University instructors? RQ2: What are the common assessment strategies of Twitter use for teaching and learning used by instructors (if any)? RQ3: What analytical techniques would instructors like to see in a LA dashboard to support their assessment of Twitter-facilitated discussions?
The first section of the survey asked participants about course details, such as the subject area, whether it was required course, and its enrollment numbers. We then asked instructors if they used Twitter in the past, what their objectives were in using Twitter, and whether those objectives were met. We also inquired about the institution in which the course was taught, and the degree of support offered for those faculty adopting social media. The survey concluded with questions of whether instructors see any benefits of using LA techniques to assess students' engagement on Twitter, and if yes, what types of analytics they might find useful to support their assessment. The survey questions were created based on our previous survey design on how higher education instructors use social media more broadly [4].
To recruit survey participants, we invited university instructors who are active on Twitter (identified through manual searches of public Twitter profiles) and contacted 150 Teaching & Learning Centers at major universities in Canada and the USA to disseminate the study invitation to their faculty members via mailing lists. We deployed the survey online in the summer of 2017 following an ethics review. As an incentive for completing the survey, respondents were invited to enter a draw for a prize. Finally, we asked instructors if they would be willing to further contribute to the study by helping us evaluate an original LA dashboard during Phase 2 (outside the scope of this paper).

Survey
Despite the potential usefulness of Twitter in interacting with students outside the classroom, the majority have used Twitter in classroom-only instruction environments. Figure  1 lists the most common practices of using Twitter for teaching. The most common practice was to engage students with outside resources (38, 20%), followed by discovering useful resources (32, 17%), and facilitating engagement and discussion among students (32, 17%). Other open-ended responses show novel uses of Twitter including using it as a data collection tool to teach analytics, help design social media campaigns, or even educate pre-service teachers. In these instances, we see that the intention of instructors is to use it as a secondary teaching tool.
We asked instructors about their future objectives in using Twitter in the classroom. The most cited objectives are: 'Expose students to practice' (37, 30%), 'Extend the range of the learning environment' (37, 30%), and 'Promote learning through collaboration' (34, 27%). Instructors overwhelmingly (47, 87%) report that they were able to meet their teaching objectives. In questions about what content students most often post on Twitter, it was revealed they commonly share opinions and commentary about course related topics, ask questions to the group, and post external links such as news articles and blog posts relating to the course topics. Students' contributions also evolve over time, moving from passive to active engagement as students become more familiar with the tool.
When asked about what analytics tools might help instructors understand students' activities on Twitter, they favored tools which summarize and automate data analysis. Some respondents indicated that they would like a way to make assessments about their students' success by qualitatively assessing their tweets. Instructors prefer to see visual reports and quantitative data such as the number of retweets/replies and posting frequencies.
To address instructors' stated objectives and improve the utility of using Twitter in teaching, we incorporate knowledge collected from the survey into the design of visualizations for a learning analytics dashboard.

Dashboard Interface Design
Among the many insights we obtained, usefulness of a LA dashboard is predicated on being able to determine the quality of tweets posted by students. Instructors should also be able to differentiate the types of content that is being shared and the patterns of usage, including relevant statistics on who is participating and how frequently. Following these observations from the survey data, we divided the interface of our prototype LA dashboard into three sections (Figure 2). The first section is activity data, such as the frequency of posts for the class (labelled 'A'). The second is content data, or information about the content itself and some automated summary that may help instructors to assess its quality or at least relevance (labelled 'B'). The third is user data, which captures the posting behavior and characteristics of students who are active on Twitter in the class (labelled 'C'). Each section (A-C) includes 2 or more visualizations that are grouped in physical proximity based on related data types.
Visualizations 1 and 2 in the first row labelled 'A' are charts showing the total number of tweets per day and per hour using a course hashtag, since some survey respondents expressed a preference for visual summaries of Twitter activity over time. These two visualizations show information derived from Twitter metadata and are designed to exhibit both an aggregated level of activities each day, as well as the times during the day that the class is most active.
Visualizations 3 to 8 in the second row labelled 'B' display information about the content properties of the tweets themselves. The stacked area graph (visualization 3) is meant to represent each content type as a segment in proportion of the total number of tweets, distributed across a timeline. Our study participants reported that students often post a range of content, from text to images to hashtags, and links to media and webpages. The height of each segment in visualization 3 represents relative frequency of each content type. Stacking each segment allows for comparisons between the most common and least common content types students tend to share. As revealed in our survey results, instructors have used or aimed to use Twitter as a way to facilitate discussions, as well as to engage with and introduce outside resources. It is, therefore, important that the dashboard affords instructors with the ability to make assessments on the degree to which discussion and engagement are occurring. Showing the presence of Twitter handles of class members, and the inclusion of URLs and media, means the instructor can draw inferences about both the degree of interpersonal exchanges and the capability of students to extend the learning environment through the inclusion of outside resources. For example, a high proportion of tweets with URLs might suggest that students are successfully engaging with topics by introducing external resources that are meaningful to the discussion. Conversely, a low proportion of user mentions may suggest a lack of discussion or interaction with other users.
Visualization 4 is a graph showing the type of tweets (retweets, replies and original tweets). This allows a visual comparison between whether students are simply redistributing others' content, if they are responding to others, or if they are creating their own content to share. This stacked histogram format, employed in previous LA dashboards, have been used to successfully allow for quick and accurate identification of the best and worst performances, (or in the case most and least frequent) in an aggregated view [5]. Like others [6], we strive to encourage and promote discussions on Twitter, and look to inform instructors about the presence of lowered engagement in classes which might occur in large audiences. This is important because academic engagement, or the time and effort students expend in education activities, is often linked to positive educational outcomes [2].
Visualizations 5 and 6 show frequency of sentiment types of tweets (positive, negative, or neutral) and subjectivity types (opinionated, non-opinionated), plotted over a period of time. We include this information since affective and emotional factors, among other aspects, have shown to impact the student motivation as well as the outcome of learning processes [7]. Tweets expressing negative sentiment, for example, could indicate frustration or boredom when engaging with a specific topic, whereas positive sentiment may reveal when students are excited to learn or are encouraging each other. The presence of subjective language reveals when students tend to offer opinions on a topic, in comparison to when they are sharing factual content with the class. This difference can highlight the distinction between merely sharing data versus actively reflecting and processing concepts.
Visualization 7 and 8 show the most frequently included hashtags (excluding the course hashtag), and the most frequently referenced named entities (persons, places and organizations). These provide clues about the topical focus of tweets. In a classroom environment, increased awareness about students learning activities, areas of difficulty, and incidents of engagement makes instructors more responsive to students' needs. On Twitter, the use of hashtags is typically included to help categorize the content of the tweet along topical lines. Whereas named entities are potentially meaningful to the discussion in that they represent launching points for further discussion, and expand knowledge through external references.
The third row, labelled 'C' includes visualizations 9-13, showing a more fine-grained summary about specific users. Visualization 9 shows users who have produced the most tweets, visualization 10 shows users with the highest number of negative tweets, and visualization 11, the highest number of positive tweets. The rationale for these visualizations is to indicate to the instructor students who might require intervention based on the pattern of affective properties in their communications. Having this awareness addresses the design goal of facilitating ways that instructors can intervene with individual students who might be experiencing specific difficulties.
Visualization 12 shows those users who are most often mentioned in other tweets. From this we provide indication of how interactive class discussions are, namely which students are contributing the most, and whether there are any students who are not particularly engaged with others on Twitter. Promoting learning through social interaction and collaboration is largely emphasized in effective collaborative systems, as learners need to engage with others and be active in one's own learning environment [8].
At the bottom of the interface is the visualization labelled '13' which provides a list of full text Twitter content produced under the course hashtag for a given day. Each row represents a single tweet, displaying both the full text content and the associated user handle that generated the original tweet. As a design feature, this table provides content which addresses survey responses where instructors expressed interest in a more qualitative assessment of tweets. One survey respondent reported that access to full text would allow them to understand the students' experience with the platform and gather suggestions on how improve Twitter strategies in the classroom. Furthermore, instructors would be able to make intuitive connections between the charts that summarize aspects of the data in other panels, and the raw data used to derive those summaries.

CONCLUSION
The proliferation of information and communication technologies like social media has expanded to its inclusion in teaching activities. The need to capitalize on potential benefits of social media for students and instructors alike and their effect on learning behaviors has inspired the development of tools that help make sense of interaction data from social media. In this work-in-progress paper, we outlined a Learning Analytics dashboard prototype based on the expressed objectives of surveyed higher education instructors. These objectives included the need to foster class discussion, to improve student engagement with external resources, and to promote sharing. Our dashboard summarizes Twitter content in a way that could provide further insights on how instructors may better comprehend interactions on social media, and thus improve teaching practices as these platforms are incorporated in classroom settings. Future instructor evaluation of our LA dashboard prototype will elaborate on the effectiveness and usefulness of the proposed design in meeting instructors' teaching objectives when using social media.