A collaborative multimedia annotation tool for enhancing knowledge sharing in CSCL

Knowledge sharing in computer supported collaborative learning (CSCL) requires intensive social interactions among participants, typically in the form of annotations. An annotation refers to an explicit expression of knowledge that is attached to a document to reveal the conceptual meanings of an annotator's implicit thoughts. In this research, we develop a Semantic Web and Web services-supported multimedia tool to facilitate collaborative annotation sharing in the context of CSCL. Our experimental results demonstrate that our tool can facilitate knowledge sharing and improve participating students' reading comprehension: it helps participants to raise good questions and provide proper answers through the practice of reading, commenting, reviewing, and discussion.


Introduction
The advance of computer science in the recent years paves a way toward a new style of cooperative learning. Computer supported collaboration techniques allow students to study in a virtual team without physically sitting at a common place (Chung, Severance, & Chung, 2003). Computer-supported collaborative learning (CSCL) was thus coined in 1996 (Koschmann, 1996) to refer to a computer and network-supported collaborative learning environment (CLE) for students to study cooperatively to acquire and share knowledge.
A CSCL-oriented software system is normally called a CLE (Littleton & Ha¨kkinen, 1999;Komis, Avouris, & Fidas, 2002;Kreijns, Kirschner, Jochems, & Buuren, 2007), focusing on using computer technologies to enhance interaction, communication, coordination, and interactivity within a learning group (Zurita & Nussbaum, 2004). Among others, the performance of a CLE is fundamentally decided by how effectively *Corresponding author. Email: jhyang@csie.ncu.edu.tw and efficiently it supports knowledge sharing in a formed learning group. Although there is no common definition for the term knowledge sharing, it is widely recognized that it involves collaborative efforts by knowledge consumers and knowledge producers to exchange ideas and experiences. Annotation-based collaboration is one important way to facilitate knowledge sharing.
The world standard organization W3C defines annotations as comments, notes, explanations, or other types of external remarks, which are attached to a document or a selected part of the document (Brickley, 2004). In general, annotations may help participants in four significant aspects: attention, organization, indexing, and discussion. Annotations may catch participants' attention, thus help them focus on annotated concepts or subjects. In addition, annotations could help participants construct their own knowledge and concepts. Furthermore, annotations could help participants bookmark annotated areas and use them as indexes or reminders for later references. Moreover, embedded annotations may help participants to discuss and review others' ideas and thoughts. Researchers have discovered that materials recorded as annotations are more likely to be recalled compared to those not recorded (Aiken, Thomas, & Shennum, 1975).
In a CLE, by reading and making annotations, participants collaboratively explore and exploit valuable ideas and knowledge. Experienced participants (e.g. experts and teachers) may also provide their annotations to help novices understand materials. Therefore, annotations can be considered as valuable knowledge in a CLE and thus require effective management. In recent years, a number of shared annotation provisioning systems have been developed (Russell, 2005), represented by W3C's Annotea (W3C, 2001b) and Microsoft's OneNote (Microsoft, 2007). Meanwhile, many researchers focus on standardizing annotation definition and description. The resulting formats have been proposed as metadata (Gasˇevic´, Jovanovic´, & Devedzˇic´, 2007), such as Dublin Core (Initiative), Learning Object Metadata (LOM), and Shareable Content Object Reference Model (SCORM).
Our research aims to apply the latest Semantic Web and Web services technologies to build a Web-based social software tool for facilitating knowledge sharing through multimedia annotation management in a CSCL group. In our previous work (Yang, Chen, & Shao, 2004), we develop a personalized annotation management system (PAMS) 1.0 as our kernel system, which provides a cluster of system-level facilities (i.e. annotation creation, editing, and retrieval) for users to annotate a shared document at various granularities. In this article, we report how we apply the Semantic Web and Web services technologies to build a Web-based PAMS 2.0 featured with interoperability. We explain how we leverage PAMS 2.0 to facilitate knowledge sharing in a CSCL context. We have also designed and conducted a series of experiments to evaluate the performance of our PAMS 2.0. Specifically, we examined whether PAMS could help participants to raise good questions and provide answers through the practice of reading, commenting, reviewing, and discussion. Our experimental results demonstrate that multimedia annotation-based collaboration facilitates knowledge sharing and significantly improves participating students' reading comprehension.
The remainder of the article is organized as follows. In Section 2, we discuss related work. In sections 3 and 4, we discuss how we built PAMS 2.0 and how it can be used to facilitate collaborative annotation management, respectively. In section 5, we present our experiments and discuss our findings. Conclusions are presented in section 6. 2.1. Knowledge sharing in CSCL Many CSCL researchers focus on how to use computer technologies to enhance interaction, communication, coordination, negotiation, and interactivity within a learning group. Kollias, Mamalougos, Vamvakoussi, Lakkala, and Vosniadou (2005) study, from a teacher's perspective, how Web-based CSCL can complement and improve classroom study. Kreijns et al. (2007) build two powerful learning environments, Conversant Media and meta-cognition. Manlove, Lazonder, and Jong (2006) conclude from a case study that CSCL tools promote student learning through scientific inquiry learning.
It has been demonstrated that social relationships have impact on knowledge acquisition in a collaboration mode (Fischer, Bruhn, Gra¨sel, & Mandl, 2002). Our previous work (Yang, Zhang, & Chen, 2007;Yang & Chen, 2008) proposes a social network-based Peer-to-Peer search method for identifying proper group study collaborators, who possess related knowledge and are willing to share corresponding knowledge. In contrast, this research builds a tool to facilitate knowledge sharing after a collaborative learning group is formed.
A term ''community of practice'' (CoP) is coined from social science to refer to a group of people who share a common interest and to study how they learn and deepen their knowledge and expertise through regular interactions (Wenger, McDermott, & Snyder, 2002;Alani, Dasmahapatra, O'Hara, & Shadbolt, 2003). Guldberg and Pilkington (2006) report that knowledge sharing is a motivation for most participants to use CoPs. Many CoP studies have elaborated on how to manage messages and documents posted in discussion forums.
Some researchers track and analyze interaction patterns to guide collaborative learning. Avouris, Komis, Margaritis, and Fiotakis (2004) design an environment for monitoring and examining group learning patterns from two aspects: activity analysis and collaboration analysis. Collaboration feedbacks are also used to enhance collaborative e-learning (Zumbach, Hillers, & Reimann, 2003). Bravo, Redondo, Ortega, and Verdejo (2006) build a learner-centered synchronous CSCL environment for students to study design. Three groups of researchers, Lundin and Magnusson (2003), Zurita and Nussbaum (2004), and Nicol and MacLeod (2005), each examines some Web-based case studies in a mobile environment using handheld devices. Through the development of two Web-based CLEs, Rubens, Emans, Leinonen, Skarmeta, and Simons (2005) summarize a set of pedagogical principles.
In summary, collaborative and cooperative learning approaches such as student team learning, group investigation, jigsaw, and dyadic cooperative learning, can be applied to many instructional scenarios (Hwang, Wang, & Sharples, 2007). Based on the pedagogy theories of collaborative and cooperative learning, the experiment class in our research was organized by group collaborative learning activities and an annotation system was provided to study whether multimedia annotation-based collaboration can facilitate knowledge sharing in a collaborative learning group. Aiken et al. (1975) find that materials recorded in a learner's notes are twice as likely to be recalled as those not recorded. Their experiments also show that the team taking notes shows better organization and verbal capability in tests. Sannomiya, Amagasa, Yokishawa, and Uemura (2001) identify the core functions that annotation sharing should comprise writing annotations, browsing annotations, and providing feedbacks. Carter, Churchill, Denoue, Helfman, and Nelson (2004) study the effectiveness of multimedia annotation by providing methods for people to post and acquire materials. They modify and annotate previously posted annotations to create publicly observable threads. Hwang et al. (2007) propose a multimedia annotation system oriented to Web-based learning. They develop Web-based tools for creating and sharing annotations, and conduct experiments at college level students. Their experimental results show the benefits of using annotation systems on both individual learning and group learning. However, their systems lack a direct relationship between semantic technology and multimedia annotation, which is important for Web-based annotation query and management.

Multimedia annotation and annotation management
W3C has proposed a Web-based shared annotation system based on a generalpurpose open Resource Definition Framework (RDF) infrastructure, called Annotea (W3C, 2001b). Users use a Web-based Amaya editor (W3C, 2001a;Ran, 2003) to browse Web content and make annotations in Annotea. The annotations can be stored either on annotation servers or on local machines. To associate annotations with annotated objects, Annotea utilizes the XPointer technology to insert annotation positions into the annotated document in XML. However, XPointer requires that an XML document labels the id attribute of every element that may be associated with annotations. The overhead of maintaining linking relations makes XPointer only suitable for static documents but not for frequently changed documents. Furthermore, XPointer can only be applied to XML documents; therefore, Annotea only supports XML-based annotation. In contrast, our proposed tool is not limited to XML documents, and our tool does not require a document to be labeled before it can be annotated.
Microsoft has published OneNote that is a personalizable annotation tool. Users can choose a pen, a keyboard or voice to input annotations. However, notes can only be shared through e-mails. As a result, the usage of the tool is largely limited. IBM's Lotus Notes (IBM) allows discussions on a document. However, all comments can only be attached to the entire document, instead of to its comprising sections. CommonSpace (Surman & Wershler-Henr, 2002) allows interactions between teachers and students. A student can write annotations on a document; a teacher can comment on the annotations; and the student can further revise the annotations based on the teacher's comments. Meanwhile, the teacher can view all the annotations and comments attached to the document. However, the system is a stand-alone software product, not designed for Internet-based collaborations. In contrast with these annotation approaches that bind annotations directly to their corresponding annotated areas, our PAMS separates the storage of annotations from their corresponding annotated areas so that the annotations can be managed and retrieved independently. This feature is critical when multiple groups of people annotate on the same shared document simultaneously, which is an important requirement for a Web-based annotation system.

PAMS 2.0
In a CSCL context, we define an annotation as an explicit expression of knowledge of a participant (annotator) in the form of a comment (note) or a question, which reveals conceptual meanings of the annotator's implicit thoughts. In this research, we call people's annotations as ''personalized annotations,'' in the sense that each annotation is made by one individual to represent his/her personal understanding and thoughts. Therefore, an annotation is typically featured by the annotator's personal characteristics and preferences.
Our previous work yields a PAMS 1.0 (Yang et al., 2004). Upon a shared document, users can not only create and edit annotations but also query about related annotations. In this project, we intend to upgrade PAMS and build a Webbased collaborative annotation environment.

Annotation modeling
A Web-based annotation management system has to take into consideration its potentially much broader user base and much more complex usage scenarios. For example, a document may be simultaneously studied and annotated by multiple learning groups. Therefore, it may be desired that all annotation activities be associated with the corresponding learning groups. To enable effective and efficient annotation management, our strategy is to model annotation-related concepts using the Semantic Web techniques. Specially, we introduce a concept of annotator group to delimit the domain of a learning group, and we decouple annotations from annotated objects (portion) for flexibility.
We identify six conceptual elements regarding annotations. Figure 1 shows our annotation model as an ontological Entity-Relation model to illustrate these conceptual elements. A document with annotations is referred to an annotated document. Annotations can be created by any annotator upon any part (area) in the document, such as a word, a sentence, a section, or even the entire document. An area Figure 1. A PAMS 2.0 annotation entity-relation model. associated with annotations is called an annotated object. Annotated documents may allow role-based access control (e.g. an annotator may have read and write privileges to an annotated document). All annotators having the same access privilege (role) to a particular annotated document form an annotator group. The relationship between a specific annotation with its corresponding annotated object, annotated document, and annotator is recorded in an association. The relationship between a specific annotator with its corresponding role, annotated document, and annotator group is recorded in an assignment.

WSRF-based annotation modeling
We examined different techniques to model this annotation model at both the metadata level and the interface level. Our goal is to enable and facilitate interoperability between annotation provisioning systems. For the metadata level, we examined RDF Schema. The hyperlink structure of RDF Schema provides flexibility and versatility to support various data formats required in collaborative multimedia annotation retrieval and exchanges.
Therefore, RDF Schema is suitable for defining annotation metadata. For the interface level, we focus on finding a mechanism that can provide a standard interface for annotation resources. Recall that our goal is to allow different Webbased annotation management systems share recorded annotations. For example, it may be neither practical nor desirable for every user to use the same annotation management system, and one may expect to have access to some annotations recorded by another annotation management system. We investigated the Web Services Resource Framework (WSRF, 2004) technology that is an XML-based presentation method to capture and describe resources using the Web services technology.
The rapidly emerged Web services technology enables interoperability among software modules and components regardless of their underlying platforms and programming languages. A software module wrapped with a Web service interface becomes universally accessible using standard communication protocols. Particularly, WSRF provides a technique to model a universal resource with a Web services interface and access points. We adopted WSRF in this project for three major reasons. First, we use it to define the interface and enable interactions between annotation knowledge through Web services interfaces. Second, WSRF allows us to model stateful annotation knowledge, so that conversational status can be maintained and managed. Third, WSRF does not constrain the XML Schemas used to describe the resource data. Instead, WSRF handles application data (resource properties) in the form of XML and the application data can be described using XML Schema. Therefore, we can annotate WSRF resource schemas with semantics such as RDF.
Applying the WSRF technique, we model an annotation object as a universal resource. When an annotator tries to comment on an annotated document for the first time, the annotated object is identified as a resource and wrapped by a Web services interface. Annotations are modeled as Web services-wrapped resources as well. Figure 2 shows a simplified example of resource properties document ''Annotation1.'' The WS-Resource properties specification document is defined using XML Schema. The state of the modeling resource contains two components: annotation_id and annotation_title, with XML Schema Definition (XSD) type integer and string, respectively. For a user to know that the ''Annotation1'' defines the WS-Resource properties document associated with the Web service, the WS-Resource properties document declaration is associated with the Web Service Definition Language (WSDL) portType definition in the WSDL definition of the Web service interface, through the use of a standard attribute resourceProperties. As a result, as shown in Figure 2, the portType, with the associated resource properties document, defines the type of the WS-Resource.
Modeling annotated objects and annotations as separate WSRF resources provides several significant advantages. First, annotated documents and their annotations may be stored and managed by different groups at different locations, and be integrated or bound together on an on-demand basis. Second, this strategy enables and facilitates dynamic creation of the properties and associated values of annotation knowledge. Third, querying and updating the property values of annotation knowledge can be realized through standard message exchanges.

PAMS 2.0
On the basis of our ontological metadata presentation technique, we built PAMS 2.0 as a Web-based collaborative annotation management system. For the rest of the article, PAMS refers to PAMS version 2.0. As shown in Figure 3, PAMS 2.0 adopts a variation of the client/server architecture, whose system architecture comprises two separate parts: an annotator side and a server side. On the server side, four managers collaborate to provide four system-level supporting services: anchoring management (dynamically bind annotations to corresponding annotated sections), document management, association management, and annotator management. The technical details of implementing the four managers can be found in our previous report (Yang et al., 2004). Data recording annotations, annotated objects, annotated documents, their associations, and corresponding annotators are stored in four repositories: annotation repository, annotated document repository, association repository, and annotator repository.
On the annotator side, four handlers collaborate to process annotation creation, retrieval, discussion, and management. Annotators can create, edit, and retrieve their own annotations. In addition, annotators can retrieve annotations from other annotators in the same annotator group. Furthermore, annotators can discuss with each other through PAMS. Admission and access control is handled by the annotator manager, using a predefined set of roles. In the current version of PAMS, we realize a straightforward access control. An annotated document can only be freely annotated by the members from a predefined annotator group. The creator of a specific annotation may constraint that only certain roles in the annotator group have access to the annotation. New annotators may join in the group in the run time. More comprehensive and customizable access control mechanism will be our future research topic.

PAMS 2.0 for knowledge sharing
We apply our PAMS 2.0 to CSCL as a Web-based collaborative annotation tool to facilitate students in creating and sharing annotations over the Internet. Most of the collaborative annotation systems use electronic messages to inform the creation of a shared annotation. By contrast, our PAMS 2.0 supports collaboration on shared annotation using an online discussion board. Annotators can comment on, answer and/or ask questions on created annotations. This allows participating annotators to exchange their annotations directly on the annotated document.
4.1. Online annotation for knowledge sharing PAMS 2.0 provides a cluster of system-level supporting services such as annotation creation, editing, and retrieval. Four categories of annotation are supported in PAMS 2.0: (1) definitions (e.g. descriptions and explanations), (2) comments (e.g. opinions and arguments), (3) questions (e.g. problems and answers), and (4) associations (e.g. additional links to other resources). Each category represents a corresponding annotation type describing certain semantic meanings of annotators' intention. PAMS 2.0 supports multiple formats of annotated documents including pdf, Microsoft Word, and HTML Web pages. An annotated object may be a word, a phrase, a sentence, an icon, an image, or a block in an annotated document. An annotator may make annotations on an annotated object in various forms such as text, image, and voice. Figure 4 shows a screen shot of PAMS 2.0, where a user has created some annotations on a portion of the annotated document. Detailed annotation information is recorded, based on the semantic annotation definition items discussed in (Yang et al., 2004), including the location of the annotated object, the annotator, comment text content and its color, and a link to the recorded voice comment.  Figure 5 shows the Graphical User Interface (GUI) of PAMS 2.0 for an annotator to make annotations on a shared document. As shown in the screen shot, the central component of the GUI is a document browser, where an annotator can view the document as well as associated annotations. PAMS 2.0 integrates an input schema editor that allows annotators to specify the preferred target format such as length and type. Figure 5 also shows that annotators can attach annotation files to annotations, by specifying the location of the file in the Input Schema panel.

4.2.
Online discussion for knowledge sharing Figure 6 illustrates a screen shot of how PAMS 2.0 helps annotators further share knowledge through discussing and exchanging annotations. A group user (Alex) selects an annotation document and a collaborator (Eric) to initiate a private communication channel ''chat.'' The shared document is shown on both users' screens. One user (Alex) creates an annotation; the other user (Eric) receives the annotation; and the latter (Eric) responses to the annotation by typing some answers in the chat window. As shown in Figure 6, all annotation-related activities are recorded by PAMS for later annotation retrieval.

Experiments and discussions
We have designed and conducted a series of experiments to evaluate whether PAMS 2.0, as a Web-based annotation system, could enhance knowledge sharing in a collaborative e-learning context. Specifically, we examine whether PAMS 2.0 could help participants raise good questions and provide answers through the practice of reading, commenting, reviewing, and discussion. The rationale behind this experiment is that the efficiency of reading comprehension should be evaluated not  only by tests (e.g. examinations and assessments) but also by participants' ability to raise good questions and provide proper answers (Barak & Rafaeli, 2004).

Experimental design
We utilize PAMS 2.0 for students to practice online group reading and commenting, in order to enhance communication and collaboration among participants in a CSCL context. A group of student volunteers were randomly divided into two teams: team 1 (experiment class) that uses PAMS in their group reading activity and team 2 (counterpart class) that does not use PAMS. Each team forms a collaborative elearning group using the corresponding platform: team 1 uses PAMS as their platform; team 2 uses other channels (emails, instant messengers, and paper documents). Tutorials and demos were given to team 1 students prior to the experiments to teach them how to utilize PAMS to read articles, annotate on the articles, find additional materials related to the articles, find other collaborators in a community of practice who are also reading the same articles, and find participants' annotations on the articles.
In one experiment, both teams were given the same reading materials. Each member in a team read, commented, raised questions, and provided answers, while discussing with other collaborators in the same team. Team members may comment on each other's annotations. At the beginning of the experiment, each team identified a team leader, who was in charge of documenting and aggregating these aforementioned collaborative events. At the end of each experiment, each team turned in a detailed team comment report, together with a set of questions raised by the team members. Accumulated questions from both teams form a question bank. We then conducted a test to evaluate participants' reading comprehension regarding the assigned materials. The examination paper was compiled by the instructor by selecting questions from the question bank provided by both teams. Every participant was required to take the examination.
The experiments were conducted at National Central University in the Spring semester of 2007. Ninety-four junior students volunteered in the experiment. To ensure that the two teams have similar reading ability, we required that every volunteered student take a reading test prior to the experiments. All students were sorted based on the scores they obtained from the test; then they were alternatively assigned to the two teams. Forty-eight students were assigned to team 1 and 46 to team 2.
Lasting for the entire semester, the experiments comprised five iterations, each lasting 3 weeks. In each iteration, a set of group reading materials (articles) were assigned to both teams. At the beginning of each iteration, we gave the same test to all students in both teams; at the end of each iteration, both teams took another test generated from the question bank. The reason why we adopted five iterations instead of one was threefold. First, we tried to eliminate random issues that may affect our evaluation. Second, the students in the experiment class (use PAMS 2.0) might feel more motivated to discuss when they first used our PAMS 2.0 as a new tool, so their early test scores might not be accurate. Third, such a multiple-iteration-test assures a controlled test scenario. With five iterations being applied to the same team, we expect more reliable results since our tests remain their same reading backgrounds. In addition, in such an experimental setting, we can ensure a relatively decent number of students in every test sample (about 47 students).

Experimental results
To evaluate whether PAMS helped collaborative e-learning, we compared the average scores between the two teams before and after group reading activity, as shown in Table 1. For each test, the scores of all students in the same team were accumulated and an arithmetic mean was calculated. The mean values obtained from the five iterations were accumulated again to reach a final average score. In addition, for each team in every iteration, we calculated the mean deviation that is the mean of the distances between each score and the mean value. The mean deviation gave us an idea of how spread out from the average score the set of values were. Table 1 lists our experimental results. The students in the experiment class achieved averagely much higher scores after group reading using PAMS (76 vs. 56). The students in the counterpart class made less progress in their scores after group reading using other methods (65 vs. 58). The mean deviation values shown in Table 1 indicate that our evaluation results are reliable.
To better illustrate our experimental findings, the evaluation results are summarized in Figure 7. Before group reading, the average scores of both teams are similar. After group reading activity, team 1 using PAMS significantly exceeds team 2 in material understanding. This experimental result demonstrates that collaborative annotation helps to improve students' reading comprehension in a collaborative e-learning context.
We also designed a questionnaire with a list of yes/no questions for the students in team 1 at the end of the experiments, for examining whether they prefer using PAMS for group e-learning and the aspects they like it. The answers were positive, and the top three reasons why the students like to use PAMS are its capability of helping students to (1) comment, (2) discuss, and (3) raise question-answers.
Through the survey, the students confirmed that in addition to enhancing their thinking skills by practicing personalized annotations, PAMS encourages their attitudes toward knowledge sharing. The students also confirmed that by making annotations and discussing with other annotators in the study group through related annotations, PAMS facilitates knowledge sharing.
We further studied the data from team 1 by examining the relationships between individual students' examination performance and the number of questions raised by them as well as the number of questions adopted in the examination paper. As shown in Figure 8, by using PAMS, students' examination performance improved when they raised more questions. This indicates that the more questions students raise, the more actively they involve in knowledge sharing, and the better their examination performance.
In summary, we found that a Web-based annotation system can be applied to the collaborative e-learning field to foster knowledge sharing by enhancing interactive communication and collaboration among participating students.

Conclusions
CSCL has become an important way to enhance learning and teaching. The behaviors of making marks on reading documents are important in CSCL. Multimedia annotation-based collaboration can improve the process of reading as well as student's comprehension of learning materials. In this article, we have presented our PAMS 2.0 as a Web-based social software tool for collaborative e-learning. Our study demonstrates that collaborative multimedia annotation enhances knowledge sharing in students' online group reading activity. Our research also demonstrates that the Semantic Web and Web services technologies can be applied to enhance interoperability of collaborative e-learning software.
We expect that PAMS can be extended and commercialized to support collaborative e-learning and knowledge sharing in two ways: either as an integral component to plug into an online collaboration platform or as an independent Web service to serve as an underlying collaboration platform. Furthermore, we expect that PAMS can be extended to support a broad sense of Internet-based communication and collaboration in a context of a community of practice, not limited to e-learning but expanded to business and social interactions.
In the current version of PAMS, our role-based admission and access control is straightforward and rather simple. We plan to construct more comprehensive role-based access control facility allowing collaborators to dynamic configure and reconfigure access control rules. We also plan to construct a dedicated Web site to host our PAMS that is universally accessible to all Internet users. In addition, we plan to explore the feasibility and performance of using PAMS (including annotation creation and browsing) through various mobile devices (e.g. PDAs and mobile phones). We plan to mobilize the discussion facility of PAMS and to make the annotation retrieval more context-aware and social-aware.