Evaluations of a Large Topic Map as a Knowledge Organization Tool for Supporting Self-Regulated Learning

A large topic map was created to facilitate understanding of the impacts of the 2010 Gulf of Mexico Oil Spill Incident. It has a text topic map and a graphical topic map, which complement each other. A formative evaluation and two summative evaluations were conducted, as qualitative studies, to assess the usefulness and usability of the large topic maps as self-regulated learning tools. The topic maps were found useful for knowledge fusion and discovery, and can be useful when undertaking interdisciplinary and multidisciplinary research. Users reported some usability issues about the graphical topic map, including information overload and cluttered display of topics when displaying large number of topics and their associated topics. The text topic map was found easier to use due to displaying topics, relationships and references in a linear view.


Introduction
The 2010 Gulf of Mexico Oil Spill Incident (White House, 2012) has impacted many aspects of the coastal environment of the Gulf of Mexico and the people living in the coastal states.During the effort to stop and clean up the spill, the entities involved with the process, including state and federal agencies and contracted agents, have either conducted or contracted for numerous studies of the Gulf of Mexico as a part of their operations.These studies have published a considerable quantity of literature through news, conferences, journals, and the Web.
Government officials, journalists and the general public wanted to get a general understanding of the impact, and Gulf-based researchers wanted to investigate the fate and effects of oil, dispersed oil, and dispersant on the ecosystems of the Gulf of Mexico and affected coastal states.Furthermore, researchers are also interested in literature of oil spill incidents that happened elsewhere in the world.
To facilitate understanding of the oil spill impacts, a large oil spill topic map was created to reveal "what does what" and to organize information resources.The large topic map contains approximately 5,000 concepts and 2,000 relationships extracted from about 300 documents comprised of conference presentations, journal articles, news reports, and authoritative Web pages.The concepts and relationships were loaded to open source topic map creation tools Ontopia and Wandora (introduced below) to build a graphical topic map and a text topic map.
During the development process, a formative evaluation and two summative evaluations were conducted to assess the usefulness and usability of the topic map.The purpose of the paper is to report the design and outcomes of these evaluation studies.The contribution of the paper lies in discovering the strengths and weaknesses of the large topic map as a complex knowledge organization tool for facilitating self-regulated learning through formative and summative evaluations.To the authors' knowledge, this is the first study in evaluating a large topic map as a complex knowledge organization tool for self-regulated learning.
This paper is organized as follows: a brief introduction to the features of topic maps and the Oil Spill Topic Map is given in the next section.Related Work discusses the role of concept map in learning, compares concept maps with topic maps, and introduces formative and summative evaluation methods.The Research Questions and Method section presents the research questions and the method of investigating the research questions.The Data Analysis and Findings section presents the data analysis approach and evaluation results, and the Conclusion and Discussion section presents conclusions and limitations, and outlines future work.

Features of Topic Maps and the Oil Spill Topic Map
Topic maps are a semantic approach to information organization.They are designed to enhance navigation and information retrieval in large sets of information resources (Hatzigaidas et al., 2004), and there are numerous successful information organization projects using topic maps (Garshol, 2004;Shih et al., 2007).Topic maps are based on an international standard (i.e., ISO 13250), and are interchanged using the Extensible Markup Language (XML) defined by the W3C (Ontopia, 2002).A topic map has three constructs: (1) topics, which represent "subjects of discourse", (2) associations, which represent relationships between the subjects, (3) occurrences, which are information resources relevant to a given topic.
Topic maps have built into them the relationship between topics and between the topic and the occurrences of the topic in information resources.For example, cleanup efforts (a topic) were suspended by (a relationship) severe weather (a topic).Each topic has instances (or occurrences) of relevant information resources.The reader, by scanning the topics and relationships, can get a big picture of the topics and focus on those information resources that relate most significantly to his/her interest.
The Oil Spill Topic Map was built by integrating Ontopia and Wandora that run on a Web server (Wu & Dunaway, 2013).Both Ontopia and Wandora are open source topic map creation tools.Ontopia seems to be the most famous topic map creation tool, probably due to Pepper (2000).It has an excellent graphical visualization component, but does not take batch input of topics, associations, and occurrences in text file format.Wandora (Wandora, 2012) was used to complement Ontopia because it supports batch input in text file format, but has a less appealing graphical visualization component.Ontopia requires Java to run on a browser.
The Oil Spill Topic Map has two user interfaces: text user interface (called text topic map, see Figure 1) and graphical user interface (called graphical topic map, see Figure 2).The text topic map is ideal for searching for a topic, and viewing its directly associated topics, relationships, and references.It has several strengths and a weakness.First, it presents a linear display of a topic, its directly associated topics, their relationships and references.The linear display won't clutter.Second, the relationships are presented as directional.However, it is not ideal for viewing a network (or a chain) of topics.The graphical topic map is ideal for visualizing topics, their associated topics and relationships, and navigating the topic space.Its strength lies in presenting a network (or a chain) of topics and their relationships non-linearly, which allows users to explore the topic space.However, it has two weaknesses.First, a network of topics can be cluttered when a large number of topics are presented.This is a weakness of Ontopia.Second, the relationships are non-directional.For instance, cleanup efforts <be suspended by> severe weather can also be interpreted as severe weather <be suspended by> cleanup efforts.This is a general weakness of the topic maps standard.
The role of topic maps in learning has not been studied systematically whereas the role of concept maps has.In the next section, a brief introduction to the role of concept map in learning, and comparisons between concept map and topic maps, and between formative and summative evaluation are provided as a literature review of related concepts and theoretical framework.

Related Work
There are papers exploring how a topic maps-based ontology approach affects users' searching performance (Yi, 2008), and introducing the use of topic maps in the library field and how they might be integrated into the ILS structure (Iglesias & Hye, 2008).There is no published paper presenting formal evaluations of topic maps for learning.Rittershofer (2005) conducted some informal evaluation of the a topic map for course learning by asking the students after having finished a course with self-regulated elearning, and found that the students were always very satisfied with using the topic map for learning.No formal evaluation of topic maps for facilitating learning has been found.However, concept map and concept mapping have been found to facilitate learning.

The Role of Concept Maps and Concept Mapping in Learning
Digital concept maps, which can present topics, relationships, and information resources, may be helpful as cognitive tools for self-regulated students in open resourcebased e-learning scenarios (Rakes, 1996).Both topic maps and concept maps are knowledge visualization tools."Knowledge visualization may help students to organize and reorganize, structure and restructure, assess, evaluate, elaborate, communicate, and (co-)construct knowledge, and to utilize ideas and thoughts, as well as knowledge, about relevant content and resources" (Keller & Tergan, 2005, p.6).
In relation to knowledge visualization and topic maps, concept maps have been used in education for more than forty years.Concept maps are graphical tools for organizing and representing knowledge, which include concepts and relationships between concepts (Novak & Cañas, 2008).They are intended to represent the knowledge structures that human beings store in their minds (Jonassen, Beissner, & Yacci, 1993).Wang et al. (2011) reported a visualized knowledge map "JAVA e-teacher" by integrating the visualization of domain knowledge structure with curriculum design, learning resources, learning assessment, and social learning.The knowledge map can be considered as an extended concept map.Their small-scale survey and interview with students who completed self-regulated learning of an entry-level Java-programming course using the system demonstrated high user satisfaction and acceptance.
Concept maps are useful tools for designers to structure their knowledge, but "readymade concept maps might not be so useful for learning.For the learner, the process of creating and/or modifying concept maps seems to be much better" (Bruillard & Baron, 2000, p.331).Theoretically, concept mapping can be an effective way of learning because it requires explication and reflection (making explicit what is normally implicit) and helps students develop auto-monitoring techniques and so enhance their critical thinking skills (Hammond, 1994;Bruillard & Baron, 2000).Practically, concept mapping has shown to be an effective technique for representing knowledge (Wang et al, 2011).When properly used, it has been shown to help learners learn (Novak & Cañas, 2008).

A Comparison between Concept Maps and Topic Maps
Traditional paper and pencil-based concept maps can be used for the representation of "know-what" (i.e., concepts) and "know-how" (i.e., semantic relations of concepts) but not "know-where" (i.e., information related to the concepts) (Keller & Tergan, 2005).Digital concept maps allows for the representation of all the three, that is, concept knowledge, content knowledge, and resource knowledge in a coherent representational format (Tergan, 2005).Therefore, both digital concept maps and topic maps can represent all the three.Advanced digital concept mapping tools (such as CmapTools) can "enable a knowledge-based visual organization, search, and access of conceptual knowledge, content knowledge, and related information" (Tergan, 2005, p. 192).Advanced topic map tools (such as Wandora) have the similar functionality.
Concept maps and topic maps have some differences.The first difference lies in the directionality of relationships.Concept maps are directed graphs (Coffey 2005), that is the relationships between two concepts are directed.Topic maps are non-directed graphs.That is, A impacts B is the same as B impacts A. The second difference lies in their construction approach.Since concept maps are used for representing the designer's knowledge of a domain in his mind, concept maps are often built using a top-down approach.As a result, "concept maps structure a set of concept into a semi-hierarchical framework.More general, inclusive, superordinate concepts are found at the highest levels, with progressively more specific, less inclusive, subordinate concepts arranged below them" (Coffey, 2005, p. 288).Topic maps require a taxonomy of a domain for classifying a concept into a category in the taxonomy, but the designer does not have to use a hierarchical framework.Both top-down and bottom-up approaches can be used to build topic maps.As a result, topic maps may not present a superordinate-subordinate knowledge structure.

Formative and Summative Evaluation
A distinction between formative evaluation and summative evaluation in instructional design was indicated by Scriven (1967).Formative evaluation was intended to foster development and improvement within an ongoing object being evaluated (such as activity, program, and person).Summative evaluation, in contrast, is used to assess whether the results of the object being evaluated met the stated goals (Scriven, 1967;Clark, 1995).In human-computer interface design, formative evaluation is used to help improve the interface as part of an iterative design process, while summative evaluation is used to assess the overall quality of the interface (Nielson, 1993).In this study, formative evaluation was used to help improve the topic maps as a part of the design process, while summative evaluation was used to assess the usefulness and usability of the topic maps when the design was completed.

Research Questions and Method
This study aims to investigate two research questions: 1. Are the oil spill topic maps (i.e., the text topic map and the graphical topic map) useful as a self-regulated learning tool for facilitating the understanding of the impact of the Gulf of Mexico Oil Spill Incident?2. Are the topic maps usable as a self-regulated learning tool?
A formative evaluation and two summative evaluations were conducted to assess the usefulness and usability of the topic maps as self-regulated learning tools.The evaluation experiments are qualitative studies, which aim to identify strengths and weaknesses of the topic maps in terms of usefulness and usability.Therefore a large sample of test subjects was not pursued.Although quantitative data was collected and analyzed, it served description purpose, and no statistical significance tests were performed.The participants of the evaluation experiments were recruited as convenience samples.Every participant's evaluation outcome was analyzed to provide insights into the usefulness and usability of the topic maps.
Immediately after the initial topic maps were developed, a formative evaluation was conducted with 68 high school students in a chemistry class of the Louisiana State University (LSU) Laboratory School.The formative evaluation was focused on the graphical topic map because of two reasons: the text topic map was relatively straightforward to use and it would be too much work for the students to evaluate two topic maps in two hours.

Design of Formative Evaluation
A tutorial was developed to teach the students how to manoeuver the graphical topic map, such as setting topics sticky, expanding/collapsing/hiding topics, viewing relationship between two topics, and showing information resources.Due to scheduling issues, only 11 students could attend our on-site demo of the topic maps.The students were not able to complete the evaluation on site because the school happened to be testing its Internet on that day.All the participants were asked to complete their evaluations at home as a lagniappe assignment for the chemistry class.The assignment has two partsa questionnaire and a short essay.The questionnaire includes three openended or semi-open-ended questions about the topic maps (see the next section for the questions).The purpose of the essay was to urge the participants to use the topic maps (especially the graphical topic map).However, it turned out that students could still bypass the graphical topic map and write the essay using external information resources.Their essays were collected but not analyzed.
The students were instructed to complete the evaluation in two hours.They were instructed to study the tutorial for 20 minutes, to explore the topic maps (with a focus on the graphical topic map) with a topic of their interests for 40 minutes, and then to write a one-page essay for 40 minutes, and to complete the questionnaire for 20 minutes.Preferably more time should be assigned to each task, especially training, in order to achieve better performance on each task, but requiring more than two hours may discourage participation.To answer the two research questions, the following questions were asked to the students in the questionnaire: FE_Q1: How useful is the topic map in facilitating the understanding of the topic(s) you have explored?You may talk about whether the integrated knowledge in the topic map speeded up or slowed down the understanding of the oil spill incident, and whether the graphical presentation of the topic map helped or hindered your understanding of knowledge.
FE_Q2: What new knowledge have you learned or discovered by using the topic map?You may talk about your most interesting learning experience (such as anything you did not know before) when exploring the topic map.
FE_Q3: Did you have any problems starting and navigating through the topic map?If yes, what are they?
Please select one of the following two options: (1) __ It is easy to find out where to start.
(2) __ It is difficult to find out where to start.If this is the case, please tell us where you wanted to start and how you figured out how to start.
Please select one of the following two options: (3) __ It is easy to navigate through the topic map.
(4) __ It is difficult to navigate through the topic map.If this is the case, please tell us what you wanted to do using the topic map, and what made it difficult for you.
The findings of the formative evaluation were taken into consideration for the improvement of the topic maps.When the project was close to its end, two summative evaluations were conducted, one with eight undergraduate and graduate students from various departments at LSU, and the other with five professors from various departments at LSU.

Design of Summative Evaluation 1
For the eight undergraduate and graduate students, their evaluations were completed at our laboratory in two hours per person.They were first provided with an online tutorial and a face-to-face training or demo of how to manoeuver the text and graphical topic maps for 15 minutes.Then they were instructed to explore the topic maps with a topic (or two if necessary) of their interests for 40 minutes, and to write a 400-600 word essay on the topic(s) they have explored in 40 minutes, discussing what new knowledge they have learned about the topic(s) and how they used the topic maps to learn the topic(s).Finally they were asked to complete the questionnaire for 15 minutes.Each participant was paid with $24 for their time.The following questions were asked in the questionnaire: SE1_Q1: same as FE_Q1.
SE1_Q2: What functionalities do you wish to have in an information/knowledge organization tool like the topic map? (College students are expected to be able to answer this question.)SE1_Q3: What are the most positive and negative experience that you have had with the text/graphic topic map? (This is a paraphrase of FE_Q3 but is broader.)Note that FE_Q2 is expected to be addressed in the essay.

Design of Summative Evaluation 2
For the five professors, they were provided with the online tutorial on how to use the topic maps, but no face-to-face training or demo was provided, assuming they were technologically savvy and could teach themselves how to operate the topic maps by following the online tutorial.They were instructed to explore the topic maps and to complete a questionnaire in two hours.Each participant was paid with $300 for their time.The following questions were asked in the questionnaire: SE2_Q1: same as SE1_Q1.SE2_Q2: same as SE1_Q2.SE2_Q3: Can the text/graphic topic map useful to you?If yes, for what purpose would you use the graphic/text topic map?SE2_Q4: same as SE1_Q3.SE2_Q5: Please provide any comments on improving the topic map as a knowledge organization and knowledge discovery tool.

Data Analysis and Findings
The participants submitted their responses to the questionnaires and rich linguistic description of their comments on the usefulness and usability of the topic maps.Qualitative data analysis was applied to the evaluation data to identify the categories of usefulness and usability issues.Quantitative data of each category was collected to serve description purpose.No statistical significance tests were performed on the quantitative data because quantitative research methods do not serve to answer our research questions, and no control groups were set up for statistical significance tests.

Formative Evaluation
In the formative evaluation, the 68 participants were classified into two groups for data analysis purpose.Group A consists of 11 students who participated in the demo.Ten of them submitted the questionnaire.Group B consists of 57 students who did not participate in the demo, but only 37 of whom submitted the questionnaire.
Two groups of feedbacks were collected from the 47 students who submitted the questionnaires.A small group of students could not figure out how to operate the graphical topic map, either due to not being able to make Java run on their browser, or not being able to follow the instructions to operate the graphical topic map.Consequently they reported that they did not learn anything from using the graphical topic map.Most of the students were able to operate the graphical topic map, and reported both positive and negative comments.
Annotator A (first author) and Annotator B (second author) analyzed (or annotated) the questionnaires.Since there is a limited number of categories of responses for usefulness (e.g., useful, useful with some issues, not useful) and for usability (e.g., usable, usable with some issues, not usable), it is possible for two annotators to reach a consensus about the categories.Therefore a coordination approach was taken by the two annotators to generate the categories.Annotator A processed the participants' responses to the questionnaire to generate an initial taxonomy of responses.It was supplemented and revised by Annotator B when she processed the responses more thoroughly.Then Annotator A refined and finalized the taxonomy.Annotator B categorized the participants' responses according to the finalized taxonomy.Annotator A slightly revised her classification during data analysis.The categories of responses to each question are presented in Table 1, 2, and 3. Note that in the tables, the entries starting with a (x) where x is a number are response categories, the entries starting with a "─" under a category are typical instances of a response category.Number of responses and percentage of respondents are assigned to categories and may not be assigned to instances.A check mark (√) indicates that a certain typical instance belongs to a certain group (either Group A or B).Note that a respondent's responses can span across categories (e.g., the topic map is useful in one sense, but not useful in the other sense), the percentage of respondents do not add up to 100%, and is often bigger than 100%.about the topic when clicking on it, instead of expanding the topic to show related topics √ The formative evaluation was an exploratory study.The quantitative data shown in the tables serves a description purpose.The response categories which were generated from the qualitative data provide more insights into the usefulness and usability of the topic maps.
It is expected that the training or demo (which was provide to Group A only) helped users maneuver the topic maps.The data shown in the above three tables confirms the expectation quantitatively.Table 3 shows that Group A reported higher rate of usability problems than Group B did.This implies that the training or demo session (which was provided to Group A) might have helped the students use the graphical topic map.However, there is still a high percentage of respondents in Group A found it difficult to start (30%) and navigate through the topic map (50%).Two reasons might explain this.One is that there is a learning curve in operating the large topic map as a complex knowledge organization tool.The other is the low motivation (a lagniappe assignment for the chemistry class) that did not stimulate some participants to conquer the learning curve.Since knowing how to operate the topic map is the prerequisite to evaluating the usefulness of the tool, the high percentage of "not useful" shown in Table 1 and 2 may be partially explained by the real usability issues in the topic map and the low motivation of some participants in conquering the learning curve.

Summary of Formative Evaluation
Although 100% of Group A and 81% of Group B participants found the topic map useful in some aspects, 30% of Group A and 40% of Group B participants found it not useful in other aspects.In terms of usability, 30% of Group A and 50% of Group B participants found it difficult to find out where to start the graphical topic map, 50% of Group A and 61% of Group B participants found it difficult to navigate.
There is a learning curve in operating the large topic map as a complex knowledge organization tool.The low motivation of conquering the learning curve might have increased the reporting of usability problems.Since evaluating the usefulness of the topic map requires the understanding of how to operate it, usefulness evaluation may have been negatively affected by the learning curve and usability problems.
Major usability issues identified by the participants include: -being unable to understand how to operate the complex tool, -information overload (displaying large number of associated topics at once), -cluttered display of topics when a large number of topics are displayed at once, -inaccessible facts and/or information resources about topics (due to copyright issues), Major useful functions identified by the participants include: -capability of giving an overview of the knowledge of a domain, -capability of knowledge fusion (showing large span of knowledge), -knowledge discovery (or serendipity) through chains of topics.
To respond to the negative comments, we created a straightforward, best-practice style tutorial explaining how to operate the graphical topic map.To solve the cluttered display problem, we added middle layer topics (subcategory labels) to reduce the number of associations that are directly made to some heavily associated topics, and users are suggested to switch to the text topic map to display a large number of associated topics linearly.We also added quotes (one to three sentences) to the references to provide users with more information about topics when the full content of copyrighted information resources could not be made accessible on the Web.

Summative Evaluation 1
The eight undergraduate and graduate students evaluated the usefulness and usability of the text and graphical topic maps.All the eight participants found the topic maps useful.Positive comments on the usefulness of the topic maps are as follows: (1) Users found the text/graphical topic map useful in facilitating learning about topics related to oil spills and in finding a research topic.Users also found the quotes in the references about topics useful.
(2) The graphical topic map was useful in making connections to topics that users had not thought about before.It helped users discover new topics related to the topic they were researching.
(3) The text topic map was useful when the graphical topic map was overwhelming since it put information in a linear view and makes it much easier to read and understand.Some users found the text topic map more useful and easier to navigate than the graphical topic map.
(4) Visual and kinetic learners found that being able to see connections of topics helped them understand things much better than if they were reading them on a page linearly.
Negative comments on the usability of the graphical topic map are as follows: (1) Users had to learn to master navigation.
(2) Relationships do not show directions, which can be confusing in understanding the relationship between two topics.(Note that this is a limitation of the topic map standard.)(3) The cluttered display of heavily associated topics prevented users reading and locating the topics of their choice.Users found it tedious to collapse many topics.(4) Some users wanted bigger font size.Some liked the color combinations, and some did not.

Summative Evaluation 2
The five professors evaluated the usefulness and usability of the text and graphical topic maps and were asked to suggest areas for improvement.All the five participants found the topic maps useful.The following are some positive comments about the usefulness of the topic map: (1) The topic map has the added advantages of being easy to browse and bringing together associations that may not be widely made otherwise.It would be particularly useful for independent research courses in which students are tasked with formulating a thesis and building an argument to support it.The map allows the user to follow a trail and, unlike many Web sites currently online, uses information from verified sources.(2) Especially useful is the way the topic map suggests topics and associations.
Researchers may not be familiar with some areas of study, so the linkages provide insights into other types and subjects of research and information.
(3) The topic map can be of great utility when undertaking interdisciplinary and multidisciplinary research.(4) The information resources could potentially help researchers to draw connections to form the basis for research projects that incorporates ideas from multiple disciplines.The following are some suggestions about improving the usability of the topic map: (1) The graphical topic map is difficult to navigate and utilize because of information overload.Consider creating a reduced/simplified version of the topic map for the general public or novice users, or consider setting it up in modules that could be accessed one at a time.
(2) Two evaluators experienced technological problems installing and initiating Java on their browsers.They suggested to consider other topic maps visualization software programs that require low technology demand.

Lessons Learned
By comparing the outcomes of the formative evaluation and two summative evaluations, we found that appropriate economic compensation is necessary to help encourage the participants to conquer the learning curve in understanding how to operate the complex knowledge organization tool.We also found it wrong to assume that welleducated participants could naturally solve technological issues, such as installing Java.It is necessary to require low technology at the user's end in using a complex tool.
The two summative evaluation studies are more effective than the formative evaluation They identified the same, and even more, strengths and weaknesses of the topic maps in terms of usefulness and usability, although fewer number of participants were involved.Methodologically, more participants does not naturally lead to better results in a qualitative study.A thorough study of fewer participants can be more effective in figuring out the usefulness and usability issues in a complex knowledge organization tool.

Conclusion and Discussion
A formative evaluation with 47 high school students, a summative evaluation with eight undergraduate and graduate students at LSU, and a summative evaluation with five professors at LSU were conducted to evaluate the usefulness and usability of the Oil Spill Topic Map, which is composed of a text topic map and a graphical topic map.The two topic maps complement each other.The text topic map is good for searching for a topic and presenting the topic's heavily associated topics, relationships, and references in a linear way.The graphical topic map is good for browsing and visualizing a topic's associated topics and relationships by following a trail.
The oil spill topic maps are useful knowledge organization tools in learning the impacts of the Gulf of Mexico Oil Spill Incident.The topic maps were found useful for knowledge fusion (by making connections to the topics that users had not thought about before), which facilitates knowledge discovery (by helping users discover new topics related to the topics they were researching).The topic maps can be of great utility when undertaking interdisciplinary and multidisciplinary research because information resources in multiple disciplines were collected to construct the topic maps.
Several issues hurt the usefulness of the graphical topic map.The relationships between two topics in the graphical topic map are not directional, which can prevent users from understanding the relationships correctly.This is a flaw in the topic maps standard and the topic map creation software programs Wandora and Ontopia.Copyright protection of some of the information resources that were used to construct the topic maps prevents users access them directly from the Web.Presenting short quotes in the references related to topics helps to satisfy user's information needs.
There are some technical prerequisites for using topic maps as self-regulated learning tools.The particular implementation of the topic maps using Wandora and Ontopia requires Java to run on a browser at user's end.Users have to learn to operate the graphical topic map.Either face-to-face training or interactive tutorials on how to use the topic maps needs to be provided to the users so that they can be self-regulated learners.
The text topic map was found easier to use due to displaying topics, relationships and references in a linear view, which probably fits user's mental model of Web search engines.The graphical topic map has some usability issues.Major usability problems are information overload and cluttered display of topics when displaying large number of topics and their associated topics at once.
In the future, we plan to experiment with topic map creation software which can control the number of topics and their associated topics to be displayed or highlighted.This may solve the information overload problem.Topic map creation software programs that not require Java on a browser at user's end are to be found and tested.We also plan to create a simplified version of the topic maps for novice users to get familiar with this type of knowledge organization and self-regulated learning tool before being overwhelmed by the full complexity of the large topic maps.
There are several limitations in this study.First, we evaluated only one topic map system in a single domain.Second, we did not compare the oil spill topic map with other similar knowledge organization tools such as concept maps (since a concept map was not constructed, and it would be expensive to construct a large oil spill concept map).Third, we did not test the topic map with general users who are not high school students and academics.These factors may limit the generality of our findings.

Figure 1 .
Figure 1.Text topic map for topic "cleanup efforts"

Table 1 .
Responses to Q1: How useful is the topic map in facilitating the understanding of the topic(s) you have explored?