To cheat or not to cheat? A trial of the JISC Plagiarism Detection Service with biological sciences students

In the UK, there is great concern about the perceived increase in plagiarized work being submitted by students in higher educations. Although there is much debate, the reasons for the perceived change are not completely clear. Here we present the results of a 2‐year trial of the JISC Plagiarism Detection Service (PDS) involving hundreds of students. The effectiveness of the service in detecting plagiarized material and in acting as a deterrent are discussed. Although an increased number of cases of plagiarism were detected during the trial, the relative contributions of the electronic detection system and increased staff awareness remain unknown.


Introduction
Plagiarism is said to be on the increase in higher education (Park, 2003;Kilner, 2004), although in truth, little data is available to properly support this claim, in particular for UK university biology courses. The risk factors for plagiarism are believed to be well known: the vastly increased availability of online information, tempting students to cut and paste (with or without subtle rewording), poor training in citation skills, poor supervision, academically weak students, real or perceived pressure of deadlines and a previous history of plagiarism. The pressures to plagiarize increase with widening access (mature students, outside pressure of part time work, deadline pressure) (Bennett, 2005). As levels of awareness of the problem have increased among academic staff, so many have become more assiduous in identifying plagiarism, which has contributed to the rise in the number of cases recorded in the School of Biological Sciences at the University of Leicester.
One response to this perceived increase has been the use of digital or automatic plagiarism detection methods (Culwin & Lancaster, 2000). Submission of work by electronic means, such as email and virtual learning environments, has allowed instructors to easily copy and paste phrases from student work into search engines like Google to search for close matches. This method can sometimes work but is time consuming, especially if the student has plagiarized from multiple sources, and is a very 'hit or miss' approach in terms of which phrases are chosen. If there was a long lead-time to prepare the piece of work, sites they may have copied from may have been updated and no longer provide a match. Furthermore, search engines such as Google cannot be used to detect collusion between students, or across cohorts of students. The JISC Plagiarism Detection Service (http://submit.ac.uk), based on the Turnitin system (http://www.turnitin.com), archives source material daily, thus avoiding this problem and will show matches to multiple sources in a single report.
As in many institutions, in the University of Leicester School of Biological Sciences, students undergo training through formal lectures and a tutorial exercise to teach them about plagiarism (Willmott & Harrison, 2003). In the latter, students are given a paragraph of text from a paper and a selection of paragraphs that range from a direct copy of the original, quotations including the paragraph, through paraphrasing to complete rewording. They are asked to comment on whether each statement is plagiarism or not and encouraged to discuss their results. The Turnitin system is demonstrated to show how matching text can be easily detected. Training is certainly of the utmost importance in avoiding plagiarism. The success of this training programme has been indicated by recent surveys of students, undertaken as part of the year questionnaire feedback, where over 97% of students indicate that they understand the nature of plagiarism and the School's policies. Despite this, the most frequently encountered claim by the small proportion that attend disciplinary hearings is 'I didn't realize this was plagiarism'. On the other side of the coin, deterrence is also important as the other component of the 'carrot and stick' policy that is most likely to minimize the incidence. This has been supported by numerical evidence that the policy of scanning all second and third year coursework seems to have reduced the incidence of plagiarism during the current year.

Method
The JISC Plagiarism Detection Service (JISC PDS) is a nationally organized system that compares submitted text against a database. The database contains information from the Internet, which is updated daily by automated web robots, the output from a publisher's database of peer-review journals, and information from essay mills and other cheat sites. Student papers are also checked against one another and against all other papers in the database. Papers are never deleted from the database, which therefore can be used for checking for collusion between years as well as within years. The JISC PDS is a web-based system that can be linked to virtual learning environments (VLEs) such as WebCT or Blackboard to provide a seamless submission and review process. An originality report is generated showing the matching text highlighted in colour, with each different source shown in a different colour. The highlighted text is hyper-linked to the original source. A side-by-side view enables the source and To cheat or not to cheat? 435 matching text to be compared directly, with matching text highlighted in both the source and the student paper. A percentage score is given to represent the number of words found that match another source in relation to the whole word count for the student paper. Instructors still need to view each report and analyse the sources used by the student, for example, some may be legitimate quotations and often matches to other student papers are where the question has been left in the submitted documents and are therefore common pieces of text between all papers. Figure 1. Screenshot of JISC PDS Originality report on specially created sample essay. The top panel shows information about the author, number of words in the student essay and percentage matching score. The bottom left frame shows student work with matching text highlighted in colour and surrounded by a box, each source has a different colour and is numbered. The right frame shows the list of matching sources found and their origin in corresponding colours and numbered. Figure 2. Screenshot of JISC originality report on specially created sample essay showing a direct comparison between student essay in the left hand panel and one of the detected source texts on the right The data used in our preliminary trial of the JISC PDS system in the School of Biological Sciences at the University of Leicester were mostly undergraduate essays collected during the previous academic year. Individual pieces of work were anonymized and submitted for plagiarism scanning. This work had already been marked and, in the work of one module, the marker had detected plagiarism. The JISC PDS found all but one of the cases, confirmed one suspected case and found two further undetected cases. This pilot study demonstrated that material copied from the web was not time sensitive, since matching text from archived Internet sites was detected. This encouraged us to attempt a, more extensive trial using data collected directly from students during the second semester of 2004-2005. Nine module convenors were recruited to use the system on coursework during their modules. Some convenors collected work from more than one module, so a total of 12 undergraduate (first, second and third year) and 2 postgraduate modules were involved in the study (plus a final year report). In all, 513 pieces of work were collected from approximately 465 students (some students were on multiple modules included in the trial). The majority of the work collected were essays, though the study included some practical reports, mini reviews, assessed online discussion board postings, final year projects and tutorial essays written under open book conditions live online.
Student work was collected online through Blackboard. Students completed an online submission form, including School's standard agreement statement regarding plagiarism, and attached their paper as a word or other text document. These papers were automatically transferred to the JISC PDS for analysis. Results were viewed and interpreted by the module convenor. Any cases of plagiarism detected were reported as usual through the School's established disciplinary system.
Results for the study were collected by means of a questionnaire for each convenor with answers taken during face-to-face interviews or by email correspondence. Data from the JISC PDS are only available to each instructor and cannot usually be shared.

Results
In our initial small-scale trial, the detection rate was 2.06% (n = 2 out of 97 submissions tested), representing 18% of the total cases of plagiarism detected that year. In the first proper year of use, the detection rate rose to 2.73% (n = 14 out of 513 submissions tested), representing 40% of the total number of cases detected. In the second year of use, the both the total number of cases and the detection rate fell to 0.94% (n = 10 out of 1060 submissions tested), representing 71% of the total number of cases detected. These results suggest that the JISC PDS is an effective means of both detecting and deterring plagiarism, although the difference in the detection rate was not statistically significant in a one-way chi-square goodness of fit test.
The JISC PDS does not always detect copying from peer-reviewed subscription journal articles. Most publishers use password protection for their subscription journals, so they are not freely available online and therefore hidden from the JISC PDS web robots. However, the growing database of student work within the JISC PDS can provide a match for copying from such a source: for example, a piece of work conducted in semester two of 2003-2004 was submitted to the JISC PDS as part of the original trial. The same assignment was repeated in 2004-2005 with a new cohort of students and a match from that work was found to the previous year's work. Close inspection revealed that both students had copied from a third source, a review article not accessible to the JISC PDS.
Generally, plagiarism from second year students consisted predominantly of cutting and pasting from websites, while third year students were more likely to include unattributed sections of peer-reviewed articles available online.
Convenors commented that the JISC PDS appeared to prevent plagiarism. In one module, the convenor had demonstrated the system, showing the coloured matching text and side-by-side view in an early lecture in his module. Students were told that this system would be used for all their work. Three timed tutorial essays were written online under open book and 'open computing' conditions. This meant that students were faced with being able to copy and paste during the tutorial. However, not a single case of plagiarism was detected on this module. Similarly, in another module the convenor informed the students that on-line submission was being used specifically to enable detection of plagiarism, again, no positive cases were identified.

Discussion
Recent feedback from the School's Staff-Student Committee has been very supportive of stamping out plagiarism, indeed the student representatives actually felt that punishments were currently too lenient. Our trial has shown that the JISC PDS is an effective tool for a first scan for plagiarism of work submitted by biology students. Indeed, the outcome of this trial was such that School policy has now been changed, with agreement from the student representatives, to require all work to be submitted electronically and checked for plagiarism. Students will be alerted to this policy and given a visual demonstration of the JISC PDS as part of their training on plagiarism.
However, the JICS PDS system is not without its problems. As indicated above, it currently does not include peer-reviewed journal articles of relevance to biological sciences-though more open source, the growing database of student work, selfarchiving and poor copyright compliance by authors and others in the electronic environment are making this of less significance.
The rates of plagiarism detected in our trial are not as high as might be expected from a self-selected group of tutors keen to detect it, nor as high as some literature reports suggest (Weinstein & Dobkin, 2002;Kilner, 2004;Marsden et al., 2005). However, during the year of the trial, disciplinary hearings for plagiarism rose dramatically (from 11 to 35 cases). It is far from clear that all of this increase results directly from the use of the JISC PDS since the relative contribution made by increased staff awareness is unknown and the proportion of cases detected using the scanning system remained relatively constant (2.06% in 2003-2004 and 2.73% in 2004-2005). What does seem likely is that the continued high profile use of the JISC PDS on all coursework during 2005-2006 has led to a decrease in the proportion of cases detected (0.94%).
Increased detection of plagiarism due to software scanning raises issues of institutional procedures for dealing with it once detected. Thresholds are difficult to define absolutely-when does stupidity become cheating? Consistent advice needs to be readily available to staff as to whether an assignment is plagiarized sufficiently to warrant a penalty. Feedback to students needs to be timely. Some literature is starting to look at this now (Larkham & Manns, 2002;Sims, 2002;Park, 2004). The report of the Plagiarism: Prevention, Practice and Policy Conference 2004 has this to say: There is a clear difference between cheating and plagiarism. The latter is 'mostly a matter of ignorance. We are too often punishing students for not knowing what we are there to teach them'. However, plagiarism and the coverage it has received offers us the opportunity for us to rethink how we teach. Academic discourse is founded on the use of others' work and 'others' voices' in one's own work. We need to enable students to marshal legitimately other's work in their own work, one of the key skills that all students should acquire. (www.jisc.ac.uk/index.cfm?name=plagiarism_conf_news_290604) Although we have not detected the high rates of plagiarism reported by other authors, we are sufficiently encouraged by our trial of the JISC PDS to incorporate it into our standard procedures for submission of student work. Whether we will be as successful in meeting the challenge of where plagiarism begins and ends and what to do in response to an increasingly diverse student body motivated by many different demands remains to be seen.